FreeBSD lean ZFS root installation to a whole disk

Boot the FreeBSD 11.2-RELEASE installer, the latest that seems to work with zfsboot, choose Live CD and login as root.

kldload zfs
sysctl vfs.zfs.min_auto_ashift=13
zpool create -O compression=lz4 -O recordsize=8K -O logbias=throughput -O atime=off -m / -R /mnt zroot ada0
(zpool set bootfs=zroot zroot)
cd /mnt
dhclient em0
fetch -o - ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/11.2-RELEASE/base.txz | tar xSf -
fetch -o - ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/11.2-RELEASE/kernel.txz | tar xSf -
cd boot
echo 'zfs_load="YES"' >> loader.conf
sysctl kern.geom.debugflags=16
dd if=zfsboot of=/dev/ada0 count=1
dd if=zfsboot of=/dev/ada0 iseek=1 oseek=1024
shutdown -r now

Slackware

Slackware is the most UNIX-like of Linux distributions. Alpine is also a good option. Here is how the basics are installed without X for simultaneous booting with both BIOS and UEFI.

Boot the installer from
http://www.slackware.com/~alien/slackboot/mini/
or
ftp://ftp.slackware.com/pub/slackware/slackware64-14.2/usb-and-pxe-installers/usbboot.img

Partitioning (these sector sizes are for a 20 GiB virtual disk)

# parted
unit s
mklabel msdos
mkpart primary fat32 2048s 204799s
mkpart primary linux-swap 204800s 2097151s
mkpart primary ext2 2097152s 41943039s
toggle 1 boot
toggle 1 esp
quit
# setup

Install from FTP
ftp://ftp.slackware.com/pub/slackware/slackware64-14.2/

Select following package series:
A - Base system
AP - Additional utilities
D - Compilers
L - Libraries
N - Networking utilities

Install LILO to MBR

Uncomment a mirror from /etc/slackpkg/mirrors

# slackpkg update
# slackpkg upgrade-all

Fetch the Latest Stable Kernel from https://www.kernel.org/
Make use of ccache

ccache-swig -M 5G
ln -s /usr/bin/ccache-swig /usr/local/bin/cc
ln -s /usr/bin/ccache-swig /usr/local/bin/c++
ln -s /usr/bin/ccache-swig /usr/local/bin/c89
ln -s /usr/bin/ccache-swig /usr/local/bin/c99
ln -s /usr/bin/ccache-swig /usr/local/bin/gcc
ln -s /usr/bin/ccache-swig /usr/local/bin/g++
ln -s /usr/bin/ccache-swig /usr/local/bin/clang
ln -s /usr/bin/ccache-swig /usr/local/bin/clang++
ln -s /usr/bin/ccache-swig /usr/local/bin/swig
watch -d ccache-swig -s

Build the new kernel with

make defconfig
make menuconfig
#CONFIG_HAVE_KERNEL_XZ=y
#CONFIG_KERNEL_XZ=y
#CONFIG_MODULE_COMPRESS=y
#CONFIG_MODULE_COMPRESS_XZ=y
#CONFIG_EFI_STUB=y
#CONFIG_EFI_STUB_MIXED=y
#CONFIG_CMDLINE_BOOL=y
#CONFIG_CMDLINE="root=/dev/sda3"
#CONFIG_LOGO=n # not set
time make bzImage

Build kernel modules when neccessary

Install the new kernel allowing both UEFI or BIOS booting on same system

mkfs.fat -F32 /dev/sda1 # ESP
mount /dev/sda1 /boot
echo "/dev/sda1 /boot fat defaults 0 2" >> /etc/fstab
cp arch/x86/boot/bzImage /boot/vmlinuz
mkdir -p /boot/efi/boot
cp /boot/vmlinuz /boot/efi/boot/bootx64.efi
lilo -v # update lilo from /etc/lilo.conf to boot the new kernel

Virtual console autologin /etc/inittab

c1:12345:respawn:/sbin/agetty --noclear 38400 -a root tty1 linux

Disable screen blanking with /etc/rc.d/rc.local

# Disable screen blanking
/bin/setterm -blank 0 -powersave off -powerdown 0

Faster reboot / shutdown procedure /etc/rc.d/rc.6
Search for sleep and adjust to a smaller value where necessary.

ATA Security Erase

It is best to run these commands under tmux or screen to make sure that the erase is not interrupted.

FreeBSD

Check security status

camcontrol security adaX

If the drive security is frozen reconnect the power cable or suspend to ram:

zzz

Execute enhanced SECURITY ERASE UNIT

camcontrol security adaX -U user -s Eins
date; time camcontrol security adaX -U user -h Eins

OR issue the SCSI SANITIZE command CRYPTO SCRAMBLE EXT for instant secure erase (should be faster for rotating disks as only the crypto key is changed rendering the data unreadable) The -r option is used to report status.

date; time camcontrol sanitize adaX -U -a crypto
camcontrol sanitize adaX -U -r

Linux

Check security status

hdparm -I /dev/sdX

If the drive security is frozen reconnect the power cable or suspend to ram:

echo -n mem > /sys/power/state

Execute enhanced SECURITY ERASE UNIT

hdparm --security-set-pass Eins /dev/sdX
date; time hdparm --security-erase-enhanced Eins /dev/sdX

Failed disks

Model	Enabled	Disabled	Hours
WD30EZRX	2011-11	2017-03	44289

Install Windows 10 LTS without extra partitions

BIOS systems with MBR require only one NTFS partition to boot Windows.

UEFI systems with GPT requires two. A FAT(32) system partition for EFI boot files and a NTFS partition for Windows.

First make a bootable USB drive of the latest LTS Windows. Then boot and press shift-F10 when the installer loads to start the command prompt.

The instructions for an UEFI system (GPT):

diskpart
sel dis 0
cle
con gpt
cre par efi size=31
for quick fs=fat
ass letter=s
cre par pri
for quick
ass
exi
cd /d d:\sources
dism /apply-image /imagefile:install.wim /index:1 /compact /applydir:c:\
bcdboot c:\windows /s s: /f uefi
exit

Then exit the installer and boot into your new two-partition Windows installation.

Smallest possible EFI partition size seems to be around 36 MB for FAT32 and 25 MB for FAT(16) to fit all the boot files (only 4.5 MB if files are trimmed to minimum). Diskpart automatically aligns the created partitions to 1 MB boundaries. The image is applied in compressed compact mode and the Windows Recovery Environment is automatically installed to C:\Recovery\WindowsRE\Winre.wim.

The instructions for a legacy BIOS system (MBR):

diskpart
sel dis 0
cle
cre par pri
act
for quick
ass
exi
cd /d d:\sources
dism /apply-image /imagefile:install.wim /index:1 /compact /applydir:c:\
bcdboot c:\windows /s c: /f bios
exit

Then exit the installer and boot into your new one-partition Windows installation.

When using a NVMe SSD enable the faster Direct Access Storage mode with format /DAX c:.

The best of both worlds is to use MBR with EFI system partition. This setup allows booting with both BIOS and UEFI modes:

diskpart
sel dis 0
cle
cre par pri size=63
act
for quick fs=fat32
ass letter=s
cre par pri
for quick
ass
exi
cd /d d:\sources
dism /apply-image /imagefile:install.wim /index:1 /compact /applydir:c:\
bcdboot c:\windows /s s: /f all
diskpart
sel dis 0
sel par 1
set id=ef
exi
exit

Then exit the installer and boot into your new MBR-EFI Windows installation.

How to split Windows 10 installer to FAT32 and NTFS partitions (bootfiles and wimfiles) allowing larger than 4 GiB install.wim/esd on UEFI systems as UEFI needs a FAT partition to boot

copy boot.sdi to \sources
edit \boot\bcd and \efi\microsoft\boot\bcd
bcdedit /store bcd {default} device ramdisk=[C:]\sources\boot.wim,{ramdiskoptions}
bcdedit /store bcd {default} oadevice ramdisk=[C:]\sources\boot.wim,{ramdiskoptions}
bcdedit /store bcd /set {ramdiskoptions} ramdisksdidevice partition=C:
bcdedit /store bcd /set {ramdiskoptions} ramdisksdipath \sources\boot.sdi

docs.microsoft.com

Restore MacBook Air SSD with OS X 10.11 El Capitan

Backup /Users/ directory (with tar) and make a list of useful installed /Applications/.

Download El Capitan from App Store.

Format at least 8 GB usb drive to OS X Extended file system and create install media:

cd "/Applications/Install OS X El Capitan.app/Contents/Resources"
./createinstallmedia --volume /Volumes/new --applicationpath "/Applications/Install OS X El Capitan.app"

Optionally create a slim DMG from that install media:

cd "/Volumes/Install OS X El Capitan"
rm -R .Trashes/
hdiutil create -nospotlight -srcfolder "/Volumes/Install OS X El Capitan" -format UDBZ /Users/swegen/Desktop/ElCapitan.dmg

Download latest Ubuntu ISO and dd it to another at least 2 GB usb drive.

Boot to Ubuntu with option key pressed at boot. Put device to sleep by closing the lid and wake it up to unfreeze the SSD.

Open terminal and gain root rights with:

sudo passwd
su

Restore SSD memory cells to factory condition with ATA secure erase command (takes about six seconds):

hdparm -I /dev/sda
hdparm --user-master u --security-set-pass Eins /dev/sda
time hdparm --user-master u --security-erase-enhanced Eins /dev/sda

Optionally clear SMC and NVRAM to start install afresh.

Boot with El Capitan usb install media and start terminal to create 1MB aligned OS partition while preserving default EFI partition starting location and size to allow for future firmware updates (sizes are calculated for 250 GB SSD):

gpt create /dev/disk0
gpt add -b40 -s409600 -tefi /dev/disk0
gpt label -i1 -l "EFI System Partition" /dev/disk0
gpt add -b411648 -s488552448 -thfs /dev/disk0
gpt label -i2 -lSSD /dev/disk0
newfs_hfs -s -J -v SSD /dev/disk0s2
diskutil mount /dev/disk0s2
cd /Volumes/SSD
rm -R .Trashes/

Exit terminal and install El Capitan.

Installer replaces basic OS partition with unnecessary Core Storage volume manager. After installation is complete Remove Core Storage layer from GPT:

diskutil cs list
diskutil cs revert %lvuid%

Reboot and check again with above list command that it worked. Then restore your backups. Final partition layout (efi, hfs and recovery) is:

# gpt show /dev/disk0
      start       size  index  contents
          0          1         PMBR
          1          1         Pri GPT header
          2         32         Pri GPT table
         34          6         
         40     409600      1  GPT part - C12A7328-F81F-11D2-BA4B-00A0C93EC93B
     409640       2008         
     411648  488552448      2  GPT part - 48465300-0000-11AA-AA11-00306543ECAC
  488964096    1269536      3  GPT part - 426F6F74-0000-11AA-AA11-00306543ECAC
  490233632       1087         
  490234719         32         Sec GPT table
  490234751          1         Sec GPT header

iOS jailbreak checklist

• Update iTunes
• Backup iDevice with iTunes
• Backup iBooks and ringtones manually
• Keep iDevice connected to computer and go into dfu mode (home and sleep buttons until black screen)
• Update iDevice in dfu mode
• Do not restore backup yet
• Jailbreak iDevice
• Run Cydia once to make space to OS partition
• Restore data from backup using iTunes
• Add Cydia tweaks

If an app crashes on launch, you can try to fix it with Link Identity Editor:

ldid -S Program

Windows CMD

Change owner to administrator

takeown /f FILE/DIR /a /r /d y

Change owner to current user

takeown /f FILE/DIR /r /d y

Change owner to given user

icacls FILE/DIR /setowner USER /t

Give full permissions to given user

icacls FILE/DIR /grant:r USER:f /t

*nix tips

Find all unique ip addresses from webserver logs:

# awk '{print $1}' access.log | sort | uniq
# zcat access.log.gz | awk '{print $1}' | sort | uniq

Copy files using tar (this preserves metadata. add 'p' option to also preserve ownership data):

# cd source-dir
# tar cSf - . | tar xSf - -C target-dir

Count total size in bytes of files in current directory tree (different from disk usage):

# find . -type f -ls | awk '{total += $7} END {print total}'

ZFS with 4k sectors on Debian GNU/kFreeBSD

Debian GNU/kFreeBSD is an operating system for IA-32 and x86-64 computer architectures. It is a distribution of GNU with Debian package management APT and the kernel of FreeBSD. The 'k' in kFreeBSD is an abbreviation for kernel of, and reflects the fact that only the kernel of the complete FreeBSD operating system is used.

Current stable distribution (Squeeze) provides zpool version 14.
Current testing distribution (Wheezy) provides zpool version 15.
FreeBSD 9.0-BETA2 provides zpool version 28.

For some reason the current kFreeBSD ZFS implementation creates zpools with property whole_disk=0 even when creating a zpool with whole device vdev. To remedy this, one can use FreeBSD 9.0-BETA2 to create a zpool with whole_disk=1 and with an older zpool version. Then export it and import to Debian.

# zpool create -o version=14 <pool> <vdev>

When creating raidz zpools to be used with 4k advanced format hard drives,  it is best to use raidz1 with 3, 5 or 9 disks, raidz2 with 6 or 10 disks, and raidz3 with 11 or 19 disks.

ZFS aligns zpools with hard disk logical sector size. All current hard drives (including the 4k advanced format ones) report 512 bytes as the logical sector size (ashift=9, 2⁹ = 512). To make ZFS align zpools with 4k sectors the ashift value has to be ashift=12, 2¹² = 4096.

I have created this way a 12 TB NAS using six Western Digital 3 TB disks in raidz2.

Install zfsutils which provides zpool, zfs and zdb commands, and freebsd-geom which provides gnop (through geom nop).

# aptitude install zfsutils
# aptitude install freebsd-geom

Find out the device names in your system.

# atacontrol list

or if using FreeBSD-9 kernel:
# camcontrol devlist

Create a NOP device for simulating 4k sector.

# geom nop create -v -S 4096 ad6

Create a zfs pool and export it.

# zpool create datapool ad6.nop
# zpool export datapool

Destroy the NOP device since it's only needed to set ashift=12 when creating the pool.

# geom nop destroy -v ad6.nop

Import back the pool.

# zpool import datapool

Confirm that the ashift value is 12.

# zdb datapool | grep ashift

(For testing the zpool creation with files one can use):

# dd if=/dev/zero of=<file> bs=1G seek=4096 count=0
# zpool create <pool> `mdconfig -f <file> -S 4096`

The zdb command seems to have a bug even still in Solaris 11 Express, when using it with an older zpool version. Update: this is fixed in Solaris 11 EA.

# zpool create -o version=14 <pool> <vdev>
# zdb <pool>
Assertion failed: mp->initialized == B_TRUE, file ../common/kernel.c, line 127,
function mutex_enter

How to mount an ext2 filesystem with FreeBSD?

# kldload -v ext2fs
# mount -t ext2fs <device> <mountpoint>

Solaris

Solaris OS x86-64 and its derivatives seem to mainly use 32-bit applications although the kernel is 64-bit.

Unlike on x86_64, 64-bit SPARC code requires more memory for no performance gain over 32-bit, so it is best only used when the 32-bit address space is insufficient. SPARC userspace is therefore 32-bit by default, even when the hardware and kernel are 64-bit.

"By default, perl-5.6.0 (or later) is compiled as a 32-bit application with largefile and long-long support.", and the Sun docs say this: "The Solaris version of Perl was compiled to include system malloc, 64-bit integer and large file support. In addition, appropriate patches have been applied."

You should have what they promised. A 32-bit application that can do 64-bit integer arithmetic and will not have problems with files over 2 GB in size. But it internally uses 32 bit pointers and so should be linked to the 32-bit library.

Solaris 11 Express 2010.11 64-bit
Perl 5.8.4 ELF 32-bit LSB executable 80386, built for i86pc-solaris-64int
PHP 5.2.12 32-bit

NexentaCore 3.1 (Hardy 8.04/b134+) SunOS Release 5.11 Version NexentaOS_134f 64-bit
Perl 5.8.8 ELF 32-bit LSB executable, Intel 80386, built for i386pc-solaris2.11-thread-multi
PHP 5.2.4-build1 with Susoshin-Patch 0.9.6.2 (cli) ELF 32-bit LSB executable, Intel 80386

GNU/kFreeBSD debian 8.2.1-amd64
Perl 5.12.4 ELF 64-bit LSB executable, x86-64, built for x86_64-kfreebsd-gnu-thread-multi
PHP 5.3.8-1 with Susoshin-Patch (cli) ELF 64-bit LSB executable, x86-64

$large_number = 9223372036854775807;var_dump($large_number);                     // int(9223372036854775807)

This above php-script outputs int if php is compiled for 64-bit. If php is 32-bit it outputs float.
The only OS which passed this 64-bit test was Debian/kFreeBSD-amd64.


Some useful Solaris commands:

Find out the kernel architecture in use:

# isainfo -k

Find out a service state:

# svcs -a | grep <service name>

Enable / disable / restart / refresh a service:

# svcadm -v enable/disable/restart/refresh <service name>

Find out disk sizes:

# iostat -E

Find out device names:

# format

Install Apache, MySQL and PHP (default document root is at /var/apache2/2.2/htdocs/):

# pkg install amp
# svcadm enable mysql
# svcadm enable http

Create a large (20 TiB) sparse file and create a loop device to use with ZFS testing:

# dd if=/dev/zero of=<file> bs=1G seek=20480 count=0
# zpool create <pool> `lofiadm -a <file>`

boot.efi x64 patch for Mac OS X Lion and Snow Leopard

If the Apple computer you are using does not allow booting into 64-bit kernel, applying this patch will help.

• x64patch-10.6.8.tar.bz2 for Snow Leopard
• x64patch-10.7.3.tbz for Lion (updated)
• x64patch-10.8-dp1.tbz for Mountain Lion Developer Preview 1 (new)

It patches the boot.efi system file to allow x64 booting.
Should work for all Apple models.


Apple does not include 64-bit graphics drivers for some models.
Here are the Intel GMA X3100 64-bit kexts to use with 64-bit kernel
and a install script for:
MacBook3,1
MacBook4,1
MacBook4,2
MacBookAir1,1
AppleIntelGMAX3100-x64.tar.bz2

I cannot support these kernel extensions though, as I no longer have any of the above models. This forum has some info about getting the GMA X3100 to work. projectosx.com/forum/index.php?showtopic=763

Mac OS X Lion tips

These are random notes, will clean this up later...

How to Disable Journaling?

1. open Disk Utility
2. select desired partition from the list on left
3. while holding down option (alt) key, open file menu and select Disable Journaling

Kernel Flags in /Library/Preferences/SystemConfiguration/com.apple.Boot.plist
Kernel Flags

• -s (or hold down Command-S during boot) boot to single user mode
• -v (or hold down Command-V during boot) enable verbose boot
• -x (or hold down Shift during boot) boot in safe mode

Kernel Architectures

• i386
• x86_64

trying to disable journaling with disk utility on an unmounted filesystem resulted in a crash in lion 10.7?

How to set the password for root account?
In terminal: sudo passwd root

How to select boot device at startup?
Hold down the option (alt) key

How to disable Secure Virtual Memory?

sudo defaults write /Library/Preferences/com.apple.virtualMemory DisableEncryptedSwap -boolean yes

To check if swap is encrypted, run:
$ sysctl vm.swapusage


How to enable 64-bit Kernel and Extensions?
modify /System/Library/CoreServices/boot.efi
sudo systemsetup -setkernelbootarchitecture x86_64 (not neccessary if correctly modified)

How to prevent creation of .DS_Store files?

How to Disable remote control infrared receiver?

1. open System Preferences
2. open Security & Privacy
3. disable lock and enable the checkbox 

How to disable Location Services?

How to disable Guest Account and Sharing?

Create a standard (non-admin) account for normal use.

Disable automatic login and user list.


Utilities/Terminal: Disable unnecessary services by using the chart to the right with this command:sudo launchctl unload -w /System/Library/ and LaunchDaemons/com.apple.blued.plist.

http://www.macgeekery.com/tips/security/basic_mac_os_x_security
http://poller.se/2010/08/optimizing-mac-os-x-for-ssd-drives/
http://netkas.org/?p=189

Fallout New Vegas stats

 

Fallout 3 final stats

 

cheap 3TB

WD30EZRS w/64MB cache is cheaper to buy in an external enclosure.

3.0 TB is actually 3 × 1000⁴ / 1024⁴ = 2.7 TiB.

WD30EZRS has 5 860 533 168 logical 512 B sectors or 732 566 646 physical 4096 B sectors.

WD20EARS has 3 907 029 168 logical 512 B sectors or 488 378 646 physical 4096 B sectors.

HD204UI has 3 907 029 168 logical 512 B sectors or 488 378 646 physical 4096 B sectors.

Going to use this disk in my linux server as unpartitioned superfloppy (no need for mbr / gpt) xfs filesystem mounted at /media. That way I don't have to worry at all about the 4 kB alignment issue with these "Advanced Format" drives.

How to securely erase MBA SSD

1. boot linux from usb key using grub-efi
2. run 'hdparm -I /dev/sda'
3. see that the disk security is frozen
4. put mba to sleep
5. resume from sleep
6. run 'hdparm -I /dev/sda' again
7. see that the disk security is not frozen
8. follow instructions from tinyapps.org/docs/wipe_drives_hdparm.html