This article is the first of a four-part series on building your own NAS on FreeBSD. This series will cover:

1. Selecting a storage drive interface that meets your capacity and performance requirements both now and into the future.
2. Why it makes sense to build your own NAS using FreeBSD rather than installing a NAS distribution (even a FreeBSD-based one). We’ll also discuss which configuration and tuning settings are needed.
3. The nitty-gritty on sharing: configuring NFS, Samba, and iSCSI shares.
4. Software maintenance and monitoring your NAS. //

https://klarasystems.com/articles/part-2-tuning-your-freebsd-configuration-for-your-nas/

https://klarasystems.com/articles/part-3-building-your-own-freebsd-based-nas-with-zfs/

• In this example we upgrade XigmaNAS to version XigmaNAS-x64-embedded-11.2.0.4.5748.img.xz.

The Console Menu is not presented under SSH by default, but it can be called directly with the command /etc/rc.initial

The "write hole" effect can happen if a power failure occurs during the write. It happens in all the array types, including but not limited to RAID5, RAID6, and RAID1. In this case it is impossible to determine which of data blocks or parity blocks have been written to the disks and which have not. In this situation the parity data does not match to the rest of the data in the stripe. Also, you cannot determine with confidence which data is incorrect - parity or one of the data blocks.
Write hole in RAID5

"Write hole" is widely recognized to affect a RAID5, and most of the discussions of the "write hole" effect refer to RAID5. It is important to know that other array types are affected as well.

If the user data is not written completely, usually a filesystem corrects the errors during the reboot by replaying the transaction log. If a file system does not support journaling, the errors will still be corrected during the next consistency check (CHKDSK or fsck).

If the parity (in RAID5) or the mirror copy (in RAID1) is not written correctly, it would be unnoticed until one of the array member disks fails. If the disk fails, you need to replace the failed disk and start RAID rebuild. In this case one of the blocks would be recovered incorrectly. If a RAID recovery is needed because of a controller failure, a mismatch of parity doesn't matter.

A mismatch of parity or mirrored data can be recovered without user intervention, if at some later point a full stripe is written on a RAID5, or the same data block is written again in a RAID1. In such a case the old (incorrect) parity is not used, but new (correct) parity data would be calculated and then written. Also, new parity data would be written if you force the resynchronization of the array (this option is available for many RAID controllers and NAS). //

How to avoid the "write hole"?

In order to completely avoid the write hole, you need to provide write atomicity. We call the operations which cannot be interrupted in the middle of the process "atomic". The "atomic" operation is either fully completed or is not done at all. If the atomic operation is interrupted because of external reasons (e.g. a power failure), it is guaranteed that a system stays either in original or in final state.

In a system which consists of several independent devices, natural atomicity doesn't exist. Variance of mechanical hard drives characteristics and data bus particularities don't allow to provide required synchronization. In these cases, transactions are typically used. Transaction is a group of operations for which atomicity is provided artificially. However, expensive overhead is required to provide transaction atomicity. Hence, transactions are not used in RAIDs.

One more option to avoid a write hole id to use a ZFS which is a hybrid of a filesystem and a RAID. ZFS uses "copy-on-write" to provide write atomicity. However, this technology requires a special type of RAID (RAID-Z) which cannot be reduced to a combination of common RAID types (RAID 0, RAID 1, or RAID 5).

1. Create a new file, e.g.delrecycle.sh
   Enter the following content to delrecycle.sh:

     #!/bin/sh 
     find /mnt/[MountPoint]/.recycle/* -atime +[Age] -exec rm -rf '{}' \;
     find /mnt/[MountPoint]/.recycle/ -depth -type d -empty -exec rmdir {} \;

1. Adjust [MountPoint] (mount point of your share) and [Age] (age of the files in days which you want to erase) to your environment and needs.
2. Upload delrecycle.sh to your share.
3. Create a cronjob (System|Advanced|Cron) with root– privileges, which calls the command:
   sh /path/to/MountPoint/delrecycle.sh

I'm very pleased to announce the extension OneButtonInstaller (OBI) which is meant as a centralized extension manager and an attempt to get a common interface for hassle-free installing of extensions to XigmaNAS systems without the need to work with system console or shell CLI.

The aim to create this extension was the reason that many other systems provide such an interface for the installation of extensions, add-ons and plug-ins already (FreeNAS, OpenMediaVault, pfSense, etc ...). On the basis of the Push Button Installer system I decided to name this extension OneButtonInstaller even if it don't compete in no way with this much more complex installer system.

https://github.com/crestAT/nas4free-onebuttoninstaller

For the lighttpd WebGUI web server, follow the below steps:
In System > Advanced > rc.conf,

• set lighttpd_certpem_webgui to NO. This will tell the lighttpd WebGUI instance to ignore the certificate shown in the WebGUI.
• set lighttpd_certpem to the full qualified file name of your certificate, this should be a permanent location which you can update via ssh.
  Whenever the certificate gets updated, the lighttpd service must be restarted as well.

For the lighttpd websrv web server, follow the below steps:
In System > Advanced > rc.conf,

• set websrv_certpem_webgui to NO. This will tell the lighttpd websrv instance to ignore the certificate shown in the WebGUI.
• set websrv_certpem to the full qualified file name of your certificate, this should be a permanent location which you can update via ssh.
  Whenever the certificate gets updated. the websrv service must be restarted as well.

I wanted to use acme.sh (https://github.com/Neilpang/acme.sh) to generate my internal SSL certificates with Let's Encrypt using the ACME protocol. I also wanted to automate the renewal said certs and use them with the web UI (lighttpd) and with the syncthing UI on my embedded N4F system. I've summarized my method below. The rough procedure is as follows:

• Installing packages on embedded system
• Meeting prereqs for acme.sh
• Running acme.sh and scheduling with cron
• Importing the certs for the web UI and syncthing

This is step-by-step guide for XigmaNAS embedded upgrade using a virtual machine.

By following this guide, you can prepare a new bootable USB stick with already restored XigmaNAS configuration. Such a USB stick can be useful when replacing an old, worn-out one, or when moving between major FreeBSD versions, for example, from 11th to 12th. The new bootable USB stick allows you to upgrade with minimal server downtime, reducing the entire procedure to a simple replacement of sticks in a USB socket.

Part 1 contains some useful information that you need to get ready.
Part 2 describes the sequence of steps to create XigmaNAS embedded stick with default server settings.
Part 3 introduces additional steps to get XigmaNAS embedded stick with actual server configuration.

Olivier Cochard-Labbé, the original founder of FreeNAS, shares with us a presentation outlining the history and beginnings of what would become the world's most popular storage operating system.

XigmaNAS Full RootOnZFS features and abilities:

• Can be installed on one or more drives.
  • Firmware Upgrades from the WebGUI with .TGZ official files available at SF.
• Can be installed on virtually all ZFS raid levels for desired redundancy.
• Support for UEFI capable hardware and virtualized environments.
• Support for Boot Environments at OS level and from boot loader time for convenience.
• Support for System OS compression, Installation almost halved in space to save SSD space.
• Previous Boot Environment will be left intact after Firmware upgrades.
• User ability to rollback to a previous working Boot Environment from OS or at boot time at any time.
• User can make new Boot Environments and reboot into it to make system wide changes for convenience.
• User can delete, rename, clone, mount etc etc. Boot Environments.
• User can add, edit, modify to XigmaNAS Full with peace of mind thanks to Boot Environments/Snapshots.

Recently the requirement came up to take a KVM based virtual machine and move it over to a VirtualBox image. Which turned out to be a fairly simple endeavour, and was fairly painless. The longest part was transferring over the 40 GB image from one machine to the other where the conversion could take place. The machine the image was coming from was only on a 100 Mbit/sec connection so that took a good hour.

Converting from KVM to VirtualBox for a FreeBSD image was pretty simple, the VBoxManage command has a convertdd command that allows you to convert from raw disk .img format to .vdi format.

VBoxManage convertdd KVM-image.img VB-image.vdi

After this, unfortunately, there is no way to to automatically convert over the settings that the virtual machine had, such as the network cards, the memory allocations and hard drive settings. You will have to go to VirtualBox and create a new virtual machine and replicate all of the settings. Once that is done make sure to select the same type of disk controller (SATA or IDE) so that the drive will hopefully be assigned the same name in the device tree so that you don't need to alter your /etc/fstab.

zfs destroy -rv dataset_or_vol_name@%

test first:

   zfs destroy -rnv dataset_or_vol_name@%

will also indicate how much data will be freed

XigmaNAS is an embedded Open Source NAS (Network-Attached Storage) distribution based on the latest FreeBSD 11-2 Releases.

It supports sharing across multiple operating systems, including Windows, Apple, and UNIX-like systems. XigmaNAS is easy to set up in most home and enterprise environments and will allow you to manage and share large amounts of data easily across your network. XigmaNAS also incorporates many different streaming features for sharing your multimedia with other devices on your network.

XigmaNAS includes ZFS v5000 (Feature Flags) (RAID-Z, RAIDZ-2 & RAIDZ-3) Software RAID (0,1,5), JBOD, Disk Encryption, S.M.A.R.T. / Email Reports, includes the following protocols: CIFS/SMB (Samba), Active Directory Domain Controller (Samba), FTP, NFS, TFTP, AFP, RSYNC, Unison, iSCSI (initiator and target), HAST, CARP, Bridge, UPnP, and Bittorent, which are all highly configurable by using XigmaNAS's great web user interface.

XigmaNAS (formerly Nas4Free) is a great free solution for a NAS box, trouble is that without proper configuration it will not properly work with Active Directory consistently and can have issues with inheritance of permissions. These are the steps we take at RMTT to get them stable in an Active Directory environment. ///

Build hardware RAID Array (hardware RAID still faster than software RAID of XigmaNAS)

-- Hardware RAID might be marginally faster than software RAID, but completely negates all the benefits of using ZFS, including portability of drives to new server without needing a proprietary hardware controller...

#!/bin/sh 
find /mnt/tank0/.recycle/* -atime +14 -exec rm -rf '{}' \; 
find /mnt/tank0/.recycle/ -depth -type d -empty -exec rmdir {} \;

Update 2011/03/04: Updated the script to delete empty directories by adding:

 find /mnt/DATA/.recycle/ -depth -type d -empty -exec rmdir {} \; 

which was suggested by Alan in the comments below. Thanks Alan for the updated script!

#!/bin/sh 
find /mnt/netzwerk/.recycle/* -atime +90 -exec rm -rf '{}' \;
find /mnt/netzwerk/.recycle/ -depth -type d -empty -exec rmdir {} \; 
find /mnt/netzwerk2/.recycle/* -atime +90 -exec rm -rf '{}' \; 
find /mnt/netzwerk2/.recycle/ -depth -type d -empty -exec rmdir {} \;

SYSTEM > ADVANCED > CRON

sh /mnt/netzwerk/delrecycle.sh

A nice feature of a Samba server is the optional recycle bin, which stores all files and folders that were deleted from a client on a network share. However the Samba recycle bin behaves different in comparison to the Desktop recycle bins of Windows, Mac OS X and Linux.

This posting describes the peculiarities of this Samba feature and how to use this function reasonably.

Peculiarities

The recycle bin of Samba is different to those on Desktop computers. The objects that were deleted on a network share by connected client computers does not appear in the user’s Desktop recycle bin but in a hidden folder on the share. The name can be specified in the Samba configuration, e.g. .recycle.

Every share has its own .recycle folder. All deleted files and folder of a share of all users are stored in this special folder. The deleted objects are stored under the user name while preserving the original path if the option keeptree is activated in Samba’s configuration

That means, if a user john deletes the file \nas\my-share\Documents\Text.txt it appears in \nas\my-share.recycle\john\Documents\Text.txt.