5333 private links
The zpool configuration at rsync.net is simple and straightforward: we create 12 disk raidz-3 vdevs (which can sustain 3 disk failures without data loss) and concatenate them into zpools.
This allows for a nice, even, five vdevs per 60 drive JBOD. 12x16TB disks in one vdev is still a number we (and our retained consultants at Klara Systems[1]) are comfortable with.
We also employ three SSD cache drives per zpool - one single drive for L2ARC and a mirrored pair of drives for the SLOG. This is a very standard configuration and anyone working professionally with ZFS will recognize it. //
We maintain a full complement of checksum commands that you can run, over ssh, to verify files:
ssh user@rsync.net rmd160 some/file
The full list of checksum commands is:
md5 sha1 sha224 sha256 sha384 sha512 sha512t256 rmd160 skein256 skein512 skein1024 cksum
... and note the last one in the list, 'cksum' ...
cksum is interesting because if your use-case is not security or collision detection but, rather, simple verification of file transfer and/or integrity, cksum is much, much faster than the "serious" checksum tools. You should consider cksum if you have simple integrity checks to do on millions of files, etc.