Daily Shaarli

All links of one day in a single page.

August 30, 2021

Solved - Best way to verify disk is zeroed? | The FreeBSD Forums
thumbnail

Q:

I've written a script to wipe all disks on any machine that netboots. It works fairly well, but I'd like to add verification of the final "zeroing" pass, and only shutdown the machines if the drive reads all zeros successfully, and otherwise show the error. I'm also trying to both minimize dependencies on ports if possible (currently only using pv(1) to show status), and make the final "read" pass perform quickly (the write passes are already fast).

However, I can't figure out what the best way to do this is. It seems the easiest way is to use another port, security/bcwipe, and run it with bcwipe -bfmz /dev/adaX, which both zeros the disks and verifies the write, but this requires another port. I've looked at other, simpler ways of doing using basic system utilities, like od(1), but then the disks read slowly (about 1/3rd of the speed dd(1) will do with bs=1m). I suspect this is because you can't specify a buffer size for od, and it's reading small chunks. //

dd if=/dev/zero of=/dev/adaX bs=1m

A:

In this particular case I would write a tiny C program for this purpose. The C compiler is in the base system, so you don't need to install additional ports.

Code:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>

#define BLOCKSIZE (128 * 1024)
uint64_t data[(BLOCKSIZE + 7) / 8];

int
main (void)
{
int inbytes, i;

    while ((inbytes = read(STDIN_FILENO, data, BLOCKSIZE)) > 0)
            for (i = 0; i < (inbytes + 7) / 8; i++)
                    if (data[i])
                            exit (1);
    if (inbytes < 0) {
            perror (NULL);
            exit (2);
    }
    return 0;

}

Save that source code as “testzero.c”.

Then compile it like this: cc -O2 -o testzero testzero.c

That'll give you a binary named “testzero”. Put it somewhere in your $PATH so your shell can find it, or type “./testzero” to run it from the current directory. The program reads a file (or device) from standard input and exits with code 0 if the file is all zeros, so you can use it in a shell script like this:

Code:

if testzero < /dev/ada0; then
echo "allright, disk is zero"
shutdown -p now
else
echo "something went wrong!"
yes | tr -c x '\a' # call attention
fi

Note that I have set the block size to 128 KB, not 1 MB. In my (limited) testing it was faster with 128 KB (actually, as fast as the speed of the physical disk). This may depend on your kernel settings, CPU cache or file system parameters, though. YMMV, so you might want to test different values. If you change it in the source code, don't forget to recompile the program. //

For simplicity, the program reads the file or device from standard input, so there is no reason to parse the argument vector, so I just specified it as “void” instead of the usual “int argc, char *argv[]”. Besides, the compiler issues a warning if you specify argc and argv without actually using them. Note that you could omit the formal parameter completely (i.e. “()” instead of “(void)”), but I think it's a good habit to specify “void” for documentation purposes, so the author's intention is clear. For similar reasons the return type is specified as “int” – actually that wouldn't be necessary because int is the default return type (not void!). In other words: You can just write “main ()” instead of “int main (void)” if you want – both mean exactly the same, but I adhere to the Python motto “explicit is better than implicit”.

Just in case someone wonders: There is no difference (performance-wise) between using standard input vs. opening a file specified on the command line. Just make sure that the shell opens the file directly and passes it to the program (using redirection syntax with “<”). Do not use something like “cat /dev/foo | testzero” because this will create an additional process for the cat command and connect it with a pipe to the testzero command – the pipe may reduce the performance considerably. (NB: The cat command is often abused; I guess that 99% of uses of cat in shell scripts are superfluous, sometimes even harmful.) //

I made a few modifications to his code to report where the error lies if it finds one, but otherwise this does appear to read at the maximum disk speed. Thanks!

Edit: If anyone is interested here's the code after the changes I made. It's also a bit more verbose.
C:

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>

#define BLOCKSIZE (128 * 1024)
uint64_t data[(BLOCKSIZE + 7) / 8];

int main(void) {
int inbytes;
uint64_t intotal = 0;

while((inbytes = read(STDIN_FILENO, data, BLOCKSIZE)) > 0) {
    inbytes = (inbytes + 7) >> 3;

    for(int i = 0; i < inbytes; i++) {
        if(data[i]) {
            intotal = (intotal + i) << 3;    // Convert back to byte offset
            printf("Non-zero byte detected at offset range: %lu to %lu\n", intotal, intotal + 7);
            exit(1);
        }
    }

    intotal += inbytes;
}

if(inbytes < 0) {
    perror(NULL);
    exit(2);
}

printf("Disk is fully zeroed.\n");
return 0;

}

smartctl.8.in in trunk/smartmontools – smartmontools

smartctl - Control and Monitor Utility for SMART Disks

SYNOPSIS
smartctl [options] device

DESCRIPTION
smartctl controls the Self-Monitoring, Analysis and Reporting Technology (SMART) system built into most ATA/SATA and SCSI/SAS hard drives and solid-state drives. The purpose of SMART is to monitor the reliability of the hard drive and predict drive failures, and to carry out different types of drive self-tests. smartctl also supports some features not related to SMART. This version of smartctl is compatible with ACS-3, ACS-2, ATA8-ACS, ATA/ATAPI-7 and earlier standards (see REFERENCES below).

Amazon.com: IO CREST SI-PEX40138 Internal 4 Port Non-Raid SATA III 6GB/S with M.2 B-Key 22x42 Pci-E X4 Controller Card for Desktop PC Support SSD and HDD with Low Profile Bracket. JMB585 Chipset : Everything Else

IO CREST SI-PEX40138 Internal 4 Port Non-Raid SATA III 6GB/S with M.2 B-Key 22x42 Pci-E X4 Controller Card for Desktop PC Support SSD and HDD with Low Profile Bracket. JMB585 Chipset

$55

  • Connect four SATA drives and ONE M.2 B-Key SSD to any free PCI-E x4 Slot, providing up to 1700 MB/s Speed.
  • Plug and play supported, No additional software installation or setting required.
  • Can support port Multiplier that is not in a RAID configuration.
  • Complies with PCI Express base Specification Revision 3.1a. Compatible with SATA hard drives & Solid state drives and computer systems with SATA controllers.
  • This card is only design for storage Use only, can not Boot any OS/ system drives. Backed by syba 3 Year hassle-free warranty and lifetime customer service.
  • Important Note: only work with PCI-E slot, not for PCI slot.