Jonathan - Magical ZFS

I thought that I'd share something surprising that happened with me and ZFS
the other day.

For a while now, I've had my desktop system set up so that it has 2 SSDs for
the OS and home directory, and most of my data sits on a bunch of spinning
hard drives. The two SSDs are in a zpool as mirrors, and the spinning drives
are mirrored pairs in another pool. The interesting thing here is that one
of the two SSDs is 512GB, and the other is 1TB, which of course, leaves
512GB free on the second drive. PC-BSD is smart enough that if I pick the
smaller drive for the OS and the second drive as its mirror, the installer
gives the second drive a partition which is identical in size to the one of
the first drive rather than making the partition cover the whole disk and
then ZFS just not using it, because the other drive in the vdev was too
small.

So, I've had 512GB to play around with. At times I've used it as a cache for
my data pool, and other times, I've just used it as a place to put some
extra data (so long as I can afford to lose it, since it's then in a pool by
itself with no redundancy). Recently, I've used it for extra storage, and
when something went wrong with my computer that necessitated that I
reinstall the OS, I sent the data from that extra pool to another pool so
that I could restore it after PC-BSD had wiped out my installation when
putting the new one on.

So, I installed PC-BSD and then re-added the partition to the end of the 1TB
drive so that I could re-add the extra zpool, and lo and behold, what
happens? zpool complains that there's already a pool there. _That_ was
unexpected. The disk was repartitioned. Its data should be gone.

Now, gpart obviously doesn't overwrite the whole drive when you repartition
a drive. It just rewrites the partition table. But I'm used to thinking that
if you repartation a drive, everything on it is toast. But apparently, if
you line up your new partition in exactly the same spot that it was before,
ZFS is capable of recognizing what's there and using it.

I did a scrub of that pool, and it was completely fine. I didn't need to
restore it at all. It was completely unharmed in spite of the disk being
repartationed. It makes some sense when I think through the details of how
partitions and ZFS work, but it's not something that I ever would have
expected.

- Jonathan