Dear Maintainer,
Using the JFS filesystem for Monero blockchain storage and maybe other
general purpose data, someday. Please don't judge me ;)
There are severely wrong semantics, expectations from a North American
English speaker & reader, or just plain crazy bugs, mainly (and maybe
only) with the LABEL bits & pieces of JFS filesystems, in both the
manpages and JFS-related tool(s) dealing with LABELs. There also exist
Debian patches for LABEL-related work.
As an aside, there are comments claiming to right-align the LABEL in the
JFS log device's superblock. I have seen this occur in my testing. Why?
Perhaps a legacy change from an old metadata format to a new one,
utilizing previously unused space in the old superblock of a JFS log
device (confirmed this did historically occur). I wonder if that unused
space overlapped, in a manner, with which the right-alignment was kept
for historical or compatibility reasons. Or, maybe it is a lower-level
vfs thing I do not yet understand.
Preface:
I do not know the underlying layout of the JFS filesystem, beyond what I
peeked at in jfs_superblock.h, super.c, and tune.c of the distribution's
jfsutils package (Debian Bookworm). Time is always a commodity in short
supply. But, I am very glad to have found a suggestion to use JFS for my
workload here. I have need to store the blockchain of Monero on this
filesystem, which seems to be a weird database in-and-of-itself (not
what I picture a "blockchain" to be) from my minimal analysis of its
huge file, which changes as if there are a lot of random-access writes.
Sure, ext[2..4] would have been just peachy, too!
btrfs, while otherwise pretty decent, was terrible regarding this
workload previously, even with the no-data-cow attributes and flags
toggled in all the right places and manners. I realize blockchain
verification can be CPU-heavy, but btrfs (and my configuration of it)
/seemed/ to be a huge part of the CPU *and* disk I/O bottleneck. Slowed
down my `apt` and friends a great deal, too. I should have stuck closer
to the defaults ;)
Also, I am happy to have learned a decent deal of information while
performing these experiments. It was interesting to see JFS again, which
I believe was the default filesystem (or at least a prominent option) on
one of my first set of compact discs obtained from CompUSA as a teenager
(SUSE Linux, I believe). Already, I was past the three, four, five, or
six (or however many) floppy-disks needed to bootstrap a Linux system at
that time, so that must have been around 2001/2002, shortly before they
were going out of business in my area :( The nostalgia JFS conjured up
for me, though :) Feels good!
Landed on RedHat for a while very shortly after, then Gentoo, then
Ubuntu, which then ultimately led me here, to my favorite folks, at
Debian.
First bug:
If the LABEL of the external logs device's superblock is exactly 16
characters, strange things may appear (at least with the `jfs_tune -l
<log_device>` utility command). I suspect the metadata is correct
on-disk, in compliance with the JFS layout, but I know it can yield
incorrect results with the tooling, explained more below. Because these
off-by-one things can sometimes be a trippy nagging at my OCD, and at
other times be an exploitable vulnerability in the real world, I felt
compelled to diclose. Any admin (or non-admin) innocently running the
tool /could/ trigger some malicious payload if the log device's UUID is
"just the right" (or "just the wrong") value... maybe. I do not know,
which is the main reason why I write verbosely here, using up that
comodity of time ;) When more time is available, I might poke around. I
know non-printable terminal characters can cause a mess of things.
Naively, I attempted to trick the jfs_tune utility to first re-align the
LABEL left-justified, *and* to remove the existing spaces. I do not
recall if the (left|write)-alignment issue suited me, but the tool still
printed out single quotes around the LABEL, and it padded it with spaces
to exactly 16 characters. That whitespace, I wished away. Even more
naively, I attempted (unsuccessfully of course!) to echo and/or printf a
string to trick the tool into accepting NUL characters. Bash does not
like this and complains loudly. I did read that some other shells might
allow this! I tried no other shells.
After this point, I had settled on a LABEL of exactly 16 characters,
with no whitespace: XmrBlockchainLog. I would /love/ to report that the
jfs_tune utility finally showed me something I liked. However, the C
code only does printf on a char *, interpreted as a string, which we
know is expected to always hold 16 bytes (any of them could be NUL). At
16 bytes in this example, there seems to be no room for a NUL character
to terminate it. What printed on the terminal, after the LABEL, was
"garbage". As explained above, after peeking at the source code, this
content after the LABEL is probably a string representation of the log
device's UUID, until it encounters a NUL or some other character causing
the standard output to flush its contents, and allow the tool to
continue executing.
If the filesystem spec allows this, there is probably no vulnerability
or corruption in the JFS filesystem or its external log device itself.
The tooling could then be trivially fixed. If the filesystem is corrupt
due this, then there are much deeper problems, where the quick-and-dirty
fix would be to change the tooling *and* the documentation to warn
against storing more than 15 characters for the LABEL, and truncate
offenders. Ideally, a filesystem expert would jump on a more proper fix.
...
I freshly reformatted the devices with different LUKS keys, and just
shortened the LABEL(s) for now, and I can live with the strange
right-aligned LABEL on the JFS filesystem log device, if I even choose
to use a LABEL there again (I have not yet). I promise I will not use
more than 15 characters, at least until I have better understanding.
More bugs, unclearness, or opportunities for enhancement:
The wording in the manpages is weird, and -J device=LABEL=<foo_whatever>
(only implicitly hinted at with an explicit comment claiming it
possible) does not appear to be functioning.
The -L option, when provided alongside the -J option, did not seem to
work as described... at first. I believe it is working, but the wording
in the manpage, and my platform experience, confused me...
UI improvements could be made to allow -L to be specified more than
once. Before the -j or -J option, -L could only affect the JFS
filesystem's LABEL, whereas after the -j or -J option, it could affect
the specified JFS log device's LABEL. Quite a few common utilities I use
take this route, so it was second nature for me to try, and it sent me
back to the manpage more than a few times to read it more carefully and
"literally" (I am not sure that is the right word here. But, well...).
In the end, I can live without a LABEL for the log device. After trying
all sorts of combinations with -J, this one-liner suited me well:
$ sudo jfs_mkfs -j /dev/mapper/luks-xmr-blockchain-log -L XmrBlockchain -s 128 /dev/mapper/luks-xmr-blockchain 4096
...
The luks stuff is not important. Linux / LUKS / dm* (Device Manager) all
works so brilliantly together, why not encrypt everything? Especially on
SSD drives, the very first time you obtain them, wipe with random
data... And, do no forget to turn off discard/trim. In the file
/etc/crypttab, this is something that must be enabled (thankfully, it is
disabled by default). As an extra precaution, it is wise
(defense-in-depth, perhaps) to make sure all possibly offending
mountpoints in /etc/fstab use the "nodiscard" mount option. ***
Read-The-Friendly-Manual, of course!
*** Performance may suffer, or eventually suffer, when "abusing" SSDs in
this manner. I still much prefer the spinning disks with strong
magnets.
...
If this filesystem (JFS) treats me well, which time will tell, I may
consider working with its tools and verifying corruption does not occur
on something as simple as changing a filesystem's LABEL.