LFS_O_SNAPSHOT brings back some of littlefs's idiosyncratic behavior
removed in the changes to open file syncing in a form that may be more
useful for users.
LFS_O_SNAPSHOT allows you to open a "snapshot" of a file. This is a cheap,
local copy of a file who's changes are not reflected on disk.
Internally, snapshot files use the same mechanism as pending writes. A
separate, copy-on-write CTZ skip-list is created, with read-only
references to the existing data blocks until a write occurs. The
difference is that snapshot files are not enrolled in the mlist, meaning
they won't get updates from open file syncs, and during close their
contents are simply discarded.
As an extra benefit, LFS_O_CREAT | LFS_O_SNAPSHOT is equivalent to
Linux's O_TMPFILE, making it easy to create temporary, unnamed files.
This may be useful for embedded development, where unnamed flash-backed
buffers may provide a slower, but larger, alternative to RAM-backed
buffers.
This was provided as a courtesy to hopefully make custom attributes more
easy to use, however the zeroing turned out to be a bit complicated when
syncing custom attributes across multiple open files.
Implicitly zeroing trailing buffer space is also inconsistent with the
other APIs in the filesystem, such as lfs_file_read, so this commit
removes the behavior.
If you need to handle differently sized custom attributes, you can
either pre-zero the custom attribute buffers, or use lfs_getattr to find
the on-disk size of custom attributes explicitly.
This is a bit of a complicated area for the custom-attribute API without
much precedent. littlefs allows users to provide custom attributes in
the lfs_file_config struct, which get written along with other file
metadata.
Sounds great on paper, but the devil is in the details. When does the
metadata actually get written?
What about this case?
lfs_file_opencfg(lfs, file, "path", LFS_O_WRONLY, cfg_with_attrs);
lfs_file_close(lfs, file); // does not write metadata
This normally doesn't write out metadata! We've opened the file for
writing, but made no changes, so normally littlefs doesn't bother to
commit anything to disk.
Before, as a courtesy, littlefs marked the file as dirty if it noticed
the file was opened for writing with custom attributes, but this is
inaccurate could to leave to problems after a file is synced:
lfs_file_opencfg(lfs, file, "path", LFS_O_WRONLY, cfg_with_attrs);
lfs_file_sync(lfs, file);
change_attrs();
lfs_file_close(lfs, file); // does not write metadata
Unfortunately, it isn't easy to know when metadata needs to be written.
Custom attributes are provided as read-only pointers to buffers which
may be updated without additional filesystem calls, this means we don't
know if custom attributes have actually changed on the device side. If
they haven't changed, writing out metadata on every sync would be
wasteful.
Another solution would be to compare our device-side attributes with
the disk-side attributes every sync, but that would be even more
expensive.
---
So for now, the simpliest and most efficient solution wins. Custom
attributes attached to open files, are not written unless the file data
itself changes.
Note that explicit calls to lfs_setattr always update on-disk
attributes, and opening a file with LFS_O_CREATE | LFS_O_TRUNC will also
always update the on-disk attributes (though not with just LFS_O_CREAT!).
There are a few ways we could provide an API that manually forces a write
of custom attributes, such as lfs_file_setattr, though without dynamic
memory, providing these APIs gets a bit complicated. So for now we will
see if users run into issues with the current scheme.
- Removed old tests and test scripts
- Reorganize the block devices to live under one directory
- Plugged new test framework into Makefile
renamed:
- scripts/test_.py -> scripts/test.py
- tests_ -> tests
- {file,ram,test}bd/* -> bd/*
It took a surprising amount of effort to make the Makefile behave since
it turns out the "test_%" rule could override "tests/test_%.toml.test"
which is generated as part of test.py.
