Compare commits

...

43 Commits

Author SHA1 Message Date
Shiven Gupta
87a2cb0e41 Fix assert 2020-08-18 17:36:14 -04:00
Christopher Haster
4c9146ea53 Merge pull request #405 from rojer/mfe
Fix -Wmissing-field-initializers
2020-04-09 05:42:46 -05:00
Deomid "rojer" Ryabkov
5a9f38df01 Remove -Wno-missing-field-initializers 2020-04-06 19:51:19 +01:00
Deomid "rojer" Ryabkov
1b033e9ab6 Fix -Wmissing-field-initializers 2020-04-03 02:18:14 +01:00
Christopher Haster
a049f1318e Merge pull request #372 from ARMmbed/test-revamp
Rework test framework, fix a number of related bugs
2020-03-31 18:25:13 -05:00
Christopher Haster
7257681f5d Merge branch 'master' into test-revamp 2020-03-31 18:24:54 -05:00
Christopher Haster
2da340af69 Merge pull request #373 from henrygab/patch-1
Indicate C99 standard as target for LittleFS code
2020-03-31 18:22:48 -05:00
Christopher Haster
02881e591b Merge pull request #360 from jpdoyle/master
Fix incorrect comment on `lfs_npw2`
2020-03-31 18:22:41 -05:00
Christopher Haster
38024d5a17 Merge pull request #356 from zqb-all/patch-1
Update SPEC.md
2020-03-31 18:22:34 -05:00
Christopher Haster
4a9bac4418 Merge pull request #322 from hemmick/master
Allow debug prints without __VA_ARGS__ in non-MSVC
2020-03-31 18:22:27 -05:00
Christopher Haster
6121495444 Merge pull request #266 from FreddieChopin/revert-bypass-cache
Revert "Don't bypass cache in `lfs_cache_prog()` and `lfs_cache_read()`"
2020-03-31 18:22:19 -05:00
John Hemmick
6372f515fe Allow debug prints without __VA_ARGS__
__VA_ARGS__ are frustrating in C. Even for their main purpose (printf),
they fall short in that they don't have a _portable_ way to have zero
arguments after the format string in a printf call.

Even if we detect compilers and use ##__VA_ARGS__ where available, GCC
emits a warning with -pedantic that is _impossible_ to explicitly
disable.

This commit contains the best solution we can think of. A bit of
indirection that adds a hidden "%s" % "" to the end of the format
string. This solution does not work everywhere as it has a runtime
cost, but it is hopefully ok for debug statements.
2020-03-29 21:58:49 -05:00
Christopher Haster
6622f3deee Bumped minor version to v2.2 2020-03-29 21:43:58 -05:00
Christopher Haster
5137e4b0ba Last minute tweaks to debug scripts
- Standardized littlefs debug statements to use hex prefixes and
  brackets for printing pairs.

- Removed the entry behavior for readtree and made -t the default.
  This is because 1. the CTZ skip-list parsing was broken, which is not
  surprising, and 2. the entry parsing was more complicated than useful.
  This functionality may be better implemented as a proper filesystem
  read script, complete with directory tree dumping.

- Changed test.py's --gdb argument to take [init, main, assert],
  this matches the names of the stages in C's startup.

- Added printing of tail to all mdir dumps in readtree/readmdir.

- Added a print for if any mdirs are corrupted in readtree.

- Added debug script side-effects to .gitignore.
2020-03-29 21:19:33 -05:00
Christopher Haster
ff84902970 Moved out block device tracing into separate define
Block device tracing has a lot of potential uses, of course debugging,
but it can also be used for profiling and externally tracking littlefs's
usage of the block device. However, block device tracing emits a massive
amount of output. So keeping block device tracing on by default limits
the usefulness of the filesystem tracing.

So, instead, I've moved the block device tracing into a separate
LFS_TESTBD_YES_TRACE define which switches on the LFS_TESTBD_TRACE
macro. Note that this means in order to get block device tracing, you
need to define both LFS_YES_TRACE and LFS_TESTBD_YES_TRACE. This is
needed as the LFS_TRACE definition is gated by LFS_YES_TRACE in
lfs_util.h.
2020-03-29 18:45:51 -05:00
Christopher Haster
01e42abd10 Merge pull request #401 from thrasher8390/bugfix/thrasher8390/issue-394-lookahead-buffer-corruption
Lookahead corruption fix given an IO Error during traversal
2020-03-29 17:59:00 -05:00
Christopher Haster
f9dbec3d92 Added test case catching issues with errors during a lookahead scan
Original issue found by thrasher8390
2020-03-29 14:12:58 -05:00
Derek Thrasher
f17d3d7eba Minor cleanup
- Removed the declaration of lfs_alloc_ack
- Consistent brackets
2020-03-29 14:12:30 -05:00
Derek Thrasher
5e5b5d8572 (chore) updates from PR, we decided not to move forward with changing v1 code since it can be risky. Let's improve the future! Also renamed and moved around a the lookahead free / reset function 2020-03-29 14:12:30 -05:00
Derek Thrasher
d498b9fb31 (bugfix) adding line function to clear out all the global 'free' information so that we can reset it after a failed traversal 2020-03-29 14:12:30 -05:00
Christopher Haster
4677421aba Added "evil" tests and detecion/recovery from bad pointers and infinite loops
These two features have been much requested by users, and have even had
several PRs proposed to fix these in several cases. Before this, these
error conditions usually were caught by internal asserts, however
asserts prevented users from implementing their own workarounds.

It's taken me a while to provide/accept a useful recovery mechanism
(returning LFS_ERR_CORRUPT instead of asserting) because my original thinking
was that these error conditions only occur due to bugs in the filesystem, and
these bugs should be fixed properly.

While I still think this is mostly true, the point has been made clear
that being able to recover from these conditions is definitely worth the
code cost. Hopefully this new behaviour helps the longevity of devices
even if the storage code fails.

Another, less important, reason I didn't want to accept fixes for these
situations was the lack of tests that prove the code's value. This has
been fixed with the new testing framework thanks to the additional of
"internal tests" which can call C static functions and really take
advantage of the internal information of the filesystem.
2020-03-20 09:26:07 -05:00
Chris Desjardins
cb26157880 Change assert to runtime check.
I had a system that was constantly hitting this assert, after making
this change it recovered immediately.
2020-02-23 22:18:08 -06:00
Christopher Haster
a7dfae4526 Minor tweaks to debugging scripts, fixed explode_asserts.py off-by-1
- Changed readmdir.py to print the metadata pair and revision count,
  which is useful when debugging commit issues.
- Added truncated data view to readtree.py by default. This does mean
  readtree.py must read all files on the filesystem to show the
  truncated data, hopefully this does not end up being a problem.
- Made overall representation hopefully more readable, including moving
  superblock under the root dir, userattrs under files, fixing a gstate
  rendering issue.
- Added rendering of soft-tails as dotted-arrows, hopefully this isn't
  too noisy.
- Fixed explode_asserts.py off-by-1 in #line mapping caused by a strip
  call in the assert generation eating newlines. The script matches
  line numbers between the original+modified files by emitting assert
  statements that use the same number of lines. An off-by-1 here causes
  the entire file to map lines incorrectly, which can be very annoying.
2020-02-22 23:50:03 -06:00
Christopher Haster
50fe8ae258 Renamed test_format -> test_superblocks, tweaked superblock tests
With the superblock expansion stuff, the test_format tests have grown
to test more advanced superblock-related features. This is fine but
deserves a rename so it's more clear.

Also fixed a typo that meant tests never ran with block cycles.
2020-02-22 23:35:28 -06:00
Christopher Haster
0990296619 Limited byte-level tests to native testing due to time
Byte-level writes are expensive and not suggested (caches >= 4 bytes
make much more sense), however there are many corner cases with
byte-level writes that can be easy to miss (power-loss leaving single
bytes written to disk).

Unfortunately, byte-level writes mixed with power-loss testing, the
Travis infrastructure, and Arm Thumb instruction set simulation
exceeds the 50-minute budget Travis allocates for jobs.

For now I'm disabling the byte-level tests under Qemu, with the hope that
performance improvements in littlefs will let us turn these tests back
on in the future.
2020-02-18 18:05:08 -06:00
Christopher Haster
d04b077506 Fixed minor things to get CI passing again
- Added caching to Travis install dirs, because otherwise
  pip3 install fails randomly
- Increased size of littlefs-fuse disk because test script has
  a larger footprint now
- Skip a couple of reentrant tests under byte-level writes because
  the tests just take too long and cause Travis to bail due to no
  output for 10m
- Fixed various Valgrind errors
  - Suppressed uninit checks for tests where LFS_BLOCK_ERASE_VALUE == -1.
    In this case rambd goes uninitialized, which is fine for rambd's
    purposes. Note I couldn't figure out how to limit this suppression
    to only the malloc in rambd, this doesn't seem possible with Valgrind.
  - Fixed memory leaks in exhaustion tests
  - Fixed off-by-1 string null-terminator issue in paths tests
- Fixed lfs_file_sync issue caused by revealed by fixing memory leaks
  in exhaustion tests. Getting ENOSPC during a file write puts the file
  in a bad state where littlefs doesn't know how to write it out safely.
  In this case, lfs_file_sync and lfs_file_close return 0 without
  writing out state so that device-side resources can still be cleaned
  up. To recover from ENOSPC, the file needs to be reopened and the
  writes recreated. Not sure if there is a better way to handle this.
- Added some quality-of-life improvements to Valgrind testing
  - Fit Valgrind messages into truncated output when not in verbose mode
  - Turned on origin tracking
2020-02-18 18:05:03 -06:00
Christopher Haster
c7987a3162 Restructured .travis.yml to span more jobs
The core of littlefs's CI testing is the full test suite, `make test`, run
under a number of configurations:

- Processor architecture:
  - x86 (native)
  - Arm Thumb
  - MIPS
  - PowerPC
- Storage geometry:
  - rs=16   ps=16   cs=64   bs=512   (default)
  - rs=1    ps=1    cs=64   bs=4KiB  (NOR flash)
  - rs=512  ps=512  cs=512  bs=512   (eMMC)
  - rs=4KiB ps=4KiB cs=4KiB bs=32KiB (NAND flash)
- Other corner cases:
  - no intrinsics
  - no inline
  - byte-level read/writes
  - single block-cycles
  - odd block counts
  - odd block sizes

The number of different configurations we need to test quickly exceeds the
50 minute time limit Travis has on jobs. Fortunately, we can split these
tests out into multiple jobs. This seems to be the intended course of
action for large CI "builds" in Travis, as this gives Travis a finer
grain of control over limiting builds.

Unfortunately, this created a couple issues:

1. The Travis configuration isn't actually that flexible. It allows a
   single "matrix expansion" which can be generated from top-level lists
   of different configurations. But it doesn't let you generate a matrix
   from two seperate environment variable lists (for arch + geometry).

   Without multiple matrix expansions, we're stuck writing out each test
   permutation by hand.

   On the bright-side, this was a good chance to really learn how YAML
   anchors work. I'm torn because on one hand anchors add what feels
   like unnecessary complexity to a config language, on the other hand,
   they did help quite a bit in working around Travis's limitations.

2. Now that we have 47 jobs instead of 7, reporting a separate status
   for each job stops making sense.

   What I've opted for here is to use a special NAME variable to
   deduplicate jobs, and used a few state-less rules to hopefully have
   the reported status make sense most of the time.

   - Overwrite "pending" statuses so that the last job to start owns the
     most recent "pending" status
   - Don't overwrite "failure" statuses unless the job number matches
     our own (in the case of CI restarts)
   - Don't write "success" statuses unless the job number matches our
     own, this should delay a green check-mark until the last-to-start
     job finishes
   - Always overwrite non-failures with "failure" statuses

   This does mean a temporary "success" may appear if the last job
   terminates before earlier jobs. But this is the simpliest solution
   I can think of without storing some complex state somewhere.

   Note we can only report the size this way because it's cheap to
   calculate in every job.
2020-02-18 17:34:23 -06:00
Christopher Haster
dcae185a00 Fixed typo in LFS_MKTAG_IF_ELSE 2020-02-12 11:31:34 -06:00
Christopher Haster
f4b17b379c Added test.py support for tmpfs-backed disks
RAM-backed testing is faster than file-backed testing. This is why
test.py uses rambd by default.

So why add support for tmpfs-backed disks if we can already run tests in
RAM? For reentrant testing.

Under reentrant testing we simulate power-loss by forcefully exiting the
test program at specific times. To make this power-loss meaningful, we need to
persist the disk across these power-losses. However, it's interesting to
note this persistence doesn't need to be actually backed by the
filesystem.

It may be possible to rearchitecture the tests to simulate power-loss a
different way, by say, using coroutines or setjmp/longjmp to leave
behind ongoing filesystem operations without terminating the program
completely. But at this point, I think it's best to work with what we
have.

And simply putting the test disks into a tmpfs mount-point seems to
work just fine.

Note this does force serialization of the tests, which isn't required
otherwise. Currently they are only serialized due to limitations in
test.py. If a future change wants to perallelize the tests, it may need
to rework RAM-backed reentrant tests.
2020-02-12 10:48:54 -06:00
Christopher Haster
9f546f154f Updated .travis.yml and added additional geometry constraints
Moved .travis.yml over to use the new test framework. A part of this
involved testing all of the configurations ran on the old framework
and deciding which to carry over. The new framework duplicates some of
the cases tested by the configurations so some configurations could be
dropped.

The .travis.yml includes some extreme ones, such as no inline files,
relocations every cycle, no intrinsics, power-loss every byte, unaligned
block_count and lookahead, and odd read_sizes.

There were several configurations were some tests failed because of
limitations in the tests themselves, so many conditions were added
to make sure the configurations can run on as many tests as possible.
2020-02-11 16:01:57 -06:00
Christopher Haster
b69cf890e6 Fixed CRC check when prog_size causes multiple CRCs per commit
This is a bit of a strange case that can be caused by storage with
very large prog sizes, such as NAND flash. We only have 10 bits to store
the size of our padding, so when the prog_size gets larger than 1024
bytes, we have to use multiple padding tags to commit to the next
prog_size boundary.

This causes some complication for the new logic that checks CRCs in case
our block becomes "readonly" and contains existing commits that just happen
to match our new commit size.

Here we just check the CRC of the first commit. This isn't perfect but
does protect against pure "readonly" blocks.
2020-02-09 22:43:20 -06:00
Christopher Haster
02c84ac5f4 Cleaned up dependent fixes on branch
These should probably have been cleaned up in each commit to allow
cherry-picking, but due to time I haven't been able to.

- Went with creating an mdir copy in lfs_dir_commit. This handles a
  number of related cleanup issues in lfs_dir_compact and it does so
  more robustly. As a plus we can use the copy to update dependencies
  in the mlist.

- Eliminated code left by the ENOSPC file outlining

- Cleaned up TODOs and lingering comments

- Changed the reentrant many directory create/rename/remove test to use
  a smaller set of directories because of space issues when
  READ/PROG_SIZE=512
2020-02-09 12:37:39 -06:00
Christopher Haster
6530cb3a61 Fixed lfs_fs_size doubling metadata-pairs
This was caused by the previous fix for allocations during
lfs_fs_deorphan in this branch. To catch half-orphans during block
allocations we needed to duplicate all metadata-pairs reported to
lfs_fs_traverse. Unfortunately this causes lfs_fs_size to report 2x the
number of metadata-pairs, which would undoubtably confuse users.

The fix here is inelegantly simple, just do a different traversale for
allocations and size measurements. It reuses the same code but touches
slightly different sets of blocks.

Unfortunately, this causes the public lfs_fs_traverse and lfs_fs_size
functions to split in how they report blocks. This is technically
allowed, since lfs_fs_traverse may report blocks multiple times due to
CoW behavior, however it's undesirable and I'm sure there will be some
confusion.

But I don't have a better solution, so from this point lfs_fs_traverse
will be reporting 2x metadata-blocks and shouldn't be used for finding
the number of available blocks on the filesystem.
2020-02-09 12:00:23 -06:00
Christopher Haster
fe957de892 Fixed broken wear-leveling when block_cycles = 2n-1
This was an interesting issue found during a GitHub discussion with
rmollway and thrasher8390.

Blocks in the metadata-pair are relocated every "block_cycles", or, more
mathy, when rev % block_cycles == 0 as long as rev += 1 every block write.

But there's a problem, rev isn't += 1 every block write. There are two
blocks in a metadata-pair, so looking at it from each blocks
perspective, rev += 2 every block write.

This leads to a sort of aliasing issue, where, if block_cycles is
divisible by 2, one block in the metadata-pair is always relocated, and
the other block is _never_ relocated. Causing a complete failure of
block-level wear-leveling.

Fortunately, because of a previous workaround to avoid block_cycles = 1
(since this will cause the relocation algorithm to never terminate), the
actual math is rev % (block_cycles+1) == 0. This means the bug only
shows its head in the much less likely case where block_cycles is a
multiple of 2 plus 1, or, in more mathy terms, block_cycles = 2n+1 for
some n.

To workaround this we can bitwise or our block_cycles with 1 to force it
to never be a multiple of 2n.

(Maybe we should do this during initialization? But then block_cycles
would need to be mutable.)

---

There's a few unrelated changes mixed into this commit that shouldn't be
there since I added this as part of a branch of bug fixes I'm putting
together rather hastily, so unfortunately this is not easily cherry-pickable.
2020-02-09 12:00:23 -06:00
Christopher Haster
6a550844f4 Modified readmdir/readtree to make reading non-truncated data easier
Added indention so there was a more clear separation between the tag
description and tag data.

Also took the best parts of readmdir.py and added it to readtree.py.
Initially I was thinking it was best for these to have completely
independent data representations, since you could always call readtree
to get more info, but this becomes tedius when needed to look at
low-level tag info across multiple directories on the filesystem.
2020-02-09 12:00:23 -06:00
Christopher Haster
f9c2fd93f2 Removed file outlining on ENOSPC in lfs_file_sync
This was initially added as protection against the case where a file
grew to no longer fit in a metadata-pair. While in most cases this
should be caught by the math in lfs_file_write, it doesn't handle a
problem that can happen if the files metadata is large enough that even
small inline files can't fit. This can happen if you combine a small
block size with large file names and many custom attributes.

But trying to outline on ENOSPC creates creates a lot of problems.

If we are actually low on space, this is one of the worst things we can
do. Inline files take up less space than CTZ skip-lists, but inline
files are rendered useless if we outline inline files as soon as we run
low on space.

On top of this, the outlining logic tries multiple mdir commits if it
gets ENOSPC, which can hide errors if ENOSPC is returned for other
reasons.

In a perfect world, we would be using a different error code for
no-room-in-metadata-pair, and no-blocks-on-disk.

For now I've removed the outlining logic and we will need to figure out
how to handle this situation more robustly.
2020-02-09 12:00:23 -06:00
Christopher Haster
44d7112794 Fixed tests/*.toml.* in .gitignore
Running test.py creates a log of garbage here
2020-02-09 12:00:22 -06:00
Christopher Haster
77e3078b9f Added/fixed tests for noop writes (where bd error can't be trusted)
It's interesting how many ways block devices can show failed writes:
1. prog can error
2. erase can error
3. read can error after writing (ECC failure)
4. prog doesn't error but doesn't write the data correctly
5. erase doesn't error but doesn't erase correctly

Can read fail without an error? Yes, though this appears the same as
prog and erase failing.

These weren't all simulated by testbd since I unintentionally assumed
the block device could always error. Fixed by added additional bad-black
behaviors to testbd.

Note: This also includes a small fix where we can miss bad writes if the
underlying block device contains a valid commit with the exact same
size in the exact same offset.
2020-02-09 12:00:22 -06:00
Christopher Haster
517d3414c5 Fixed more bugs, mostly related to ENOSPC on different geometries
Fixes:
- Fixed reproducability issue when we can't read a directory revision
- Fixed incorrect erase assumption if lfs_dir_fetch exceeds block size
- Fixed cleanup issue caused by lfs_fs_relocate failing when trying to
  outline a file in lfs_file_sync
- Fixed cleanup issue if we run out of space while extending a CTZ skip-list
- Fixed missing half-orphans when allocating blocks during lfs_fs_deorphan

Also:
- Added cycle-detection to readtree.py
- Allowed pseudo-C expressions in test conditions (and it's
  beautifully hacky, see line 187 of test.py)
- Better handling of ctrl-C during test runs
- Added build-only mode to test.py
- Limited stdout of test failures to 5 lines unless in verbose mode

Explanation of fixes below

1. Fixed reproducability issue when we can't read a directory revision

   An interesting subtlety of the block-device layer is that the
   block-device is allowed to return LFS_ERR_CORRUPT on reads to
   untouched blocks. This can easily happen if a user is using ECC or
   some sort of CMAC on their blocks. Normally we never run into this,
   except for the optimization around directory revisions where we use
   uninitialized data to start our revision count.

   We correctly handle this case by ignoring whats on disk if the read
   fails, but end up using unitialized RAM instead. This is not an issue
   for normal use, though it can lead to a small information leak.
   However it creates a big problem for reproducability, which is very
   helpful for debugging.

   I ended up running into a case where the RAM values for the revision
   count was different, causing two identical runs to wear-level at
   different times, leading to one version running out of space before a
   bug occured because it expanded the superblock early.

2. Fixed incorrect erase assumption if lfs_dir_fetch exceeds block size

   This could be caused if the previous tag was a valid commit and we
   lost power causing a partially written tag as the start of a new
   commit.

   Fortunately we already have a separate condition for exceeding the
   block size, so we can force that case to always treat the mdir as
   unerased.

3. Fixed cleanup issue caused by lfs_fs_relocate failing when trying to
   outline a file in lfs_file_sync

   Most operations involving metadata-pairs treat the mdir struct as
   entirely temporary and throw it out if any error occurs. Except for
   lfs_file_sync since the mdir is also a part of the file struct.

   This is relevant because of a cleanup issue in lfs_dir_compact that
   usually doesn't have side-effects. The issue is that lfs_fs_relocate
   can fail. It needs to allocate new blocks to relocate to, and as the
   disk reaches its end of life, it can fail with ENOSPC quite often.

   If lfs_fs_relocate fails, the containing lfs_dir_compact would return
   immediately without restoring the previous state of the mdir. If a new
   commit comes in on the same mdir, the old state left there could
   corrupt the filesystem.

   It's interesting to note this is forced to happen in lfs_file_sync,
   since it always tries to outline the file if it gets ENOSPC (ENOSPC
   can mean both no blocks to allocate and that the mdir is full). I'm
   not actually sure this bit of code is necessary anymore, we may be
   able to remove it.

4. Fixed cleanup issue if we run out of space while extending a CTZ
   skip-list

   The actually CTZ skip-list logic itself hasn't been touched in more
   than a year at this point, so I was surprised to find a bug here. But
   it turns out the CTZ skip-list could be put in an invalid state if we
   run out of space while trying to extend the skip-list.

   This only becomes a problem if we keep the file open, clean up some
   space elsewhere, and then continue to write to the open file without
   modifying it. Fortunately an easy fix.

5. Fixed missing half-orphans when allocating blocks during
   lfs_fs_deorphan

   This was a really interesting bug. Normally, we don't have to worry
   about allocations, since we force consistency before we are allowed
   to allocate blocks. But what about the deorphan operation itself?
   Don't we need to allocate blocks if we relocate while deorphaning?

   It turns out the deorphan operation can lead to allocating blocks
   while there's still orphans and half-orphans on the threaded
   linked-list. Orphans aren't an issue, but half-orphans may contain
   references to blocks in the outdated half, which doesn't get scanned
   during the normal allocation pass.

   Fortunately we already fetch directory entries to check CTZ lists, so
   we can also check half-orphans here. However this causes
   lfs_fs_traverse to duplicate all metadata-pairs, not sure what to do
   about this yet.
2020-02-09 11:54:22 -06:00
zhuangqiubin
4fb188369d Update SPEC.md
1.fix size in Layout of the CRC tag
2.update (size) to (size * 8)
2020-02-02 17:42:42 +08:00
Henry Gabryjelski
c8e9a64a21 Indicate C99 standard as target for LittleFS code
Resolve #358
2020-01-27 21:51:12 -08:00
Joe Doyle
626006af0c Fix incorrect comment on lfs_npw2
`lfs_npw2` returns a value v such that `2^v >= a` and `2^(v-1) < a`, but
the previous comment incorrectly describes it as "less than or equal to
a".
2020-01-02 13:46:07 -08:00
Freddie Chopin
5a12c443b8 Revert "Don't bypass cache in lfs_cache_prog() and lfs_cache_read()"
This reverts commit fdd239fe21.

Bypassing cache turned out to be a mistake which causes more problems
than it solves. Device driver should deal with alignment if this is
required - trying to do that in a file system is not a viable solution
anyway.
2019-08-09 23:02:33 +02:00
30 changed files with 1623 additions and 999 deletions

4
.gitignore vendored
View File

@@ -7,4 +7,6 @@
blocks/
lfs
test.c
tests_/*.toml.*
tests/*.toml.*
scripts/__pycache__
.gdb_history

View File

@@ -1,49 +1,70 @@
# Environment variables
# environment variables
env:
global:
- CFLAGS=-Werror
- MAKEFLAGS=-j
# Common test script
script:
# cache installation dirs
cache:
pip: true
directories:
- $HOME/.cache/apt
# common installation
_: &install-common
# need toml, also pip3 isn't installed by default?
- sudo apt-get install python3 python3-pip
- sudo pip3 install toml
# setup a ram-backed disk to speed up reentrant tests
- mkdir disks
- sudo mount -t tmpfs -o size=100m tmpfs disks
- export TFLAGS="$TFLAGS --disk=disks/disk"
# test cases
_: &test-example
# make sure example can at least compile
- sed -n '/``` c/,/```/{/```/d; p;}' README.md > test.c &&
- sed -n '/``` c/,/```/{/```/d; p}' README.md > test.c &&
make all CFLAGS+="
-Duser_provided_block_device_read=NULL
-Duser_provided_block_device_prog=NULL
-Duser_provided_block_device_erase=NULL
-Duser_provided_block_device_sync=NULL
-include stdio.h"
# default tests
_: &test-default
# normal+reentrant tests
- make test TFLAGS+="-nrk"
# common real-life geometries
_: &test-nor
# NOR flash: read/prog = 1 block = 4KiB
- make test TFLAGS+="-nrk -DLFS_READ_SIZE=1 -DLFS_BLOCK_SIZE=4096"
_: &test-emmc
# eMMC: read/prog = 512 block = 512
- make test TFLAGS+="-nrk -DLFS_READ_SIZE=512 -DLFS_BLOCK_SIZE=512"
_: &test-nand
# NAND flash: read/prog = 4KiB block = 32KiB
- make test TFLAGS+="-nrk -DLFS_READ_SIZE=4096 -DLFS_BLOCK_SIZE=\(32*1024\)"
# other extreme geometries that are useful for testing various corner cases
_: &test-no-intrinsics
- make test TFLAGS+="-nrk -DLFS_NO_INTRINSICS"
_: &test-no-inline
- make test TFLAGS+="-nrk -DLFS_INLINE_MAX=0"
_: &test-byte-writes
- make test TFLAGS+="-nrk -DLFS_READ_SIZE=1 -DLFS_CACHE_SIZE=1"
_: &test-block-cycles
- make test TFLAGS+="-nrk -DLFS_BLOCK_CYCLES=1"
_: &test-odd-block-count
- make test TFLAGS+="-nrk -DLFS_BLOCK_COUNT=1023 -DLFS_LOOKAHEAD_SIZE=256"
_: &test-odd-block-size
- make test TFLAGS+="-nrk -DLFS_READ_SIZE=11 -DLFS_BLOCK_SIZE=704"
# run tests
- make test QUIET=1
# run tests with a few different configurations
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=1 -DLFS_CACHE_SIZE=4"
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=512 -DLFS_CACHE_SIZE=512 -DLFS_BLOCK_CYCLES=16"
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=8 -DLFS_CACHE_SIZE=16 -DLFS_BLOCK_CYCLES=2"
- make test QUIET=1 CFLAGS+="-DLFS_BLOCK_COUNT=1023 -DLFS_LOOKAHEAD_SIZE=256"
- make clean test QUIET=1 CFLAGS+="-DLFS_INLINE_MAX=0"
- make clean test QUIET=1 CFLAGS+="-DLFS_EMUBD_ERASE_VALUE=0xff"
- make clean test QUIET=1 CFLAGS+="-DLFS_NO_INTRINSICS"
# additional configurations that don't support all tests (this should be
# fixed but at the moment it is what it is)
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=1 -DLFS_BLOCK_SIZE=4096"
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=\(2*1024\) -DLFS_BLOCK_SIZE=\(64*1024\)"
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=\(8*1024\) -DLFS_BLOCK_SIZE=\(64*1024\)"
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=11 -DLFS_BLOCK_SIZE=704"
# report size
_: &report-size
# compile and find the code size with the smallest configuration
- make clean size
OBJ="$(ls lfs*.o | tr '\n' ' ')"
- make -j1 clean size
OBJ="$(ls lfs*.c | sed 's/\.c/\.o/' | tr '\n' ' ')"
CFLAGS+="-DLFS_NO_ASSERT -DLFS_NO_DEBUG -DLFS_NO_WARN -DLFS_NO_ERROR"
| tee sizes
# update status if we succeeded, compare with master if possible
- |
if [ "$TRAVIS_TEST_RESULT" -eq 0 ]
@@ -51,10 +72,10 @@ script:
CURR=$(tail -n1 sizes | awk '{print $1}')
PREV=$(curl -u "$GEKY_BOT_STATUSES" https://api.github.com/repos/$TRAVIS_REPO_SLUG/status/master \
| jq -re "select(.sha != \"$TRAVIS_COMMIT\")
| .statuses[] | select(.context == \"$STAGE/$NAME\").description
| .statuses[] | select(.context == \"${TRAVIS_BUILD_STAGE_NAME,,}/$NAME\").description
| capture(\"code size is (?<size>[0-9]+)\").size" \
|| echo 0)
STATUS="Passed, code size is ${CURR}B"
if [ "$PREV" -ne 0 ]
then
@@ -62,257 +83,347 @@ script:
fi
fi
# CI matrix
# stage control
stages:
- name: test
- name: deploy
if: branch = master AND type = push
# job control
jobs:
include:
# native testing
- stage: test
env:
- STAGE=test
- NAME=littlefs-x86
# native testing
- &x86
stage: test
env:
- NAME=littlefs-x86
install: *install-common
script: [*test-example, *report-size]
- {<<: *x86, script: [*test-default, *report-size]}
- {<<: *x86, script: [*test-nor, *report-size]}
- {<<: *x86, script: [*test-emmc, *report-size]}
- {<<: *x86, script: [*test-nand, *report-size]}
- {<<: *x86, script: [*test-no-intrinsics, *report-size]}
- {<<: *x86, script: [*test-no-inline, *report-size]}
- {<<: *x86, script: [*test-byte-writes, *report-size]}
- {<<: *x86, script: [*test-block-cycles, *report-size]}
- {<<: *x86, script: [*test-odd-block-count, *report-size]}
- {<<: *x86, script: [*test-odd-block-size, *report-size]}
# cross-compile with ARM (thumb mode)
- stage: test
env:
- STAGE=test
- NAME=littlefs-arm
- CC="arm-linux-gnueabi-gcc --static -mthumb"
- EXEC="qemu-arm"
install:
- sudo apt-get install
gcc-arm-linux-gnueabi
libc6-dev-armel-cross
qemu-user
- arm-linux-gnueabi-gcc --version
- qemu-arm -version
# cross-compile with ARM (thumb mode)
- &arm
stage: test
env:
- NAME=littlefs-arm
- CC="arm-linux-gnueabi-gcc --static -mthumb"
- TFLAGS="$TFLAGS --exec=qemu-arm"
install:
- *install-common
- sudo apt-get install
gcc-arm-linux-gnueabi
libc6-dev-armel-cross
qemu-user
- arm-linux-gnueabi-gcc --version
- qemu-arm -version
script: [*test-example, *report-size]
- {<<: *arm, script: [*test-default, *report-size]}
- {<<: *arm, script: [*test-nor, *report-size]}
- {<<: *arm, script: [*test-emmc, *report-size]}
- {<<: *arm, script: [*test-nand, *report-size]}
- {<<: *arm, script: [*test-no-intrinsics, *report-size]}
- {<<: *arm, script: [*test-no-inline, *report-size]}
# it just takes way to long to run byte-level writes in qemu,
# note this is still tested in the native tests
#- {<<: *arm, script: [*test-byte-writes, *report-size]}
- {<<: *arm, script: [*test-block-cycles, *report-size]}
- {<<: *arm, script: [*test-odd-block-count, *report-size]}
- {<<: *arm, script: [*test-odd-block-size, *report-size]}
# cross-compile with PowerPC
- stage: test
env:
- STAGE=test
- NAME=littlefs-powerpc
- CC="powerpc-linux-gnu-gcc --static"
- EXEC="qemu-ppc"
install:
- sudo apt-get install
gcc-powerpc-linux-gnu
libc6-dev-powerpc-cross
qemu-user
- powerpc-linux-gnu-gcc --version
- qemu-ppc -version
# cross-compile with MIPS
- &mips
stage: test
env:
- NAME=littlefs-mips
- CC="mips-linux-gnu-gcc --static"
- TFLAGS="$TFLAGS --exec=qemu-mips"
install:
- *install-common
- sudo apt-get install
gcc-mips-linux-gnu
libc6-dev-mips-cross
qemu-user
- mips-linux-gnu-gcc --version
- qemu-mips -version
script: [*test-example, *report-size]
- {<<: *mips, script: [*test-default, *report-size]}
- {<<: *mips, script: [*test-nor, *report-size]}
- {<<: *mips, script: [*test-emmc, *report-size]}
- {<<: *mips, script: [*test-nand, *report-size]}
- {<<: *mips, script: [*test-no-intrinsics, *report-size]}
- {<<: *mips, script: [*test-no-inline, *report-size]}
# it just takes way to long to run byte-level writes in qemu,
# note this is still tested in the native tests
#- {<<: *mips, script: [*test-byte-writes, *report-size]}
- {<<: *mips, script: [*test-block-cycles, *report-size]}
- {<<: *mips, script: [*test-odd-block-count, *report-size]}
- {<<: *mips, script: [*test-odd-block-size, *report-size]}
# cross-compile with MIPS
- stage: test
env:
- STAGE=test
- NAME=littlefs-mips
- CC="mips-linux-gnu-gcc --static"
- EXEC="qemu-mips"
install:
- sudo apt-get install
gcc-mips-linux-gnu
libc6-dev-mips-cross
qemu-user
- mips-linux-gnu-gcc --version
- qemu-mips -version
# cross-compile with PowerPC
- &powerpc
stage: test
env:
- NAME=littlefs-powerpc
- CC="powerpc-linux-gnu-gcc --static"
- TFLAGS="$TFLAGS --exec=qemu-ppc"
install:
- *install-common
- sudo apt-get install
gcc-powerpc-linux-gnu
libc6-dev-powerpc-cross
qemu-user
- powerpc-linux-gnu-gcc --version
- qemu-ppc -version
script: [*test-example, *report-size]
- {<<: *powerpc, script: [*test-default, *report-size]}
- {<<: *powerpc, script: [*test-nor, *report-size]}
- {<<: *powerpc, script: [*test-emmc, *report-size]}
- {<<: *powerpc, script: [*test-nand, *report-size]}
- {<<: *powerpc, script: [*test-no-intrinsics, *report-size]}
- {<<: *powerpc, script: [*test-no-inline, *report-size]}
# it just takes way to long to run byte-level writes in qemu,
# note this is still tested in the native tests
#- {<<: *powerpc, script: [*test-byte-writes, *report-size]}
- {<<: *powerpc, script: [*test-block-cycles, *report-size]}
- {<<: *powerpc, script: [*test-odd-block-count, *report-size]}
- {<<: *powerpc, script: [*test-odd-block-size, *report-size]}
# self-host with littlefs-fuse for fuzz test
- stage: test
env:
- STAGE=test
- NAME=littlefs-fuse
if: branch !~ -prefix$
install:
- sudo apt-get install libfuse-dev
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v2
- fusermount -V
- gcc --version
before_script:
# setup disk for littlefs-fuse
- rm -rf littlefs-fuse/littlefs/*
- cp -r $(git ls-tree --name-only HEAD) littlefs-fuse/littlefs
# test under valgrind, checking for memory errors
- &valgrind
stage: test
env:
- NAME=littlefs-valgrind
install:
- *install-common
- sudo apt-get install valgrind
- valgrind --version
script:
- make test TFLAGS+="-k --valgrind"
- mkdir mount
- sudo chmod a+rw /dev/loop0
- dd if=/dev/zero bs=512 count=4096 of=disk
- losetup /dev/loop0 disk
script:
# self-host test
- make -C littlefs-fuse
# self-host with littlefs-fuse for fuzz test
- stage: test
env:
- NAME=littlefs-fuse
if: branch !~ -prefix$
install:
- *install-common
- sudo apt-get install libfuse-dev
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v2
- fusermount -V
- gcc --version
- littlefs-fuse/lfs --format /dev/loop0
- littlefs-fuse/lfs /dev/loop0 mount
# setup disk for littlefs-fuse
- rm -rf littlefs-fuse/littlefs/*
- cp -r $(git ls-tree --name-only HEAD) littlefs-fuse/littlefs
- ls mount
- mkdir mount/littlefs
- cp -r $(git ls-tree --name-only HEAD) mount/littlefs
- cd mount/littlefs
- stat .
- ls -flh
- make -B test_dirs test_files QUIET=1
- mkdir mount
- sudo chmod a+rw /dev/loop0
- dd if=/dev/zero bs=512 count=128K of=disk
- losetup /dev/loop0 disk
script:
# self-host test
- make -C littlefs-fuse
# self-host with littlefs-fuse for fuzz test
- stage: test
env:
- STAGE=test
- NAME=littlefs-migration
if: branch !~ -prefix$
install:
- sudo apt-get install libfuse-dev
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v2 v2
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v1 v1
- fusermount -V
- gcc --version
before_script:
# setup disk for littlefs-fuse
- rm -rf v2/littlefs/*
- cp -r $(git ls-tree --name-only HEAD) v2/littlefs
- littlefs-fuse/lfs --format /dev/loop0
- littlefs-fuse/lfs /dev/loop0 mount
- mkdir mount
- sudo chmod a+rw /dev/loop0
- dd if=/dev/zero bs=512 count=4096 of=disk
- losetup /dev/loop0 disk
script:
# compile v1 and v2
- make -C v1
- make -C v2
- ls mount
- mkdir mount/littlefs
- cp -r $(git ls-tree --name-only HEAD) mount/littlefs
- cd mount/littlefs
- stat .
- ls -flh
- make -B test
# run self-host test with v1
- v1/lfs --format /dev/loop0
- v1/lfs /dev/loop0 mount
# test migration using littlefs-fuse
- stage: test
env:
- NAME=littlefs-migration
if: branch !~ -prefix$
install:
- *install-common
- sudo apt-get install libfuse-dev
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v2 v2
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v1 v1
- fusermount -V
- gcc --version
- ls mount
- mkdir mount/littlefs
- cp -r $(git ls-tree --name-only HEAD) mount/littlefs
- cd mount/littlefs
- stat .
- ls -flh
- make -B test_dirs test_files QUIET=1
# setup disk for littlefs-fuse
- rm -rf v2/littlefs/*
- cp -r $(git ls-tree --name-only HEAD) v2/littlefs
# attempt to migrate
- cd ../..
- fusermount -u mount
- mkdir mount
- sudo chmod a+rw /dev/loop0
- dd if=/dev/zero bs=512 count=128K of=disk
- losetup /dev/loop0 disk
script:
# compile v1 and v2
- make -C v1
- make -C v2
- v2/lfs --migrate /dev/loop0
- v2/lfs /dev/loop0 mount
# run self-host test with v1
- v1/lfs --format /dev/loop0
- v1/lfs /dev/loop0 mount
# run self-host test with v2 right where we left off
- ls mount
- cd mount/littlefs
- stat .
- ls -flh
- make -B test_dirs test_files QUIET=1
- ls mount
- mkdir mount/littlefs
- cp -r $(git ls-tree --name-only HEAD) mount/littlefs
- cd mount/littlefs
- stat .
- ls -flh
- make -B test
# Automatically create releases
- stage: deploy
env:
- STAGE=deploy
- NAME=deploy
script:
- |
bash << 'SCRIPT'
set -ev
# Find version defined in lfs.h
LFS_VERSION=$(grep -ox '#define LFS_VERSION .*' lfs.h | cut -d ' ' -f3)
LFS_VERSION_MAJOR=$((0xffff & ($LFS_VERSION >> 16)))
LFS_VERSION_MINOR=$((0xffff & ($LFS_VERSION >> 0)))
# Grab latests patch from repo tags, default to 0, needs finagling
# to get past github's pagination api
PREV_URL=https://api.github.com/repos/$TRAVIS_REPO_SLUG/git/refs/tags/v$LFS_VERSION_MAJOR.$LFS_VERSION_MINOR.
PREV_URL=$(curl -u "$GEKY_BOT_RELEASES" "$PREV_URL" -I \
| sed -n '/^Link/{s/.*<\(.*\)>; rel="last"/\1/;p;q0};$q1' \
|| echo $PREV_URL)
LFS_VERSION_PATCH=$(curl -u "$GEKY_BOT_RELEASES" "$PREV_URL" \
| jq 'map(.ref | match("\\bv.*\\..*\\.(.*)$";"g")
.captures[].string | tonumber) | max + 1' \
|| echo 0)
# We have our new version
LFS_VERSION="v$LFS_VERSION_MAJOR.$LFS_VERSION_MINOR.$LFS_VERSION_PATCH"
echo "VERSION $LFS_VERSION"
# Check that we're the most recent commit
CURRENT_COMMIT=$(curl -f -u "$GEKY_BOT_RELEASES" \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/commits/master \
| jq -re '.sha')
[ "$TRAVIS_COMMIT" == "$CURRENT_COMMIT" ] || exit 0
# Create major branch
git branch v$LFS_VERSION_MAJOR HEAD
# Create major prefix branch
git config user.name "geky bot"
git config user.email "bot@geky.net"
git fetch https://github.com/$TRAVIS_REPO_SLUG.git \
--depth=50 v$LFS_VERSION_MAJOR-prefix || true
./scripts/prefix.py lfs$LFS_VERSION_MAJOR
git branch v$LFS_VERSION_MAJOR-prefix $( \
git commit-tree $(git write-tree) \
$(git rev-parse --verify -q FETCH_HEAD | sed -e 's/^/-p /') \
-p HEAD \
-m "Generated v$LFS_VERSION_MAJOR prefixes")
git reset --hard
# Update major version branches (vN and vN-prefix)
git push --atomic https://$GEKY_BOT_RELEASES@github.com/$TRAVIS_REPO_SLUG.git \
v$LFS_VERSION_MAJOR \
v$LFS_VERSION_MAJOR-prefix
# Build release notes
PREV=$(git tag --sort=-v:refname -l "v*" | head -1)
if [ ! -z "$PREV" ]
then
echo "PREV $PREV"
CHANGES=$(git log --oneline $PREV.. --grep='^Merge' --invert-grep)
printf "CHANGES\n%s\n\n" "$CHANGES"
fi
case ${GEKY_BOT_DRAFT:-minor} in
true) DRAFT=true ;;
minor) DRAFT=$(jq -R 'endswith(".0")' <<< "$LFS_VERSION") ;;
false) DRAFT=false ;;
esac
# Create the release and patch version tag (vN.N.N)
curl -f -u "$GEKY_BOT_RELEASES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/releases \
-d "{
\"tag_name\": \"$LFS_VERSION\",
\"name\": \"${LFS_VERSION%.0}\",
\"target_commitish\": \"$TRAVIS_COMMIT\",
\"draft\": $DRAFT,
\"body\": $(jq -sR '.' <<< "$CHANGES")
}" #"
SCRIPT
# attempt to migrate
- cd ../..
- fusermount -u mount
# Manage statuses
- v2/lfs --migrate /dev/loop0
- v2/lfs /dev/loop0 mount
# run self-host test with v2 right where we left off
- ls mount
- cd mount/littlefs
- stat .
- ls -flh
- make -B test
# automatically create releases
- stage: deploy
env:
- NAME=deploy
script:
- |
bash << 'SCRIPT'
set -ev
# Find version defined in lfs.h
LFS_VERSION=$(grep -ox '#define LFS_VERSION .*' lfs.h | cut -d ' ' -f3)
LFS_VERSION_MAJOR=$((0xffff & ($LFS_VERSION >> 16)))
LFS_VERSION_MINOR=$((0xffff & ($LFS_VERSION >> 0)))
# Grab latests patch from repo tags, default to 0, needs finagling
# to get past github's pagination api
PREV_URL=https://api.github.com/repos/$TRAVIS_REPO_SLUG/git/refs/tags/v$LFS_VERSION_MAJOR.$LFS_VERSION_MINOR.
PREV_URL=$(curl -u "$GEKY_BOT_RELEASES" "$PREV_URL" -I \
| sed -n '/^Link/{s/.*<\(.*\)>; rel="last"/\1/;p;q0};$q1' \
|| echo $PREV_URL)
LFS_VERSION_PATCH=$(curl -u "$GEKY_BOT_RELEASES" "$PREV_URL" \
| jq 'map(.ref | match("\\bv.*\\..*\\.(.*)$";"g")
.captures[].string | tonumber) | max + 1' \
|| echo 0)
# We have our new version
LFS_VERSION="v$LFS_VERSION_MAJOR.$LFS_VERSION_MINOR.$LFS_VERSION_PATCH"
echo "VERSION $LFS_VERSION"
# Check that we're the most recent commit
CURRENT_COMMIT=$(curl -f -u "$GEKY_BOT_RELEASES" \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/commits/master \
| jq -re '.sha')
[ "$TRAVIS_COMMIT" == "$CURRENT_COMMIT" ] || exit 0
# Create major branch
git branch v$LFS_VERSION_MAJOR HEAD
# Create major prefix branch
git config user.name "geky bot"
git config user.email "bot@geky.net"
git fetch https://github.com/$TRAVIS_REPO_SLUG.git \
--depth=50 v$LFS_VERSION_MAJOR-prefix || true
./scripts/prefix.py lfs$LFS_VERSION_MAJOR
git branch v$LFS_VERSION_MAJOR-prefix $( \
git commit-tree $(git write-tree) \
$(git rev-parse --verify -q FETCH_HEAD | sed -e 's/^/-p /') \
-p HEAD \
-m "Generated v$LFS_VERSION_MAJOR prefixes")
git reset --hard
# Update major version branches (vN and vN-prefix)
git push --atomic https://$GEKY_BOT_RELEASES@github.com/$TRAVIS_REPO_SLUG.git \
v$LFS_VERSION_MAJOR \
v$LFS_VERSION_MAJOR-prefix
# Build release notes
PREV=$(git tag --sort=-v:refname -l "v*" | head -1)
if [ ! -z "$PREV" ]
then
echo "PREV $PREV"
CHANGES=$(git log --oneline $PREV.. --grep='^Merge' --invert-grep)
printf "CHANGES\n%s\n\n" "$CHANGES"
fi
case ${GEKY_BOT_DRAFT:-minor} in
true) DRAFT=true ;;
minor) DRAFT=$(jq -R 'endswith(".0")' <<< "$LFS_VERSION") ;;
false) DRAFT=false ;;
esac
# Create the release and patch version tag (vN.N.N)
curl -f -u "$GEKY_BOT_RELEASES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/releases \
-d "{
\"tag_name\": \"$LFS_VERSION\",
\"name\": \"${LFS_VERSION%.0}\",
\"target_commitish\": \"$TRAVIS_COMMIT\",
\"draft\": $DRAFT,
\"body\": $(jq -sR '.' <<< "$CHANGES")
}" #"
SCRIPT
# manage statuses
before_install:
- |
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"$STAGE/$NAME\",
\"state\": \"pending\",
\"description\": \"${STATUS:-In progress}\",
\"target_url\": \"https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$TRAVIS_JOB_ID\"
}"
# don't clobber other (not us) failures
if ! curl https://api.github.com/repos/$TRAVIS_REPO_SLUG/status/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
| jq -e ".statuses[] | select(
.context == \"${TRAVIS_BUILD_STAGE_NAME,,}/$NAME\" and
.state == \"failure\" and
(.target_url | endswith(\"$TRAVIS_JOB_NUMBER\") | not))"
then
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"${TRAVIS_BUILD_STAGE_NAME,,}/$NAME\",
\"state\": \"pending\",
\"description\": \"${STATUS:-In progress}\",
\"target_url\": \"$TRAVIS_JOB_WEB_URL#$TRAVIS_JOB_NUMBER\"
}"
fi
after_failure:
- |
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"$STAGE/$NAME\",
\"state\": \"failure\",
\"description\": \"${STATUS:-Failed}\",
\"target_url\": \"https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$TRAVIS_JOB_ID\"
}"
# don't clobber other (not us) failures
if ! curl https://api.github.com/repos/$TRAVIS_REPO_SLUG/status/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
| jq -e ".statuses[] | select(
.context == \"${TRAVIS_BUILD_STAGE_NAME,,}/$NAME\" and
.state == \"failure\" and
(.target_url | endswith(\"$TRAVIS_JOB_NUMBER\") | not))"
then
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"${TRAVIS_BUILD_STAGE_NAME,,}/$NAME\",
\"state\": \"failure\",
\"description\": \"${STATUS:-Failed}\",
\"target_url\": \"$TRAVIS_JOB_WEB_URL#$TRAVIS_JOB_NUMBER\"
}"
fi
after_success:
- |
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"$STAGE/$NAME\",
\"state\": \"success\",
\"description\": \"${STATUS:-Passed}\",
\"target_url\": \"https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$TRAVIS_JOB_ID\"
}"
# Job control
stages:
- name: test
- name: deploy
if: branch = master AND type = push
# don't clobber other (not us) failures
# only update if we were last job to mark in progress,
# this isn't perfect but is probably good enough
if ! curl https://api.github.com/repos/$TRAVIS_REPO_SLUG/status/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
| jq -e ".statuses[] | select(
.context == \"${TRAVIS_BUILD_STAGE_NAME,,}/$NAME\" and
(.state == \"failure\" or .state == \"pending\") and
(.target_url | endswith(\"$TRAVIS_JOB_NUMBER\") | not))"
then
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"${TRAVIS_BUILD_STAGE_NAME,,}/$NAME\",
\"state\": \"success\",
\"description\": \"${STATUS:-Passed}\",
\"target_url\": \"$TRAVIS_JOB_WEB_URL#$TRAVIS_JOB_NUMBER\"
}"
fi

View File

@@ -26,8 +26,6 @@ endif
override CFLAGS += -I.
override CFLAGS += -std=c99 -Wall -pedantic
override CFLAGS += -Wextra -Wshadow -Wjump-misses-init -Wundef
# Remove missing-field-initializers because of GCC bug
override CFLAGS += -Wno-missing-field-initializers
ifdef VERBOSE
override TFLAGS += -v
@@ -45,7 +43,7 @@ test:
./scripts/test.py $(TFLAGS)
.SECONDEXPANSION:
test%: tests/test$$(firstword $$(subst \#, ,%)).toml
./scripts/test.py $(TFLAGS) $@
./scripts/test.py $@ $(TFLAGS)
-include $(DEP)

View File

@@ -115,6 +115,9 @@ the filesystem until sync or close is called on the file.
## Other notes
Littlefs is written in C, and specifically should compile with any compiler
that conforms to the `C99` standard.
All littlefs calls have the potential to return a negative error code. The
errors can be either one of those found in the `enum lfs_error` in
[lfs.h](lfs.h), or an error returned by the user's block device operations.

18
SPEC.md
View File

@@ -289,8 +289,8 @@ Layout of the name tag:
```
tag data
[-- 32 --][--- variable length ---]
[1| 3| 8 | 10 | 10 ][--- (size) ---]
^ ^ ^ ^ ^- size ^- file name
[1| 3| 8 | 10 | 10 ][--- (size * 8) ---]
^ ^ ^ ^ ^- size ^- file name
| | | '------ id
| | '----------- file type
| '-------------- type1 (0x0)
@@ -470,8 +470,8 @@ Layout of the inline-struct tag:
```
tag data
[-- 32 --][--- variable length ---]
[1|- 11 -| 10 | 10 ][--- (size) ---]
^ ^ ^ ^- size ^- inline data
[1|- 11 -| 10 | 10 ][--- (size * 8) ---]
^ ^ ^ ^- size ^- inline data
| | '------ id
| '------------ type (0x201)
'----------------- valid bit
@@ -556,8 +556,8 @@ Layout of the user-attr tag:
```
tag data
[-- 32 --][--- variable length ---]
[1| 3| 8 | 10 | 10 ][--- (size) ---]
^ ^ ^ ^ ^- size ^- attr data
[1| 3| 8 | 10 | 10 ][--- (size * 8) ---]
^ ^ ^ ^ ^- size ^- attr data
| | | '------ id
| | '----------- attr type
| '-------------- type1 (0x3)
@@ -764,9 +764,9 @@ Layout of the CRC tag:
```
tag data
[-- 32 --][-- 32 --|--- variable length ---]
[1| 3| 8 | 10 | 10 ][-- 32 --|--- (size) ---]
^ ^ ^ ^ ^ ^- crc ^- padding
| | | | '- size (12)
[1| 3| 8 | 10 | 10 ][-- 32 --|--- (size * 8 - 32) ---]
^ ^ ^ ^ ^ ^- crc ^- padding
| | | | '- size
| | | '------ id (0x3ff)
| | '----------- valid state
| '-------------- type1 (0x5)

View File

@@ -12,7 +12,7 @@
int lfs_filebd_createcfg(const struct lfs_config *cfg, const char *path,
const struct lfs_filebd_config *bdcfg) {
LFS_TRACE("lfs_filebd_createcfg(%p {.context=%p, "
LFS_FILEBD_TRACE("lfs_filebd_createcfg(%p {.context=%p, "
".read=%p, .prog=%p, .erase=%p, .sync=%p, "
".read_size=%"PRIu32", .prog_size=%"PRIu32", "
".block_size=%"PRIu32", .block_count=%"PRIu32"}, "
@@ -30,16 +30,16 @@ int lfs_filebd_createcfg(const struct lfs_config *cfg, const char *path,
bd->fd = open(path, O_RDWR | O_CREAT, 0666);
if (bd->fd < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_createcfg -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_createcfg -> %d", err);
return err;
}
LFS_TRACE("lfs_filebd_createcfg -> %d", 0);
LFS_FILEBD_TRACE("lfs_filebd_createcfg -> %d", 0);
return 0;
}
int lfs_filebd_create(const struct lfs_config *cfg, const char *path) {
LFS_TRACE("lfs_filebd_create(%p {.context=%p, "
LFS_FILEBD_TRACE("lfs_filebd_create(%p {.context=%p, "
".read=%p, .prog=%p, .erase=%p, .sync=%p, "
".read_size=%"PRIu32", .prog_size=%"PRIu32", "
".block_size=%"PRIu32", .block_count=%"PRIu32"}, "
@@ -51,26 +51,27 @@ int lfs_filebd_create(const struct lfs_config *cfg, const char *path) {
path);
static const struct lfs_filebd_config defaults = {.erase_value=-1};
int err = lfs_filebd_createcfg(cfg, path, &defaults);
LFS_TRACE("lfs_filebd_create -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_create -> %d", err);
return err;
}
int lfs_filebd_destroy(const struct lfs_config *cfg) {
LFS_TRACE("lfs_filebd_destroy(%p)", (void*)cfg);
LFS_FILEBD_TRACE("lfs_filebd_destroy(%p)", (void*)cfg);
lfs_filebd_t *bd = cfg->context;
int err = close(bd->fd);
if (err < 0) {
err = -errno;
LFS_TRACE("lfs_filebd_destroy -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_destroy -> %d", err);
return err;
}
LFS_TRACE("lfs_filebd_destroy -> %d", 0);
LFS_FILEBD_TRACE("lfs_filebd_destroy -> %d", 0);
return 0;
}
int lfs_filebd_read(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_filebd_read(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
LFS_FILEBD_TRACE("lfs_filebd_read(%p, "
"0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_filebd_t *bd = cfg->context;
@@ -89,24 +90,24 @@ int lfs_filebd_read(const struct lfs_config *cfg, lfs_block_t block,
(off_t)block*cfg->block_size + (off_t)off, SEEK_SET);
if (res1 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_read -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_read -> %d", err);
return err;
}
ssize_t res2 = read(bd->fd, buffer, size);
if (res2 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_read -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_read -> %d", err);
return err;
}
LFS_TRACE("lfs_filebd_read -> %d", 0);
LFS_FILEBD_TRACE("lfs_filebd_read -> %d", 0);
return 0;
}
int lfs_filebd_prog(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, const void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_filebd_prog(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
LFS_FILEBD_TRACE("lfs_filebd_prog(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_filebd_t *bd = cfg->context;
@@ -121,7 +122,7 @@ int lfs_filebd_prog(const struct lfs_config *cfg, lfs_block_t block,
(off_t)block*cfg->block_size + (off_t)off, SEEK_SET);
if (res1 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_prog -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_prog -> %d", err);
return err;
}
@@ -130,7 +131,7 @@ int lfs_filebd_prog(const struct lfs_config *cfg, lfs_block_t block,
ssize_t res2 = read(bd->fd, &c, 1);
if (res2 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_prog -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_prog -> %d", err);
return err;
}
@@ -143,23 +144,23 @@ int lfs_filebd_prog(const struct lfs_config *cfg, lfs_block_t block,
(off_t)block*cfg->block_size + (off_t)off, SEEK_SET);
if (res1 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_prog -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_prog -> %d", err);
return err;
}
ssize_t res2 = write(bd->fd, buffer, size);
if (res2 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_prog -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_prog -> %d", err);
return err;
}
LFS_TRACE("lfs_filebd_prog -> %d", 0);
LFS_FILEBD_TRACE("lfs_filebd_prog -> %d", 0);
return 0;
}
int lfs_filebd_erase(const struct lfs_config *cfg, lfs_block_t block) {
LFS_TRACE("lfs_filebd_erase(%p, 0x%"PRIx32")", (void*)cfg, block);
LFS_FILEBD_TRACE("lfs_filebd_erase(%p, 0x%"PRIx32")", (void*)cfg, block);
lfs_filebd_t *bd = cfg->context;
// check if erase is valid
@@ -170,7 +171,7 @@ int lfs_filebd_erase(const struct lfs_config *cfg, lfs_block_t block) {
off_t res1 = lseek(bd->fd, (off_t)block*cfg->block_size, SEEK_SET);
if (res1 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_erase -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_erase -> %d", err);
return err;
}
@@ -178,27 +179,27 @@ int lfs_filebd_erase(const struct lfs_config *cfg, lfs_block_t block) {
ssize_t res2 = write(bd->fd, &(uint8_t){bd->cfg->erase_value}, 1);
if (res2 < 0) {
int err = -errno;
LFS_TRACE("lfs_filebd_erase -> %d", err);
LFS_FILEBD_TRACE("lfs_filebd_erase -> %d", err);
return err;
}
}
}
LFS_TRACE("lfs_filebd_erase -> %d", 0);
LFS_FILEBD_TRACE("lfs_filebd_erase -> %d", 0);
return 0;
}
int lfs_filebd_sync(const struct lfs_config *cfg) {
LFS_TRACE("lfs_filebd_sync(%p)", (void*)cfg);
LFS_FILEBD_TRACE("lfs_filebd_sync(%p)", (void*)cfg);
// file sync
lfs_filebd_t *bd = cfg->context;
int err = fsync(bd->fd);
if (err) {
err = -errno;
LFS_TRACE("lfs_filebd_sync -> %d", 0);
LFS_FILEBD_TRACE("lfs_filebd_sync -> %d", 0);
return err;
}
LFS_TRACE("lfs_filebd_sync -> %d", 0);
LFS_FILEBD_TRACE("lfs_filebd_sync -> %d", 0);
return 0;
}

View File

@@ -15,6 +15,14 @@ extern "C"
{
#endif
// Block device specific tracing
#ifdef LFS_FILEBD_YES_TRACE
#define LFS_FILEBD_TRACE(...) LFS_TRACE(__VA_ARGS__)
#else
#define LFS_FILEBD_TRACE(...)
#endif
// filebd config (optional)
struct lfs_filebd_config {
// 8-bit erase value to use for simulating erases. -1 does not simulate

View File

@@ -8,7 +8,7 @@
int lfs_rambd_createcfg(const struct lfs_config *cfg,
const struct lfs_rambd_config *bdcfg) {
LFS_TRACE("lfs_rambd_createcfg(%p {.context=%p, "
LFS_RAMBD_TRACE("lfs_rambd_createcfg(%p {.context=%p, "
".read=%p, .prog=%p, .erase=%p, .sync=%p, "
".read_size=%"PRIu32", .prog_size=%"PRIu32", "
".block_size=%"PRIu32", .block_count=%"PRIu32"}, "
@@ -27,7 +27,7 @@ int lfs_rambd_createcfg(const struct lfs_config *cfg,
} else {
bd->buffer = lfs_malloc(cfg->block_size * cfg->block_count);
if (!bd->buffer) {
LFS_TRACE("lfs_rambd_createcfg -> %d", LFS_ERR_NOMEM);
LFS_RAMBD_TRACE("lfs_rambd_createcfg -> %d", LFS_ERR_NOMEM);
return LFS_ERR_NOMEM;
}
}
@@ -38,12 +38,12 @@ int lfs_rambd_createcfg(const struct lfs_config *cfg,
cfg->block_size * cfg->block_count);
}
LFS_TRACE("lfs_rambd_createcfg -> %d", 0);
LFS_RAMBD_TRACE("lfs_rambd_createcfg -> %d", 0);
return 0;
}
int lfs_rambd_create(const struct lfs_config *cfg) {
LFS_TRACE("lfs_rambd_create(%p {.context=%p, "
LFS_RAMBD_TRACE("lfs_rambd_create(%p {.context=%p, "
".read=%p, .prog=%p, .erase=%p, .sync=%p, "
".read_size=%"PRIu32", .prog_size=%"PRIu32", "
".block_size=%"PRIu32", .block_count=%"PRIu32"})",
@@ -53,24 +53,25 @@ int lfs_rambd_create(const struct lfs_config *cfg) {
cfg->read_size, cfg->prog_size, cfg->block_size, cfg->block_count);
static const struct lfs_rambd_config defaults = {.erase_value=-1};
int err = lfs_rambd_createcfg(cfg, &defaults);
LFS_TRACE("lfs_rambd_create -> %d", err);
LFS_RAMBD_TRACE("lfs_rambd_create -> %d", err);
return err;
}
int lfs_rambd_destroy(const struct lfs_config *cfg) {
LFS_TRACE("lfs_rambd_destroy(%p)", (void*)cfg);
LFS_RAMBD_TRACE("lfs_rambd_destroy(%p)", (void*)cfg);
// clean up memory
lfs_rambd_t *bd = cfg->context;
if (!bd->cfg->buffer) {
lfs_free(bd->buffer);
}
LFS_TRACE("lfs_rambd_destroy -> %d", 0);
LFS_RAMBD_TRACE("lfs_rambd_destroy -> %d", 0);
return 0;
}
int lfs_rambd_read(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_rambd_read(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
LFS_RAMBD_TRACE("lfs_rambd_read(%p, "
"0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_rambd_t *bd = cfg->context;
@@ -82,13 +83,14 @@ int lfs_rambd_read(const struct lfs_config *cfg, lfs_block_t block,
// read data
memcpy(buffer, &bd->buffer[block*cfg->block_size + off], size);
LFS_TRACE("lfs_rambd_read -> %d", 0);
LFS_RAMBD_TRACE("lfs_rambd_read -> %d", 0);
return 0;
}
int lfs_rambd_prog(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, const void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_rambd_prog(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
LFS_RAMBD_TRACE("lfs_rambd_prog(%p, "
"0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_rambd_t *bd = cfg->context;
@@ -108,12 +110,12 @@ int lfs_rambd_prog(const struct lfs_config *cfg, lfs_block_t block,
// program data
memcpy(&bd->buffer[block*cfg->block_size + off], buffer, size);
LFS_TRACE("lfs_rambd_prog -> %d", 0);
LFS_RAMBD_TRACE("lfs_rambd_prog -> %d", 0);
return 0;
}
int lfs_rambd_erase(const struct lfs_config *cfg, lfs_block_t block) {
LFS_TRACE("lfs_rambd_erase(%p, 0x%"PRIx32")", (void*)cfg, block);
LFS_RAMBD_TRACE("lfs_rambd_erase(%p, 0x%"PRIx32")", (void*)cfg, block);
lfs_rambd_t *bd = cfg->context;
// check if erase is valid
@@ -125,14 +127,14 @@ int lfs_rambd_erase(const struct lfs_config *cfg, lfs_block_t block) {
bd->cfg->erase_value, cfg->block_size);
}
LFS_TRACE("lfs_rambd_erase -> %d", 0);
LFS_RAMBD_TRACE("lfs_rambd_erase -> %d", 0);
return 0;
}
int lfs_rambd_sync(const struct lfs_config *cfg) {
LFS_TRACE("lfs_rambd_sync(%p)", (void*)cfg);
LFS_RAMBD_TRACE("lfs_rambd_sync(%p)", (void*)cfg);
// sync does nothing because we aren't backed by anything real
(void)cfg;
LFS_TRACE("lfs_rambd_sync -> %d", 0);
LFS_RAMBD_TRACE("lfs_rambd_sync -> %d", 0);
return 0;
}

View File

@@ -15,6 +15,14 @@ extern "C"
{
#endif
// Block device specific tracing
#ifdef LFS_RAMBD_YES_TRACE
#define LFS_RAMBD_TRACE(...) LFS_TRACE(__VA_ARGS__)
#else
#define LFS_RAMBD_TRACE(...)
#endif
// rambd config (optional)
struct lfs_rambd_config {
// 8-bit erase value to simulate erasing with. -1 indicates no erase

View File

@@ -12,7 +12,7 @@
int lfs_testbd_createcfg(const struct lfs_config *cfg, const char *path,
const struct lfs_testbd_config *bdcfg) {
LFS_TRACE("lfs_testbd_createcfg(%p {.context=%p, "
LFS_TESTBD_TRACE("lfs_testbd_createcfg(%p {.context=%p, "
".read=%p, .prog=%p, .erase=%p, .sync=%p, "
".read_size=%"PRIu32", .prog_size=%"PRIu32", "
".block_size=%"PRIu32", .block_count=%"PRIu32"}, "
@@ -38,23 +38,23 @@ int lfs_testbd_createcfg(const struct lfs_config *cfg, const char *path,
if (bd->cfg->wear_buffer) {
bd->wear = bd->cfg->wear_buffer;
} else {
bd->wear = lfs_malloc(sizeof(lfs_testbd_wear_t) * cfg->block_count);
bd->wear = lfs_malloc(sizeof(lfs_testbd_wear_t)*cfg->block_count);
if (!bd->wear) {
LFS_TRACE("lfs_testbd_createcfg -> %d", LFS_ERR_NOMEM);
LFS_TESTBD_TRACE("lfs_testbd_createcfg -> %d", LFS_ERR_NOMEM);
return LFS_ERR_NOMEM;
}
}
memset(bd->wear, 0, sizeof(lfs_testbd_wear_t) * cfg->block_count);
}
// create underlying block device
if (bd->persist) {
bd->u.file.cfg = (struct lfs_filebd_config){
.erase_value = bd->cfg->erase_value,
};
int err = lfs_filebd_createcfg(cfg, path, &bd->u.file.cfg);
LFS_TRACE("lfs_testbd_createcfg -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_createcfg -> %d", err);
return err;
} else {
bd->u.ram.cfg = (struct lfs_rambd_config){
@@ -62,13 +62,13 @@ int lfs_testbd_createcfg(const struct lfs_config *cfg, const char *path,
.buffer = bd->cfg->buffer,
};
int err = lfs_rambd_createcfg(cfg, &bd->u.ram.cfg);
LFS_TRACE("lfs_testbd_createcfg -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_createcfg -> %d", err);
return err;
}
}
int lfs_testbd_create(const struct lfs_config *cfg, const char *path) {
LFS_TRACE("lfs_testbd_create(%p {.context=%p, "
LFS_TESTBD_TRACE("lfs_testbd_create(%p {.context=%p, "
".read=%p, .prog=%p, .erase=%p, .sync=%p, "
".read_size=%"PRIu32", .prog_size=%"PRIu32", "
".block_size=%"PRIu32", .block_count=%"PRIu32"}, "
@@ -80,12 +80,12 @@ int lfs_testbd_create(const struct lfs_config *cfg, const char *path) {
path);
static const struct lfs_testbd_config defaults = {.erase_value=-1};
int err = lfs_testbd_createcfg(cfg, path, &defaults);
LFS_TRACE("lfs_testbd_create -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_create -> %d", err);
return err;
}
int lfs_testbd_destroy(const struct lfs_config *cfg) {
LFS_TRACE("lfs_testbd_destroy(%p)", (void*)cfg);
LFS_TESTBD_TRACE("lfs_testbd_destroy(%p)", (void*)cfg);
lfs_testbd_t *bd = cfg->context;
if (bd->cfg->erase_cycles && !bd->cfg->wear_buffer) {
lfs_free(bd->wear);
@@ -93,11 +93,11 @@ int lfs_testbd_destroy(const struct lfs_config *cfg) {
if (bd->persist) {
int err = lfs_filebd_destroy(cfg);
LFS_TRACE("lfs_testbd_destroy -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_destroy -> %d", err);
return err;
} else {
int err = lfs_rambd_destroy(cfg);
LFS_TRACE("lfs_testbd_destroy -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_destroy -> %d", err);
return err;
}
}
@@ -145,7 +145,8 @@ static int lfs_testbd_rawsync(const struct lfs_config *cfg) {
/// block device API ///
int lfs_testbd_read(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_testbd_read(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
LFS_TESTBD_TRACE("lfs_testbd_read(%p, "
"0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_testbd_t *bd = cfg->context;
@@ -155,22 +156,22 @@ int lfs_testbd_read(const struct lfs_config *cfg, lfs_block_t block,
LFS_ASSERT(block < cfg->block_count);
// block bad?
if (bd->cfg->erase_cycles &&
bd->cfg->badblock_behavior == LFS_TESTBD_BADBLOCK_NOREAD &&
bd->wear[block] >= bd->cfg->erase_cycles) {
LFS_TRACE("lfs_testbd_read -> %d", LFS_ERR_CORRUPT);
if (bd->cfg->erase_cycles && bd->wear[block] >= bd->cfg->erase_cycles &&
bd->cfg->badblock_behavior == LFS_TESTBD_BADBLOCK_READERROR) {
LFS_TESTBD_TRACE("lfs_testbd_read -> %d", LFS_ERR_CORRUPT);
return LFS_ERR_CORRUPT;
}
// read
int err = lfs_testbd_rawread(cfg, block, off, buffer, size);
LFS_TRACE("lfs_testbd_read -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_read -> %d", err);
return err;
}
int lfs_testbd_prog(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, const void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_testbd_prog(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
LFS_TESTBD_TRACE("lfs_testbd_prog(%p, "
"0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_testbd_t *bd = cfg->context;
@@ -180,17 +181,24 @@ int lfs_testbd_prog(const struct lfs_config *cfg, lfs_block_t block,
LFS_ASSERT(block < cfg->block_count);
// block bad?
if (bd->cfg->erase_cycles &&
bd->cfg->badblock_behavior == LFS_TESTBD_BADBLOCK_NOPROG &&
bd->wear[block] >= bd->cfg->erase_cycles) {
LFS_TRACE("lfs_testbd_prog -> %d", LFS_ERR_CORRUPT);
return LFS_ERR_CORRUPT;
if (bd->cfg->erase_cycles && bd->wear[block] >= bd->cfg->erase_cycles) {
if (bd->cfg->badblock_behavior ==
LFS_TESTBD_BADBLOCK_PROGERROR) {
LFS_TESTBD_TRACE("lfs_testbd_prog -> %d", LFS_ERR_CORRUPT);
return LFS_ERR_CORRUPT;
} else if (bd->cfg->badblock_behavior ==
LFS_TESTBD_BADBLOCK_PROGNOOP ||
bd->cfg->badblock_behavior ==
LFS_TESTBD_BADBLOCK_ERASENOOP) {
LFS_TESTBD_TRACE("lfs_testbd_prog -> %d", 0);
return 0;
}
}
// prog
int err = lfs_testbd_rawprog(cfg, block, off, buffer, size);
if (err) {
LFS_TRACE("lfs_testbd_prog -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_prog -> %d", err);
return err;
}
@@ -199,18 +207,18 @@ int lfs_testbd_prog(const struct lfs_config *cfg, lfs_block_t block,
bd->power_cycles -= 1;
if (bd->power_cycles == 0) {
// sync to make sure we persist the last changes
assert(lfs_testbd_rawsync(cfg) == 0);
LFS_ASSERT(lfs_testbd_rawsync(cfg) == 0);
// simulate power loss
exit(33);
}
}
LFS_TRACE("lfs_testbd_prog -> %d", 0);
LFS_TESTBD_TRACE("lfs_testbd_prog -> %d", 0);
return 0;
}
int lfs_testbd_erase(const struct lfs_config *cfg, lfs_block_t block) {
LFS_TRACE("lfs_testbd_erase(%p, 0x%"PRIx32")", (void*)cfg, block);
LFS_TESTBD_TRACE("lfs_testbd_erase(%p, 0x%"PRIx32")", (void*)cfg, block);
lfs_testbd_t *bd = cfg->context;
// check if erase is valid
@@ -219,9 +227,14 @@ int lfs_testbd_erase(const struct lfs_config *cfg, lfs_block_t block) {
// block bad?
if (bd->cfg->erase_cycles) {
if (bd->wear[block] >= bd->cfg->erase_cycles) {
if (bd->cfg->badblock_behavior == LFS_TESTBD_BADBLOCK_NOERASE) {
LFS_TRACE("lfs_testbd_erase -> %d", LFS_ERR_CORRUPT);
if (bd->cfg->badblock_behavior ==
LFS_TESTBD_BADBLOCK_ERASEERROR) {
LFS_TESTBD_TRACE("lfs_testbd_erase -> %d", LFS_ERR_CORRUPT);
return LFS_ERR_CORRUPT;
} else if (bd->cfg->badblock_behavior ==
LFS_TESTBD_BADBLOCK_ERASENOOP) {
LFS_TESTBD_TRACE("lfs_testbd_erase -> %d", 0);
return 0;
}
} else {
// mark wear
@@ -232,7 +245,7 @@ int lfs_testbd_erase(const struct lfs_config *cfg, lfs_block_t block) {
// erase
int err = lfs_testbd_rawerase(cfg, block);
if (err) {
LFS_TRACE("lfs_testbd_erase -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_erase -> %d", err);
return err;
}
@@ -241,20 +254,20 @@ int lfs_testbd_erase(const struct lfs_config *cfg, lfs_block_t block) {
bd->power_cycles -= 1;
if (bd->power_cycles == 0) {
// sync to make sure we persist the last changes
assert(lfs_testbd_rawsync(cfg) == 0);
LFS_ASSERT(lfs_testbd_rawsync(cfg) == 0);
// simulate power loss
exit(33);
}
}
LFS_TRACE("lfs_testbd_prog -> %d", 0);
LFS_TESTBD_TRACE("lfs_testbd_prog -> %d", 0);
return 0;
}
int lfs_testbd_sync(const struct lfs_config *cfg) {
LFS_TRACE("lfs_testbd_sync(%p)", (void*)cfg);
LFS_TESTBD_TRACE("lfs_testbd_sync(%p)", (void*)cfg);
int err = lfs_testbd_rawsync(cfg);
LFS_TRACE("lfs_testbd_sync -> %d", err);
LFS_TESTBD_TRACE("lfs_testbd_sync -> %d", err);
return err;
}
@@ -262,20 +275,20 @@ int lfs_testbd_sync(const struct lfs_config *cfg) {
/// simulated wear operations ///
lfs_testbd_swear_t lfs_testbd_getwear(const struct lfs_config *cfg,
lfs_block_t block) {
LFS_TRACE("lfs_testbd_getwear(%p, %"PRIu32")", (void*)cfg, block);
LFS_TESTBD_TRACE("lfs_testbd_getwear(%p, %"PRIu32")", (void*)cfg, block);
lfs_testbd_t *bd = cfg->context;
// check if block is valid
LFS_ASSERT(bd->cfg->erase_cycles);
LFS_ASSERT(block < cfg->block_count);
LFS_TRACE("lfs_testbd_getwear -> %"PRIu32, bd->wear[block]);
LFS_TESTBD_TRACE("lfs_testbd_getwear -> %"PRIu32, bd->wear[block]);
return bd->wear[block];
}
int lfs_testbd_setwear(const struct lfs_config *cfg,
lfs_block_t block, lfs_testbd_wear_t wear) {
LFS_TRACE("lfs_testbd_setwear(%p, %"PRIu32")", (void*)cfg, block);
LFS_TESTBD_TRACE("lfs_testbd_setwear(%p, %"PRIu32")", (void*)cfg, block);
lfs_testbd_t *bd = cfg->context;
// check if block is valid
@@ -284,6 +297,6 @@ int lfs_testbd_setwear(const struct lfs_config *cfg,
bd->wear[block] = wear;
LFS_TRACE("lfs_testbd_setwear -> %d", 0);
LFS_TESTBD_TRACE("lfs_testbd_setwear -> %d", 0);
return 0;
}

View File

@@ -19,14 +19,25 @@ extern "C"
#endif
// Mode determining how "bad blocks" behave during testing. This
// simulates some real-world circumstances such as writes not
// going through (noprog), erases not sticking (noerase), and ECC
// failures (noread).
// Block device specific tracing
#ifdef LFS_TESTBD_YES_TRACE
#define LFS_TESTBD_TRACE(...) LFS_TRACE(__VA_ARGS__)
#else
#define LFS_TESTBD_TRACE(...)
#endif
// Mode determining how "bad blocks" behave during testing. This simulates
// some real-world circumstances such as progs not sticking (prog-noop),
// a readonly disk (erase-noop), and ECC failures (read-error).
//
// Not that read-noop is not allowed. Read _must_ return a consistent (but
// may be arbitrary) value on every read.
enum lfs_testbd_badblock_behavior {
LFS_TESTBD_BADBLOCK_NOPROG = 0,
LFS_TESTBD_BADBLOCK_NOERASE = 1,
LFS_TESTBD_BADBLOCK_NOREAD = 2,
LFS_TESTBD_BADBLOCK_PROGERROR,
LFS_TESTBD_BADBLOCK_ERASEERROR,
LFS_TESTBD_BADBLOCK_READERROR,
LFS_TESTBD_BADBLOCK_PROGNOOP,
LFS_TESTBD_BADBLOCK_ERASENOOP,
};
// Type for measuring wear
@@ -82,7 +93,7 @@ typedef struct lfs_testbd {
/// Block device API ///
// Create a test block device using the geometry in lfs_config
//
//
// Note that filebd is used if a path is provided, if path is NULL
// testbd will use rambd which can be much faster.
int lfs_testbd_create(const struct lfs_config *cfg, const char *path);

409
lfs.c
View File

@@ -29,8 +29,8 @@ static int lfs_bd_read(lfs_t *lfs,
lfs_block_t block, lfs_off_t off,
void *buffer, lfs_size_t size) {
uint8_t *data = buffer;
LFS_ASSERT(block != LFS_BLOCK_NULL);
if (off+size > lfs->cfg->block_size) {
if (block >= lfs->cfg->block_count ||
off+size > lfs->cfg->block_size) {
return LFS_ERR_CORRUPT;
}
@@ -71,6 +71,21 @@ static int lfs_bd_read(lfs_t *lfs,
diff = lfs_min(diff, rcache->off-off);
}
if (size >= hint && off % lfs->cfg->read_size == 0 &&
size >= lfs->cfg->read_size) {
// bypass cache?
diff = lfs_aligndown(diff, lfs->cfg->read_size);
int err = lfs->cfg->read(lfs->cfg, block, off, data, diff);
if (err) {
return err;
}
data += diff;
off += diff;
size -= diff;
continue;
}
// load to cache, first condition can no longer fail
LFS_ASSERT(block < lfs->cfg->block_count);
rcache->block = block;
@@ -173,7 +188,7 @@ static int lfs_bd_prog(lfs_t *lfs,
lfs_block_t block, lfs_off_t off,
const void *buffer, lfs_size_t size) {
const uint8_t *data = buffer;
LFS_ASSERT(block != LFS_BLOCK_NULL);
LFS_ASSERT(block == LFS_BLOCK_INLINE || block < lfs->cfg->block_count);
LFS_ASSERT(off + size <= lfs->cfg->block_size);
while (size > 0) {
@@ -269,7 +284,7 @@ typedef int32_t lfs_stag_t;
((cond) ? LFS_MKTAG(type, id, size) : LFS_MKTAG(LFS_FROM_NOOP, 0, 0))
#define LFS_MKTAG_IF_ELSE(cond, type1, id1, size1, type2, id2, size2) \
((cond) ? LFS_MKTAG(type, id, size) : LFS_MKTAG(type2, id2, size2))
((cond) ? LFS_MKTAG(type1, id1, size1) : LFS_MKTAG(type2, id2, size2))
static inline bool lfs_tag_isvalid(lfs_tag_t tag) {
return !(tag & 0x80000000);
@@ -414,6 +429,9 @@ static lfs_stag_t lfs_fs_parent(lfs_t *lfs, const lfs_block_t dir[2],
lfs_mdir_t *parent);
static int lfs_fs_relocate(lfs_t *lfs,
const lfs_block_t oldpair[2], lfs_block_t newpair[2]);
int lfs_fs_traverseraw(lfs_t *lfs,
int (*cb)(void *data, lfs_block_t block), void *data,
bool includeorphans);
static int lfs_fs_forceconsistency(lfs_t *lfs);
static int lfs_deinit(lfs_t *lfs);
#ifdef LFS_MIGRATE
@@ -434,6 +452,19 @@ static int lfs_alloc_lookahead(void *p, lfs_block_t block) {
return 0;
}
static void lfs_alloc_ack(lfs_t *lfs) {
lfs->free.ack = lfs->cfg->block_count;
}
// Invalidate the lookahead buffer. This is done during mounting and
// failed traversals
static void lfs_alloc_reset(lfs_t *lfs) {
lfs->free.off = lfs->seed % lfs->cfg->block_size;
lfs->free.size = 0;
lfs->free.i = 0;
lfs_alloc_ack(lfs);
}
static int lfs_alloc(lfs_t *lfs, lfs_block_t *block) {
while (true) {
while (lfs->free.i != lfs->free.size) {
@@ -472,18 +503,14 @@ static int lfs_alloc(lfs_t *lfs, lfs_block_t *block) {
// find mask of free blocks from tree
memset(lfs->free.buffer, 0, lfs->cfg->lookahead_size);
int err = lfs_fs_traverse(lfs, lfs_alloc_lookahead, lfs);
int err = lfs_fs_traverseraw(lfs, lfs_alloc_lookahead, lfs, true);
if (err) {
lfs_alloc_reset(lfs);
return err;
}
}
}
static void lfs_alloc_ack(lfs_t *lfs) {
lfs->free.ack = lfs->cfg->block_count;
}
/// Metadata pair and directory operations ///
static lfs_stag_t lfs_dir_getslice(lfs_t *lfs, const lfs_mdir_t *dir,
lfs_tag_t gmask, lfs_tag_t gtag,
@@ -711,7 +738,7 @@ static int lfs_dir_traverse(lfs_t *lfs,
uint16_t fromid = lfs_tag_size(tag);
uint16_t toid = lfs_tag_id(tag);
int err = lfs_dir_traverse(lfs,
buffer, 0, LFS_BLOCK_NULL, NULL, 0,
buffer, 0, 0xffffffff, NULL, 0,
LFS_MKTAG(0x600, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_STRUCT, 0, 0),
fromid, fromid+1, toid-fromid+diff,
@@ -745,6 +772,12 @@ static lfs_stag_t lfs_dir_fetchmatch(lfs_t *lfs,
// scanning the entire directory
lfs_stag_t besttag = -1;
// if either block address is invalid we return LFS_ERR_CORRUPT here,
// otherwise later writes to the pair could fail
if (pair[0] >= lfs->cfg->block_count || pair[1] >= lfs->cfg->block_count) {
return LFS_ERR_CORRUPT;
}
// find the block with the most recent revision
uint32_t revs[2] = {0, 0};
int r = 0;
@@ -771,7 +804,7 @@ static lfs_stag_t lfs_dir_fetchmatch(lfs_t *lfs,
// now scan tags to fetch the actual dir and find possible match
for (int i = 0; i < 2; i++) {
lfs_off_t off = 0;
lfs_tag_t ptag = LFS_BLOCK_NULL;
lfs_tag_t ptag = 0xffffffff;
uint16_t tempcount = 0;
lfs_block_t temptail[2] = {LFS_BLOCK_NULL, LFS_BLOCK_NULL};
@@ -779,7 +812,7 @@ static lfs_stag_t lfs_dir_fetchmatch(lfs_t *lfs,
lfs_stag_t tempbesttag = besttag;
dir->rev = lfs_tole32(dir->rev);
uint32_t crc = lfs_crc(LFS_BLOCK_NULL, &dir->rev, sizeof(dir->rev));
uint32_t crc = lfs_crc(0xffffffff, &dir->rev, sizeof(dir->rev));
dir->rev = lfs_fromle32(dir->rev);
while (true) {
@@ -802,11 +835,13 @@ static lfs_stag_t lfs_dir_fetchmatch(lfs_t *lfs,
tag = lfs_frombe32(tag) ^ ptag;
// next commit not yet programmed or we're not in valid range
if (!lfs_tag_isvalid(tag) ||
off + lfs_tag_dsize(tag) > lfs->cfg->block_size) {
if (!lfs_tag_isvalid(tag)) {
dir->erased = (lfs_tag_type1(ptag) == LFS_TYPE_CRC &&
dir->off % lfs->cfg->prog_size == 0);
break;
} else if (off + lfs_tag_dsize(tag) > lfs->cfg->block_size) {
dir->erased = false;
break;
}
ptag = tag;
@@ -848,7 +883,7 @@ static lfs_stag_t lfs_dir_fetchmatch(lfs_t *lfs,
dir->split = tempsplit;
// reset crc
crc = LFS_BLOCK_NULL;
crc = 0xffffffff;
continue;
}
@@ -959,7 +994,7 @@ static lfs_stag_t lfs_dir_fetchmatch(lfs_t *lfs,
dir->rev = revs[(r+1)%2];
}
LFS_ERROR("Corrupted dir pair at %"PRIx32" %"PRIx32,
LFS_ERROR("Corrupted dir pair at {0x%"PRIx32", 0x%"PRIx32"}",
dir->pair[0], dir->pair[1]);
return LFS_ERR_CORRUPT;
}
@@ -1226,13 +1261,14 @@ static int lfs_dir_commitattr(lfs_t *lfs, struct lfs_commit *commit,
}
static int lfs_dir_commitcrc(lfs_t *lfs, struct lfs_commit *commit) {
const lfs_off_t off1 = commit->off;
const uint32_t crc1 = commit->crc;
// align to program units
const lfs_off_t off1 = commit->off + sizeof(lfs_tag_t);
const lfs_off_t end = lfs_alignup(off1 + sizeof(uint32_t),
const lfs_off_t end = lfs_alignup(off1 + 2*sizeof(uint32_t),
lfs->cfg->prog_size);
// create crc tags to fill up remainder of commit, note that
// padding is not crcd, which lets fetches skip padding but
// padding is not crced, which lets fetches skip padding but
// makes committing a bit more complicated
while (commit->off < end) {
lfs_off_t off = commit->off + sizeof(lfs_tag_t);
@@ -1242,7 +1278,7 @@ static int lfs_dir_commitcrc(lfs_t *lfs, struct lfs_commit *commit) {
}
// read erased state from next program unit
lfs_tag_t tag = LFS_BLOCK_NULL;
lfs_tag_t tag = 0xffffffff;
int err = lfs_bd_read(lfs,
NULL, &lfs->rcache, sizeof(tag),
commit->block, noff, &tag, sizeof(tag));
@@ -1268,7 +1304,7 @@ static int lfs_dir_commitcrc(lfs_t *lfs, struct lfs_commit *commit) {
commit->off += sizeof(tag)+lfs_tag_size(tag);
commit->ptag = tag ^ ((lfs_tag_t)reset << 31);
commit->crc = LFS_BLOCK_NULL; // reset crc for next "commit"
commit->crc = 0xffffffff; // reset crc for next "commit"
}
// flush buffers
@@ -1279,10 +1315,16 @@ static int lfs_dir_commitcrc(lfs_t *lfs, struct lfs_commit *commit) {
// successful commit, check checksums to make sure
lfs_off_t off = commit->begin;
lfs_off_t noff = off1;
lfs_off_t noff = off1 + sizeof(uint32_t);
while (off < end) {
uint32_t crc = LFS_BLOCK_NULL;
uint32_t crc = 0xffffffff;
for (lfs_off_t i = off; i < noff+sizeof(uint32_t); i++) {
// check against written crc, may catch blocks that
// become readonly and match our commit size exactly
if (i == off1 && crc != crc1) {
return LFS_ERR_CORRUPT;
}
// leave it up to caching to make this efficient
uint8_t dat;
err = lfs_bd_read(lfs,
@@ -1320,6 +1362,9 @@ static int lfs_dir_alloc(lfs_t *lfs, lfs_mdir_t *dir) {
}
}
// zero for reproducability in case initial block is unreadable
dir->rev = 0;
// rather than clobbering one of the blocks we just pretend
// the revision may be valid
int err = lfs_bd_read(lfs,
@@ -1335,7 +1380,7 @@ static int lfs_dir_alloc(lfs_t *lfs, lfs_mdir_t *dir) {
// set defaults
dir->off = sizeof(dir->rev);
dir->etag = LFS_BLOCK_NULL;
dir->etag = 0xffffffff;
dir->count = 0;
dir->tail[0] = LFS_BLOCK_NULL;
dir->tail[1] = LFS_BLOCK_NULL;
@@ -1420,16 +1465,16 @@ static int lfs_dir_compact(lfs_t *lfs,
lfs_mdir_t *dir, const struct lfs_mattr *attrs, int attrcount,
lfs_mdir_t *source, uint16_t begin, uint16_t end) {
// save some state in case block is bad
const lfs_block_t oldpair[2] = {dir->pair[1], dir->pair[0]};
const lfs_block_t oldpair[2] = {dir->pair[0], dir->pair[1]};
bool relocated = false;
bool exhausted = false;
bool tired = false;
// should we split?
while (end - begin > 1) {
// find size
lfs_size_t size = 0;
int err = lfs_dir_traverse(lfs,
source, 0, LFS_BLOCK_NULL, attrs, attrcount,
source, 0, 0xffffffff, attrs, attrcount,
LFS_MKTAG(0x400, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_NAME, 0, 0),
begin, end, -begin,
@@ -1469,8 +1514,14 @@ static int lfs_dir_compact(lfs_t *lfs,
// increment revision count
dir->rev += 1;
// If our revision count == n * block_cycles, we should force a relocation,
// this is how littlefs wear-levels at the metadata-pair level. Note that we
// actually use (block_cycles+1)|1, this is to avoid two corner cases:
// 1. block_cycles = 1, which would prevent relocations from terminating
// 2. block_cycles = 2n, which, due to aliasing, would only ever relocate
// one metadata block in the pair, effectively making this useless
if (lfs->cfg->block_cycles > 0 &&
(dir->rev % (lfs->cfg->block_cycles+1) == 0)) {
(dir->rev % ((lfs->cfg->block_cycles+1)|1) == 0)) {
if (lfs_pair_cmp(dir->pair, (const lfs_block_t[2]){0, 1}) == 0) {
// oh no! we're writing too much to the superblock,
// should we expand?
@@ -1506,7 +1557,7 @@ static int lfs_dir_compact(lfs_t *lfs,
#endif
} else {
// we're writing too much, time to relocate
exhausted = true;
tired = true;
goto relocate;
}
}
@@ -1518,8 +1569,8 @@ static int lfs_dir_compact(lfs_t *lfs,
struct lfs_commit commit = {
.block = dir->pair[1],
.off = 0,
.ptag = LFS_BLOCK_NULL,
.crc = LFS_BLOCK_NULL,
.ptag = 0xffffffff,
.crc = 0xffffffff,
.begin = 0,
.end = lfs->cfg->block_size - 8,
@@ -1548,7 +1599,7 @@ static int lfs_dir_compact(lfs_t *lfs,
// traverse the directory, this time writing out all unique tags
err = lfs_dir_traverse(lfs,
source, 0, LFS_BLOCK_NULL, attrs, attrcount,
source, 0, 0xffffffff, attrs, attrcount,
LFS_MKTAG(0x400, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_NAME, 0, 0),
begin, end, -begin,
@@ -1630,29 +1681,31 @@ relocate:
// commit was corrupted, drop caches and prepare to relocate block
relocated = true;
lfs_cache_drop(lfs, &lfs->pcache);
if (!exhausted) {
LFS_DEBUG("Bad block at %"PRIx32, dir->pair[1]);
if (!tired) {
LFS_DEBUG("Bad block at 0x%"PRIx32, dir->pair[1]);
}
// can't relocate superblock, filesystem is now frozen
if (lfs_pair_cmp(oldpair, (const lfs_block_t[2]){0, 1}) == 0) {
LFS_WARN("Superblock %"PRIx32" has become unwritable", oldpair[1]);
if (lfs_pair_cmp(dir->pair, (const lfs_block_t[2]){0, 1}) == 0) {
LFS_WARN("Superblock 0x%"PRIx32" has become unwritable",
dir->pair[1]);
return LFS_ERR_NOSPC;
}
// relocate half of pair
int err = lfs_alloc(lfs, &dir->pair[1]);
if (err && (err != LFS_ERR_NOSPC || !exhausted)) {
if (err && (err != LFS_ERR_NOSPC || !tired)) {
return err;
}
exhausted = false;
tired = false;
continue;
}
if (relocated) {
// update references if we relocated
LFS_DEBUG("Relocating %"PRIx32" %"PRIx32" -> %"PRIx32" %"PRIx32,
LFS_DEBUG("Relocating {0x%"PRIx32", 0x%"PRIx32"} "
"-> {0x%"PRIx32", 0x%"PRIx32"}",
oldpair[0], oldpair[1], dir->pair[0], dir->pair[1]);
int err = lfs_fs_relocate(lfs, oldpair, dir->pair);
if (err) {
@@ -1684,6 +1737,7 @@ static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir,
}
// calculate changes to the directory
lfs_mdir_t olddir = *dir;
bool hasdelete = false;
for (int i = 0; i < attrcount; i++) {
if (lfs_tag_type3(attrs[i].tag) == LFS_TYPE_CREATE) {
@@ -1705,11 +1759,16 @@ static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir,
lfs_mdir_t pdir;
int err = lfs_fs_pred(lfs, dir->pair, &pdir);
if (err && err != LFS_ERR_NOENT) {
*dir = olddir;
return err;
}
if (err != LFS_ERR_NOENT && pdir.split) {
return lfs_dir_drop(lfs, &pdir, dir);
err = lfs_dir_drop(lfs, &pdir, dir);
if (err) {
*dir = olddir;
return err;
}
}
}
@@ -1719,7 +1778,7 @@ static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir,
.block = dir->pair[0],
.off = dir->off,
.ptag = dir->etag,
.crc = LFS_BLOCK_NULL,
.crc = 0xffffffff,
.begin = dir->off,
.end = lfs->cfg->block_size - 8,
@@ -1737,6 +1796,7 @@ static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir,
if (err == LFS_ERR_NOSPC || err == LFS_ERR_CORRUPT) {
goto compact;
}
*dir = olddir;
return err;
}
@@ -1749,6 +1809,7 @@ static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir,
if (!lfs_gstate_iszero(&delta)) {
err = lfs_dir_getgstate(lfs, dir, &delta);
if (err) {
*dir = olddir;
return err;
}
@@ -1760,6 +1821,7 @@ static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir,
if (err == LFS_ERR_NOSPC || err == LFS_ERR_CORRUPT) {
goto compact;
}
*dir = olddir;
return err;
}
}
@@ -1770,6 +1832,7 @@ static int lfs_dir_commit(lfs_t *lfs, lfs_mdir_t *dir,
if (err == LFS_ERR_NOSPC || err == LFS_ERR_CORRUPT) {
goto compact;
}
*dir = olddir;
return err;
}
@@ -1788,6 +1851,7 @@ compact:
int err = lfs_dir_compact(lfs, dir, attrs, attrcount,
dir, 0, dir->count);
if (err) {
*dir = olddir;
return err;
}
}
@@ -1800,9 +1864,8 @@ compact:
// we need to copy the pair so they don't get clobbered if we refetch
// our mdir.
for (struct lfs_mlist *d = lfs->mlist; d; d = d->next) {
if (&d->m != dir && lfs_pair_cmp(d->m.pair, dir->pair) == 0) {
if (&d->m != dir && lfs_pair_cmp(d->m.pair, olddir.pair) == 0) {
d->m = *dir;
for (int i = 0; i < attrcount; i++) {
if (lfs_tag_type3(attrs[i].tag) == LFS_TYPE_DELETE &&
d->id == lfs_tag_id(attrs[i].tag)) {
@@ -1825,9 +1888,8 @@ compact:
}
}
lfs_block_t pair[2] = {dir->pair[0], dir->pair[1]};
for (struct lfs_mlist *d = lfs->mlist; d; d = d->next) {
if (lfs_pair_cmp(d->m.pair, pair) == 0) {
if (lfs_pair_cmp(d->m.pair, olddir.pair) == 0) {
while (d->id >= d->m.count && d->m.split) {
// we split and id is on tail now
d->id -= d->m.count;
@@ -2167,7 +2229,6 @@ static int lfs_ctz_find(lfs_t *lfs,
return err;
}
LFS_ASSERT(head >= 2 && head <= lfs->cfg->block_count);
current -= 1 << skip;
}
@@ -2187,7 +2248,6 @@ static int lfs_ctz_extend(lfs_t *lfs,
if (err) {
return err;
}
LFS_ASSERT(nblock >= 2 && nblock <= lfs->cfg->block_count);
{
err = lfs_bd_erase(lfs, nblock);
@@ -2204,16 +2264,16 @@ static int lfs_ctz_extend(lfs_t *lfs,
return 0;
}
size -= 1;
lfs_off_t index = lfs_ctz_index(lfs, &size);
size += 1;
lfs_size_t noff = size - 1;
lfs_off_t index = lfs_ctz_index(lfs, &noff);
noff = noff + 1;
// just copy out the last block if it is incomplete
if (size != lfs->cfg->block_size) {
for (lfs_off_t i = 0; i < size; i++) {
if (noff != lfs->cfg->block_size) {
for (lfs_off_t i = 0; i < noff; i++) {
uint8_t data;
err = lfs_bd_read(lfs,
NULL, rcache, size-i,
NULL, rcache, noff-i,
head, i, &data, 1);
if (err) {
return err;
@@ -2231,19 +2291,19 @@ static int lfs_ctz_extend(lfs_t *lfs,
}
*block = nblock;
*off = size;
*off = noff;
return 0;
}
// append block
index += 1;
lfs_size_t skips = lfs_ctz(index) + 1;
lfs_block_t nhead = head;
for (lfs_off_t i = 0; i < skips; i++) {
head = lfs_tole32(head);
nhead = lfs_tole32(nhead);
err = lfs_bd_prog(lfs, pcache, rcache, true,
nblock, 4*i, &head, 4);
head = lfs_fromle32(head);
nblock, 4*i, &nhead, 4);
nhead = lfs_fromle32(nhead);
if (err) {
if (err == LFS_ERR_CORRUPT) {
goto relocate;
@@ -2253,15 +2313,13 @@ static int lfs_ctz_extend(lfs_t *lfs,
if (i != skips-1) {
err = lfs_bd_read(lfs,
NULL, rcache, sizeof(head),
head, 4*i, &head, sizeof(head));
head = lfs_fromle32(head);
NULL, rcache, sizeof(nhead),
nhead, 4*i, &nhead, sizeof(nhead));
nhead = lfs_fromle32(nhead);
if (err) {
return err;
}
}
LFS_ASSERT(head >= 2 && head <= lfs->cfg->block_count);
}
*block = nblock;
@@ -2270,7 +2328,7 @@ static int lfs_ctz_extend(lfs_t *lfs,
}
relocate:
LFS_DEBUG("Bad block at %"PRIx32, nblock);
LFS_DEBUG("Bad block at 0x%"PRIx32, nblock);
// just clear cache and try a new block
lfs_cache_drop(lfs, pcache);
@@ -2374,9 +2432,9 @@ int lfs_file_opencfg(lfs_t *lfs, lfs_file_t *file,
// get next slot and create entry to remember name
err = lfs_dir_commit(lfs, &file->m, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_CREATE, file->id, 0)},
{LFS_MKTAG(LFS_TYPE_CREATE, file->id, 0), NULL},
{LFS_MKTAG(LFS_TYPE_REG, file->id, nlen), path},
{LFS_MKTAG(LFS_TYPE_INLINESTRUCT, file->id, 0)}));
{LFS_MKTAG(LFS_TYPE_INLINESTRUCT, file->id, 0), NULL}));
if (err) {
err = LFS_ERR_NAMETOOLONG;
goto cleanup;
@@ -2574,7 +2632,7 @@ static int lfs_file_relocate(lfs_t *lfs, lfs_file_t *file) {
return 0;
relocate:
LFS_DEBUG("Bad block at %"PRIx32, nblock);
LFS_DEBUG("Bad block at 0x%"PRIx32, nblock);
// just clear cache and try a new block
lfs_cache_drop(lfs, &lfs->pcache);
@@ -2651,7 +2709,7 @@ static int lfs_file_flush(lfs_t *lfs, lfs_file_t *file) {
break;
relocate:
LFS_DEBUG("Bad block at %"PRIx32, file->block);
LFS_DEBUG("Bad block at 0x%"PRIx32, file->block);
err = lfs_file_relocate(lfs, file);
if (err) {
return err;
@@ -2677,66 +2735,57 @@ int lfs_file_sync(lfs_t *lfs, lfs_file_t *file) {
LFS_TRACE("lfs_file_sync(%p, %p)", (void*)lfs, (void*)file);
LFS_ASSERT(file->flags & LFS_F_OPENED);
while (true) {
int err = lfs_file_flush(lfs, file);
if (err) {
file->flags |= LFS_F_ERRED;
LFS_TRACE("lfs_file_sync -> %d", err);
return err;
}
if ((file->flags & LFS_F_DIRTY) &&
!(file->flags & LFS_F_ERRED) &&
!lfs_pair_isnull(file->m.pair)) {
// update dir entry
uint16_t type;
const void *buffer;
lfs_size_t size;
struct lfs_ctz ctz;
if (file->flags & LFS_F_INLINE) {
// inline the whole file
type = LFS_TYPE_INLINESTRUCT;
buffer = file->cache.buffer;
size = file->ctz.size;
} else {
// update the ctz reference
type = LFS_TYPE_CTZSTRUCT;
// copy ctz so alloc will work during a relocate
ctz = file->ctz;
lfs_ctz_tole32(&ctz);
buffer = &ctz;
size = sizeof(ctz);
}
// commit file data and attributes
err = lfs_dir_commit(lfs, &file->m, LFS_MKATTRS(
{LFS_MKTAG(type, file->id, size), buffer},
{LFS_MKTAG(LFS_FROM_USERATTRS, file->id,
file->cfg->attr_count), file->cfg->attrs}));
if (err) {
if (err == LFS_ERR_NOSPC && (file->flags & LFS_F_INLINE)) {
goto relocate;
}
file->flags |= LFS_F_ERRED;
LFS_TRACE("lfs_file_sync -> %d", err);
return err;
}
file->flags &= ~LFS_F_DIRTY;
}
if (file->flags & LFS_F_ERRED) {
// it's not safe to do anything if our file errored
LFS_TRACE("lfs_file_sync -> %d", 0);
return 0;
}
relocate:
// inline file doesn't fit anymore
err = lfs_file_outline(lfs, file);
int err = lfs_file_flush(lfs, file);
if (err) {
file->flags |= LFS_F_ERRED;
LFS_TRACE("lfs_file_sync -> %d", err);
return err;
}
if ((file->flags & LFS_F_DIRTY) &&
!lfs_pair_isnull(file->m.pair)) {
// update dir entry
uint16_t type;
const void *buffer;
lfs_size_t size;
struct lfs_ctz ctz;
if (file->flags & LFS_F_INLINE) {
// inline the whole file
type = LFS_TYPE_INLINESTRUCT;
buffer = file->cache.buffer;
size = file->ctz.size;
} else {
// update the ctz reference
type = LFS_TYPE_CTZSTRUCT;
// copy ctz so alloc will work during a relocate
ctz = file->ctz;
lfs_ctz_tole32(&ctz);
buffer = &ctz;
size = sizeof(ctz);
}
// commit file data and attributes
err = lfs_dir_commit(lfs, &file->m, LFS_MKATTRS(
{LFS_MKTAG(type, file->id, size), buffer},
{LFS_MKTAG(LFS_FROM_USERATTRS, file->id,
file->cfg->attr_count), file->cfg->attrs}));
if (err) {
file->flags |= LFS_F_ERRED;
LFS_TRACE("lfs_file_sync -> %d", err);
return err;
}
file->flags &= ~LFS_F_DIRTY;
}
LFS_TRACE("lfs_file_sync -> %d", 0);
return 0;
}
lfs_ssize_t lfs_file_read(lfs_t *lfs, lfs_file_t *file,
@@ -3151,7 +3200,7 @@ int lfs_remove(lfs_t *lfs, const char *path) {
// delete the entry
err = lfs_dir_commit(lfs, &cwd, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_DELETE, lfs_tag_id(tag), 0)}));
{LFS_MKTAG(LFS_TYPE_DELETE, lfs_tag_id(tag), 0), NULL}));
if (err) {
lfs->mlist = dir.next;
LFS_TRACE("lfs_remove -> %d", err);
@@ -3277,12 +3326,12 @@ int lfs_rename(lfs_t *lfs, const char *oldpath, const char *newpath) {
// move over all attributes
err = lfs_dir_commit(lfs, &newcwd, LFS_MKATTRS(
{LFS_MKTAG_IF(prevtag != LFS_ERR_NOENT,
LFS_TYPE_DELETE, newid, 0)},
{LFS_MKTAG(LFS_TYPE_CREATE, newid, 0)},
LFS_TYPE_DELETE, newid, 0), NULL},
{LFS_MKTAG(LFS_TYPE_CREATE, newid, 0), NULL},
{LFS_MKTAG(lfs_tag_type3(oldtag), newid, strlen(newpath)), newpath},
{LFS_MKTAG(LFS_FROM_MOVE, newid, lfs_tag_id(oldtag)), &oldcwd},
{LFS_MKTAG_IF(samepair,
LFS_TYPE_DELETE, newoldid, 0)}));
LFS_TYPE_DELETE, newoldid, 0), NULL}));
if (err) {
lfs->mlist = prevdir.next;
LFS_TRACE("lfs_rename -> %d", err);
@@ -3295,7 +3344,7 @@ int lfs_rename(lfs_t *lfs, const char *oldpath, const char *newpath) {
// prep gstate and delete move id
lfs_fs_prepmove(lfs, 0x3ff, NULL);
err = lfs_dir_commit(lfs, &oldcwd, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_DELETE, lfs_tag_id(oldtag), 0)}));
{LFS_MKTAG(LFS_TYPE_DELETE, lfs_tag_id(oldtag), 0), NULL}));
if (err) {
lfs->mlist = prevdir.next;
LFS_TRACE("lfs_rename -> %d", err);
@@ -3428,7 +3477,7 @@ static int lfs_init(lfs_t *lfs, const struct lfs_config *cfg) {
LFS_ASSERT(lfs->cfg->block_size % lfs->cfg->cache_size == 0);
// check that the block size is large enough to fit ctz pointers
LFS_ASSERT(4*lfs_npw2(LFS_BLOCK_NULL / (lfs->cfg->block_size-2*4))
LFS_ASSERT(4*lfs_npw2(0xffffffff / (lfs->cfg->block_size-2*4))
<= lfs->cfg->block_size);
// block_cycles = 0 is no longer supported.
@@ -3587,7 +3636,7 @@ int lfs_format(lfs_t *lfs, const struct lfs_config *cfg) {
lfs_superblock_tole32(&superblock);
err = lfs_dir_commit(lfs, &root, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_CREATE, 0, 0)},
{LFS_MKTAG(LFS_TYPE_CREATE, 0, 0), NULL},
{LFS_MKTAG(LFS_TYPE_SUPERBLOCK, 0, 8), "littlefs"},
{LFS_MKTAG(LFS_TYPE_INLINESTRUCT, 0, sizeof(superblock)),
&superblock}));
@@ -3641,7 +3690,15 @@ int lfs_mount(lfs_t *lfs, const struct lfs_config *cfg) {
// scan directory blocks for superblock and any global updates
lfs_mdir_t dir = {.tail = {0, 1}};
lfs_block_t cycle = 0;
while (!lfs_pair_isnull(dir.tail)) {
if (cycle >= lfs->cfg->block_count/2) {
// loop detected
err = LFS_ERR_CORRUPT;
goto cleanup;
}
cycle += 1;
// fetch next block in tail list
lfs_stag_t tag = lfs_dir_fetchmatch(lfs, &dir, dir.tail,
LFS_MKTAG(0x7ff, 0x3ff, 0),
@@ -3676,7 +3733,7 @@ int lfs_mount(lfs_t *lfs, const struct lfs_config *cfg) {
uint16_t minor_version = (0xffff & (superblock.version >> 0));
if ((major_version != LFS_DISK_VERSION_MAJOR ||
minor_version > LFS_DISK_VERSION_MINOR)) {
LFS_ERROR("Invalid version %"PRIu16".%"PRIu16,
LFS_ERROR("Invalid version v%"PRIu16".%"PRIu16,
major_version, minor_version);
err = LFS_ERR_INVAL;
goto cleanup;
@@ -3732,7 +3789,7 @@ int lfs_mount(lfs_t *lfs, const struct lfs_config *cfg) {
// update littlefs with gstate
if (!lfs_gstate_iszero(&lfs->gstate)) {
LFS_DEBUG("Found pending gstate %08"PRIx32" %08"PRIx32" %08"PRIx32,
LFS_DEBUG("Found pending gstate 0x%08"PRIx32"%08"PRIx32"%08"PRIx32,
lfs->gstate.tag,
lfs->gstate.pair[0],
lfs->gstate.pair[1]);
@@ -3741,10 +3798,7 @@ int lfs_mount(lfs_t *lfs, const struct lfs_config *cfg) {
lfs->gdisk = lfs->gstate;
// setup free lookahead
lfs->free.off = lfs->seed % lfs->cfg->block_size;
lfs->free.size = 0;
lfs->free.i = 0;
lfs_alloc_ack(lfs);
lfs_alloc_reset(lfs);
LFS_TRACE("lfs_mount -> %d", 0);
return 0;
@@ -3764,10 +3818,9 @@ int lfs_unmount(lfs_t *lfs) {
/// Filesystem filesystem operations ///
int lfs_fs_traverse(lfs_t *lfs,
int (*cb)(void *data, lfs_block_t block), void *data) {
LFS_TRACE("lfs_fs_traverse(%p, %p, %p)",
(void*)lfs, (void*)(uintptr_t)cb, data);
int lfs_fs_traverseraw(lfs_t *lfs,
int (*cb)(void *data, lfs_block_t block), void *data,
bool includeorphans) {
// iterate over metadata pairs
lfs_mdir_t dir = {.tail = {0, 1}};
@@ -3776,7 +3829,6 @@ int lfs_fs_traverse(lfs_t *lfs,
if (lfs->lfs1) {
int err = lfs1_traverse(lfs, cb, data);
if (err) {
LFS_TRACE("lfs_fs_traverse -> %d", err);
return err;
}
@@ -3785,11 +3837,17 @@ int lfs_fs_traverse(lfs_t *lfs,
}
#endif
lfs_block_t cycle = 0;
while (!lfs_pair_isnull(dir.tail)) {
if (cycle >= lfs->cfg->block_count/2) {
// loop detected
return LFS_ERR_CORRUPT;
}
cycle += 1;
for (int i = 0; i < 2; i++) {
int err = cb(data, dir.tail[i]);
if (err) {
LFS_TRACE("lfs_fs_traverse -> %d", err);
return err;
}
}
@@ -3797,7 +3855,6 @@ int lfs_fs_traverse(lfs_t *lfs,
// iterate through ids in directory
int err = lfs_dir_fetch(lfs, &dir, dir.tail);
if (err) {
LFS_TRACE("lfs_fs_traverse -> %d", err);
return err;
}
@@ -3809,7 +3866,6 @@ int lfs_fs_traverse(lfs_t *lfs,
if (tag == LFS_ERR_NOENT) {
continue;
}
LFS_TRACE("lfs_fs_traverse -> %"PRId32, tag);
return tag;
}
lfs_ctz_fromle32(&ctz);
@@ -3818,9 +3874,16 @@ int lfs_fs_traverse(lfs_t *lfs,
err = lfs_ctz_traverse(lfs, NULL, &lfs->rcache,
ctz.head, ctz.size, cb, data);
if (err) {
LFS_TRACE("lfs_fs_traverse -> %d", err);
return err;
}
} else if (includeorphans &&
lfs_tag_type3(tag) == LFS_TYPE_DIRSTRUCT) {
for (int i = 0; i < 2; i++) {
err = cb(data, (&ctz.head)[i]);
if (err) {
return err;
}
}
}
}
}
@@ -3835,7 +3898,6 @@ int lfs_fs_traverse(lfs_t *lfs,
int err = lfs_ctz_traverse(lfs, &f->cache, &lfs->rcache,
f->ctz.head, f->ctz.size, cb, data);
if (err) {
LFS_TRACE("lfs_fs_traverse -> %d", err);
return err;
}
}
@@ -3844,22 +3906,36 @@ int lfs_fs_traverse(lfs_t *lfs,
int err = lfs_ctz_traverse(lfs, &f->cache, &lfs->rcache,
f->block, f->pos, cb, data);
if (err) {
LFS_TRACE("lfs_fs_traverse -> %d", err);
return err;
}
}
}
LFS_TRACE("lfs_fs_traverse -> %d", 0);
return 0;
}
int lfs_fs_traverse(lfs_t *lfs,
int (*cb)(void *data, lfs_block_t block), void *data) {
LFS_TRACE("lfs_fs_traverse(%p, %p, %p)",
(void*)lfs, (void*)(uintptr_t)cb, data);
int err = lfs_fs_traverseraw(lfs, cb, data, true);
LFS_TRACE("lfs_fs_traverse -> %d", 0);
return err;
}
static int lfs_fs_pred(lfs_t *lfs,
const lfs_block_t pair[2], lfs_mdir_t *pdir) {
// iterate over all directory directory entries
pdir->tail[0] = 0;
pdir->tail[1] = 1;
lfs_block_t cycle = 0;
while (!lfs_pair_isnull(pdir->tail)) {
if (cycle >= lfs->cfg->block_count/2) {
// loop detected
return LFS_ERR_CORRUPT;
}
cycle += 1;
if (lfs_pair_cmp(pdir->tail, pair) == 0) {
return 0;
}
@@ -3902,7 +3978,14 @@ static lfs_stag_t lfs_fs_parent(lfs_t *lfs, const lfs_block_t pair[2],
// use fetchmatch with callback to find pairs
parent->tail[0] = 0;
parent->tail[1] = 1;
lfs_block_t cycle = 0;
while (!lfs_pair_isnull(parent->tail)) {
if (cycle >= lfs->cfg->block_count/2) {
// loop detected
return LFS_ERR_CORRUPT;
}
cycle += 1;
lfs_stag_t tag = lfs_dir_fetchmatch(lfs, parent, parent->tail,
LFS_MKTAG(0x7ff, 0, 0x3ff),
LFS_MKTAG(LFS_TYPE_DIRSTRUCT, 0, 8),
@@ -3921,8 +4004,6 @@ static int lfs_fs_relocate(lfs_t *lfs,
const lfs_block_t oldpair[2], lfs_block_t newpair[2]) {
// update internal root
if (lfs_pair_cmp(oldpair, lfs->root) == 0) {
LFS_DEBUG("Relocating root %"PRIx32" %"PRIx32,
newpair[0], newpair[1]);
lfs->root[0] = newpair[0];
lfs->root[1] = newpair[1];
}
@@ -3958,7 +4039,7 @@ static int lfs_fs_relocate(lfs_t *lfs,
if (lfs_gstate_hasmovehere(&lfs->gstate, parent.pair)) {
moveid = lfs_tag_id(lfs->gstate.tag);
LFS_DEBUG("Fixing move while relocating "
"%"PRIx32" %"PRIx32" %"PRIx16"\n",
"{0x%"PRIx32", 0x%"PRIx32"} 0x%"PRIx16"\n",
parent.pair[0], parent.pair[1], moveid);
lfs_fs_prepmove(lfs, 0x3ff, NULL);
if (moveid < lfs_tag_id(tag)) {
@@ -3969,7 +4050,7 @@ static int lfs_fs_relocate(lfs_t *lfs,
lfs_pair_tole32(newpair);
int err = lfs_dir_commit(lfs, &parent, LFS_MKATTRS(
{LFS_MKTAG_IF(moveid != 0x3ff,
LFS_TYPE_DELETE, moveid, 0)},
LFS_TYPE_DELETE, moveid, 0), NULL},
{tag, newpair}));
lfs_pair_fromle32(newpair);
if (err) {
@@ -3994,7 +4075,7 @@ static int lfs_fs_relocate(lfs_t *lfs,
if (lfs_gstate_hasmovehere(&lfs->gstate, parent.pair)) {
moveid = lfs_tag_id(lfs->gstate.tag);
LFS_DEBUG("Fixing move while relocating "
"%"PRIx32" %"PRIx32" %"PRIx16"\n",
"{0x%"PRIx32", 0x%"PRIx32"} 0x%"PRIx16"\n",
parent.pair[0], parent.pair[1], moveid);
lfs_fs_prepmove(lfs, 0x3ff, NULL);
}
@@ -4003,7 +4084,7 @@ static int lfs_fs_relocate(lfs_t *lfs,
lfs_pair_tole32(newpair);
err = lfs_dir_commit(lfs, &parent, LFS_MKATTRS(
{LFS_MKTAG_IF(moveid != 0x3ff,
LFS_TYPE_DELETE, moveid, 0)},
LFS_TYPE_DELETE, moveid, 0), NULL},
{LFS_MKTAG(LFS_TYPE_TAIL + parent.split, 0x3ff, 8), newpair}));
lfs_pair_fromle32(newpair);
if (err) {
@@ -4035,7 +4116,7 @@ static int lfs_fs_demove(lfs_t *lfs) {
}
// Fix bad moves
LFS_DEBUG("Fixing move %"PRIx32" %"PRIx32" %"PRIx16,
LFS_DEBUG("Fixing move {0x%"PRIx32", 0x%"PRIx32"} 0x%"PRIx16,
lfs->gdisk.pair[0],
lfs->gdisk.pair[1],
lfs_tag_id(lfs->gdisk.tag));
@@ -4051,7 +4132,7 @@ static int lfs_fs_demove(lfs_t *lfs) {
uint16_t moveid = lfs_tag_id(lfs->gdisk.tag);
lfs_fs_prepmove(lfs, 0x3ff, NULL);
err = lfs_dir_commit(lfs, &movedir, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_DELETE, moveid, 0)}));
{LFS_MKTAG(LFS_TYPE_DELETE, moveid, 0), NULL}));
if (err) {
return err;
}
@@ -4086,7 +4167,7 @@ static int lfs_fs_deorphan(lfs_t *lfs) {
if (tag == LFS_ERR_NOENT) {
// we are an orphan
LFS_DEBUG("Fixing orphan %"PRIx32" %"PRIx32,
LFS_DEBUG("Fixing orphan {0x%"PRIx32", 0x%"PRIx32"}",
pdir.tail[0], pdir.tail[1]);
err = lfs_dir_drop(lfs, &pdir, &dir);
@@ -4108,8 +4189,8 @@ static int lfs_fs_deorphan(lfs_t *lfs) {
if (!lfs_pair_sync(pair, pdir.tail)) {
// we have desynced
LFS_DEBUG("Fixing half-orphan "
"%"PRIx32" %"PRIx32" -> %"PRIx32" %"PRIx32,
LFS_DEBUG("Fixing half-orphan {0x%"PRIx32", 0x%"PRIx32"} "
"-> {0x%"PRIx32", 0x%"PRIx32"}",
pdir.tail[0], pdir.tail[1], pair[0], pair[1]);
lfs_pair_tole32(pair);
@@ -4157,7 +4238,7 @@ static int lfs_fs_size_count(void *p, lfs_block_t block) {
lfs_ssize_t lfs_fs_size(lfs_t *lfs) {
LFS_TRACE("lfs_fs_size(%p)", (void*)lfs);
lfs_size_t size = 0;
int err = lfs_fs_traverse(lfs, lfs_fs_size_count, &size);
int err = lfs_fs_traverseraw(lfs, lfs_fs_size_count, &size, false);
if (err) {
LFS_TRACE("lfs_fs_size -> %d", err);
return err;
@@ -4345,7 +4426,7 @@ static int lfs1_dir_fetch(lfs_t *lfs,
continue;
}
uint32_t crc = LFS_BLOCK_NULL;
uint32_t crc = 0xffffffff;
lfs1_dir_tole32(&test);
lfs1_crc(&crc, &test, sizeof(test));
lfs1_dir_fromle32(&test);
@@ -4372,7 +4453,7 @@ static int lfs1_dir_fetch(lfs_t *lfs,
}
if (!valid) {
LFS_ERROR("Corrupted dir pair at %" PRIx32 " %" PRIx32 ,
LFS_ERROR("Corrupted dir pair at {0x%"PRIx32", 0x%"PRIx32"}",
tpair[0], tpair[1]);
return LFS_ERR_CORRUPT;
}
@@ -4560,7 +4641,8 @@ static int lfs1_mount(lfs_t *lfs, struct lfs1 *lfs1,
}
if (err || memcmp(superblock.d.magic, "littlefs", 8) != 0) {
LFS_ERROR("Invalid superblock at %d %d", 0, 1);
LFS_ERROR("Invalid superblock at {0x%"PRIx32", 0x%"PRIx32"}",
0, 1);
err = LFS_ERR_CORRUPT;
goto cleanup;
}
@@ -4569,7 +4651,7 @@ static int lfs1_mount(lfs_t *lfs, struct lfs1 *lfs1,
uint16_t minor_version = (0xffff & (superblock.d.version >> 0));
if ((major_version != LFS1_DISK_VERSION_MAJOR ||
minor_version > LFS1_DISK_VERSION_MINOR)) {
LFS_ERROR("Invalid version %d.%d", major_version, minor_version);
LFS_ERROR("Invalid version v%d.%d", major_version, minor_version);
err = LFS_ERR_INVAL;
goto cleanup;
}
@@ -4735,7 +4817,8 @@ int lfs_migrate(lfs_t *lfs, const struct lfs_config *cfg) {
// Copy over first block to thread into fs. Unfortunately
// if this fails there is not much we can do.
LFS_DEBUG("Migrating %"PRIx32" %"PRIx32" -> %"PRIx32" %"PRIx32,
LFS_DEBUG("Migrating {0x%"PRIx32", 0x%"PRIx32"} "
"-> {0x%"PRIx32", 0x%"PRIx32"}",
lfs->root[0], lfs->root[1], dir1.head[0], dir1.head[1]);
err = lfs_bd_erase(lfs, dir1.head[1]);
@@ -4781,7 +4864,7 @@ int lfs_migrate(lfs_t *lfs, const struct lfs_config *cfg) {
dir2.pair[1] = dir1.pair[1];
dir2.rev = dir1.d.rev;
dir2.off = sizeof(dir2.rev);
dir2.etag = LFS_BLOCK_NULL;
dir2.etag = 0xffffffff;
dir2.count = 0;
dir2.tail[0] = lfs->lfs1->root[0];
dir2.tail[1] = lfs->lfs1->root[1];

2
lfs.h
View File

@@ -21,7 +21,7 @@ extern "C"
// Software library version
// Major (top-nibble), incremented on backwards incompatible changes
// Minor (bottom-nibble), incremented on feature additions
#define LFS_VERSION 0x00020001
#define LFS_VERSION 0x00020002
#define LFS_VERSION_MAJOR (0xffff & (LFS_VERSION >> 16))
#define LFS_VERSION_MINOR (0xffff & (LFS_VERSION >> 0))

View File

@@ -50,31 +50,35 @@ extern "C"
// Logging functions
#ifdef LFS_YES_TRACE
#define LFS_TRACE(fmt, ...) \
printf("%s:%d:trace: " fmt "\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_TRACE_(fmt, ...) \
printf("%s:%d:trace: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_TRACE(...) LFS_TRACE_(__VA_ARGS__, "")
#else
#define LFS_TRACE(fmt, ...)
#define LFS_TRACE(...)
#endif
#ifndef LFS_NO_DEBUG
#define LFS_DEBUG(fmt, ...) \
printf("%s:%d:debug: " fmt "\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_DEBUG_(fmt, ...) \
printf("%s:%d:debug: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_DEBUG(...) LFS_DEBUG_(__VA_ARGS__, "")
#else
#define LFS_DEBUG(fmt, ...)
#define LFS_DEBUG(...)
#endif
#ifndef LFS_NO_WARN
#define LFS_WARN(fmt, ...) \
printf("%s:%d:warn: " fmt "\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_WARN_(fmt, ...) \
printf("%s:%d:warn: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_WARN(...) LFS_WARN_(__VA_ARGS__, "")
#else
#define LFS_WARN(fmt, ...)
#define LFS_WARN(...)
#endif
#ifndef LFS_NO_ERROR
#define LFS_ERROR(fmt, ...) \
printf("%s:%d:error: " fmt "\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_ERROR_(fmt, ...) \
printf("%s:%d:error: " fmt "%s\n", __FILE__, __LINE__, __VA_ARGS__)
#define LFS_ERROR(...) LFS_ERROR_(__VA_ARGS__, "")
#else
#define LFS_ERROR(fmt, ...)
#define LFS_ERROR(...)
#endif
// Runtime assertions
@@ -107,7 +111,7 @@ static inline uint32_t lfs_alignup(uint32_t a, uint32_t alignment) {
return lfs_aligndown(a + alignment-1, alignment);
}
// Find the next smallest power of 2 less than or equal to a
// Find the smallest power of 2 greater than or equal to a
static inline uint32_t lfs_npw2(uint32_t a) {
#if !defined(LFS_NO_INTRINSICS) && (defined(__GNUC__) || defined(__CC_ARM))
return 32 - __builtin_clz(a-1);

View File

@@ -166,8 +166,8 @@ def mkassert(type, comp, lh, rh, size=None):
'type': type.lower(), 'TYPE': type.upper(),
'comp': comp.lower(), 'COMP': comp.upper(),
'prefix': PREFIX.lower(), 'PREFIX': PREFIX.upper(),
'lh': lh.strip(),
'rh': rh.strip(),
'lh': lh.strip(' '),
'rh': rh.strip(' '),
'size': size,
}
if size:

View File

@@ -2,6 +2,7 @@
import struct
import binascii
import sys
import itertools as it
TAG_TYPES = {
@@ -232,8 +233,8 @@ class MetadataPair:
def __lt__(self, other):
# corrupt blocks don't count
if not self and other:
return True
if not self or not other:
return bool(other)
# use sequence arithmetic to avoid overflow
return not ((other.rev - self.rev) & 0x80000000)
@@ -271,37 +272,39 @@ class MetadataPair:
raise KeyError(gmask, gtag)
def _dump_tags(self, tags, truncate=True):
sys.stdout.write("%-8s %-8s %-13s %4s %4s %s\n" % (
'off', 'tag', 'type', 'id', 'len',
'data (truncated)' if truncate else 12*' '+'data'))
def _dump_tags(self, tags, f=sys.stdout, truncate=True):
f.write("%-8s %-8s %-13s %4s %4s" % (
'off', 'tag', 'type', 'id', 'len'))
if truncate:
f.write(' data (truncated)')
f.write('\n')
for tag in tags:
sys.stdout.write("%08x: %08x %-13s %4s %4s" % (
f.write("%08x: %08x %-13s %4s %4s" % (
tag.off, tag,
tag.typerepr(), tag.idrepr(), tag.sizerepr()))
if truncate:
sys.stdout.write(" %-23s %-8s\n" % (
f.write(" %-23s %-8s\n" % (
' '.join('%02x' % c for c in tag.data[:8]),
''.join(c if c >= ' ' and c <= '~' else '.'
for c in map(chr, tag.data[:8]))))
else:
sys.stdout.write("\n")
f.write("\n")
for i in range(0, len(tag.data), 16):
sys.stdout.write("%08x: %-47s %-16s\n" % (
f.write(" %08x: %-47s %-16s\n" % (
tag.off+i,
' '.join('%02x' % c for c in tag.data[i:i+16]),
''.join(c if c >= ' ' and c <= '~' else '.'
for c in map(chr, tag.data[i:i+16]))))
def dump_tags(self, truncate=True):
self._dump_tags(self.tags, truncate=truncate)
def dump_tags(self, f=sys.stdout, truncate=True):
self._dump_tags(self.tags, f=f, truncate=truncate)
def dump_log(self, truncate=True):
self._dump_tags(self.log, truncate=truncate)
def dump_log(self, f=sys.stdout, truncate=True):
self._dump_tags(self.log, f=f, truncate=truncate)
def dump_all(self, truncate=True):
self._dump_tags(self.all_, truncate=truncate)
def dump_all(self, f=sys.stdout, truncate=True):
self._dump_tags(self.all_, f=f, truncate=truncate)
def main(args):
blocks = []
@@ -315,6 +318,24 @@ def main(args):
# find most recent pair
mdir = MetadataPair(blocks)
try:
mdir.tail = mdir[Tag('tail', 0, 0)]
if mdir.tail.size != 8 or mdir.tail.data == 8*b'\xff':
mdir.tail = None
except KeyError:
mdir.tail = None
print("mdir {%s} rev %d%s%s%s" % (
', '.join('%#x' % b
for b in [args.block1, args.block2]
if b is not None),
mdir.rev,
' (was %s)' % ', '.join('%d' % m.rev for m in mdir.pair[1:])
if len(mdir.pair) > 1 else '',
' (corrupted!)' if not mdir else '',
' -> {%#x, %#x}' % struct.unpack('<II', mdir.tail.data)
if mdir.tail else ''))
if args.all:
mdir.dump_all(truncate=not args.no_truncate)
elif args.log:
@@ -337,10 +358,10 @@ if __name__ == "__main__":
help="First block address for finding the metadata pair.")
parser.add_argument('block2', nargs='?', type=lambda x: int(x, 0),
help="Second block address for finding the metadata pair.")
parser.add_argument('-a', '--all', action='store_true',
help="Show all tags in log, included tags in corrupted commits.")
parser.add_argument('-l', '--log', action='store_true',
help="Show tags in log.")
parser.add_argument('-a', '--all', action='store_true',
help="Show all tags in log, included tags in corrupted commits.")
parser.add_argument('-T', '--no-truncate', action='store_true',
help="Don't truncate large amounts of data in tags.")
help="Don't truncate large amounts of data.")
sys.exit(main(parser.parse_args()))

View File

@@ -7,120 +7,24 @@ import io
import itertools as it
from readmdir import Tag, MetadataPair
def popc(x):
return bin(x).count('1')
def ctz(x):
return len(bin(x)) - len(bin(x).rstrip('0'))
def dumptags(args, mdir, f):
if args.all:
tags = mdir.all_
elif args.log:
tags = mdir.log
else:
tags = mdir.tags
for k, tag in enumerate(tags):
f.write("tag %08x %s" % (tag, tag.typerepr()))
if tag.id != 0x3ff:
f.write(" id %d" % tag.id)
if tag.size != 0x3ff:
f.write(" size %d" % tag.size)
if tag.is_('name'):
f.write(" name %s" %
json.dumps(tag.data.decode('utf8')))
if tag.is_('dirstruct'):
f.write(" dir {%#x, %#x}" % struct.unpack(
'<II', tag.data[:8].ljust(8, b'\xff')))
if tag.is_('ctzstruct'):
f.write(" ctz {%#x} size %d" % struct.unpack(
'<II', tag.data[:8].ljust(8, b'\xff')))
if tag.is_('inlinestruct'):
f.write(" inline size %d" % tag.size)
if tag.is_('gstate'):
f.write(" 0x%s" % ''.join('%02x' % c for c in tag.data))
if tag.is_('tail'):
f.write(" tail {%#x, %#x}" % struct.unpack(
'<II', tag.data[:8].ljust(8, b'\xff')))
f.write("\n")
if args.data:
for i in range(0, len(tag.data), 16):
f.write(" %-47s %-16s\n" % (
' '.join('%02x' % c for c in tag.data[i:i+16]),
''.join(c if c >= ' ' and c <= '~' else '.'
for c in map(chr, tag.data[i:i+16]))))
def dumpentries(args, mdir, f):
for k, id_ in enumerate(mdir.ids):
name = mdir[Tag('name', id_, 0)]
struct_ = mdir[Tag('struct', id_, 0)]
f.write("id %d %s %s" % (
id_, name.typerepr(),
json.dumps(name.data.decode('utf8'))))
if struct_.is_('dirstruct'):
f.write(" dir {%#x, %#x}" % struct.unpack(
'<II', struct_.data[:8].ljust(8, b'\xff')))
if struct_.is_('ctzstruct'):
f.write(" ctz {%#x} size %d" % struct.unpack(
'<II', struct_.data[:8].ljust(8, b'\xff')))
if struct_.is_('inlinestruct'):
f.write(" inline size %d" % struct_.size)
f.write("\n")
if args.data and struct_.is_('inlinestruct'):
for i in range(0, len(struct_.data), 16):
f.write(" %-47s %-16s\n" % (
' '.join('%02x' % c for c in struct_.data[i:i+16]),
''.join(c if c >= ' ' and c <= '~' else '.'
for c in map(chr, struct_.data[i:i+16]))))
elif args.data and struct_.is_('ctzstruct'):
block, size = struct.unpack(
'<II', struct_.data[:8].ljust(8, b'\xff'))
data = []
i = 0 if size == 0 else (size-1) // (args.block_size - 8)
if i != 0:
i = ((size-1) - 4*popc(i-1)+2) // (args.block_size - 8)
with open(args.disk, 'rb') as f2:
while i >= 0:
f2.seek(block * args.block_size)
dat = f2.read(args.block_size)
data.append(dat[4*(ctz(i)+1) if i != 0 else 0:])
block, = struct.unpack('<I', dat[:4].ljust(4, b'\xff'))
i -= 1
data = bytes(it.islice(
it.chain.from_iterable(reversed(data)), size))
for i in range(0, min(len(data), 256)
if not args.no_truncate else len(data), 16):
f.write(" %-47s %-16s\n" % (
' '.join('%02x' % c for c in data[i:i+16]),
''.join(c if c >= ' ' and c <= '~' else '.'
for c in map(chr, data[i:i+16]))))
for tag in mdir.tags:
if tag.id==id_ and tag.is_('userattr'):
f.write("id %d %s size %d\n" % (
id_, tag.typerepr(), tag.size))
if args.data:
for i in range(0, len(tag.data), 16):
f.write(" %-47s %-16s\n" % (
' '.join('%02x' % c for c in tag.data[i:i+16]),
''.join(c if c >= ' ' and c <= '~' else '.'
for c in map(chr, tag.data[i:i+16]))))
def main(args):
superblock = None
gstate = b'\0\0\0\0\0\0\0\0\0\0\0\0'
dirs = []
mdirs = []
corrupted = []
cycle = False
with open(args.disk, 'rb') as f:
dirs = []
superblock = None
gstate = b''
mdirs = []
tail = (args.block1, args.block2)
hard = False
while True:
for m in it.chain((m for d in dirs for m in d), mdirs):
if set(m.blocks) == set(tail):
# cycle detected
cycle = m.blocks
if cycle:
break
# load mdir
data = []
blocks = {}
@@ -129,6 +33,7 @@ def main(args):
data.append(f.read(args.block_size)
.ljust(args.block_size, b'\xff'))
blocks[id(data[-1])] = block
mdir = MetadataPair(data)
mdir.blocks = tuple(blocks[id(p.data)] for p in mdir.pair)
@@ -156,6 +61,10 @@ def main(args):
except KeyError:
pass
# corrupted?
if not mdir:
corrupted.append(mdir)
# add to directories
mdirs.append(mdir)
if mdir.tail is None or not mdir.tail.is_('hardtail'):
@@ -171,7 +80,7 @@ def main(args):
# find paths
dirtable = {}
for dir in dirs:
dirtable[tuple(sorted(dir[0].blocks))] = dir
dirtable[frozenset(dir[0].blocks)] = dir
pending = [("/", dirs[0])]
while pending:
@@ -183,67 +92,72 @@ def main(args):
npath = tag.data.decode('utf8')
dirstruct = mdir[Tag('dirstruct', tag.id, 0)]
nblocks = struct.unpack('<II', dirstruct.data)
nmdir = dirtable[tuple(sorted(nblocks))]
nmdir = dirtable[frozenset(nblocks)]
pending.append(((path + '/' + npath), nmdir))
except KeyError:
pass
dir[0].path = path.replace('//', '/')
# dump tree
if not args.superblock and not args.gstate and not args.mdirs:
args.superblock = True
args.gstate = True
args.mdirs = True
# print littlefs + version info
version = ('?', '?')
if superblock:
version = tuple(reversed(
struct.unpack('<HH', superblock[1].data[0:4].ljust(4, b'\xff'))))
print("%-47s%s" % ("littlefs v%s.%s" % version,
"data (truncated, if it fits)"
if not any([args.no_truncate, args.tags, args.log, args.all]) else ""))
if args.superblock and superblock:
print("superblock %s v%d.%d" % (
json.dumps(superblock[0].data.decode('utf8')),
struct.unpack('<H', superblock[1].data[2:2+2])[0],
struct.unpack('<H', superblock[1].data[0:0+2])[0]))
print(
" block_size %d\n"
" block_count %d\n"
" name_max %d\n"
" file_max %d\n"
" attr_max %d" % struct.unpack(
'<IIIII', superblock[1].data[4:4+20].ljust(20, b'\xff')))
# print gstate
print("gstate 0x%s" % ''.join('%02x' % c for c in gstate))
tag = Tag(struct.unpack('<I', gstate[0:4].ljust(4, b'\xff'))[0])
blocks = struct.unpack('<II', gstate[4:4+8].ljust(8, b'\xff'))
if tag.size or not tag.isvalid:
print(" orphans >=%d" % max(tag.size, 1))
if tag.type:
print(" move dir {%#x, %#x} id %d" % (
blocks[0], blocks[1], tag.id))
if args.gstate and gstate:
print("gstate 0x%s" % ''.join('%02x' % c for c in gstate))
tag = Tag(struct.unpack('<I', gstate[0:4].ljust(4, b'\xff'))[0])
blocks = struct.unpack('<II', gstate[4:4+8].ljust(8, b'\xff'))
if tag.size:
print(" orphans %d" % tag.size)
if tag.type:
print(" move dir {%#x, %#x} id %d" % (
blocks[0], blocks[1], tag.id))
# print mdir info
for i, dir in enumerate(dirs):
print("dir %s" % (json.dumps(dir[0].path)
if hasattr(dir[0], 'path') else '(orphan)'))
if args.mdirs:
for i, dir in enumerate(dirs):
print("dir %s" % (json.dumps(dir[0].path)
if hasattr(dir[0], 'path') else '(orphan)'))
for j, mdir in enumerate(dir):
print("mdir {%#x, %#x} rev %d (was %d)%s%s" % (
mdir.blocks[0], mdir.blocks[1], mdir.rev, mdir.pair[1].rev,
' (corrupted!)' if not mdir else '',
' -> {%#x, %#x}' % struct.unpack('<II', mdir.tail.data)
if mdir.tail else ''))
for j, mdir in enumerate(dir):
print("mdir {%#x, %#x} rev %d%s" % (
mdir.blocks[0], mdir.blocks[1], mdir.rev,
' (corrupted)' if not mdir else ''))
f = io.StringIO()
if args.log:
mdir.dump_log(f, truncate=not args.no_truncate)
elif args.all:
mdir.dump_all(f, truncate=not args.no_truncate)
else:
mdir.dump_tags(f, truncate=not args.no_truncate)
f = io.StringIO()
if args.tags or args.all or args.log:
dumptags(args, mdir, f)
else:
dumpentries(args, mdir, f)
lines = list(filter(None, f.getvalue().split('\n')))
for k, line in enumerate(lines):
print("%s %s" % (
' ' if j == len(dir)-1 else
'v' if k == len(lines)-1 else
'|',
line))
lines = list(filter(None, f.getvalue().split('\n')))
for k, line in enumerate(lines):
print("%s %s" % (
' ' if j == len(dir)-1 else
'v' if k == len(lines)-1 else
'|',
line))
errcode = 0
for mdir in corrupted:
errcode = errcode or 1
print("*** corrupted mdir {%#x, %#x}! ***" % (
mdir.blocks[0], mdir.blocks[1]))
return 0 if all(mdir for dir in dirs for mdir in dir) else 1
if cycle:
errcode = errcode or 2
print("*** cycle detected {%#x, %#x}! ***" % (
cycle[0], cycle[1]))
return errcode
if __name__ == "__main__":
import argparse
@@ -256,24 +170,14 @@ if __name__ == "__main__":
help="Size of a block in bytes.")
parser.add_argument('block1', nargs='?', default=0,
type=lambda x: int(x, 0),
help="Optional first block address for finding the root.")
help="Optional first block address for finding the superblock.")
parser.add_argument('block2', nargs='?', default=1,
type=lambda x: int(x, 0),
help="Optional second block address for finding the root.")
parser.add_argument('-s', '--superblock', action='store_true',
help="Show contents of the superblock.")
parser.add_argument('-g', '--gstate', action='store_true',
help="Show contents of global-state.")
parser.add_argument('-m', '--mdirs', action='store_true',
help="Show contents of metadata-pairs/directories.")
parser.add_argument('-t', '--tags', action='store_true',
help="Show metadata tags instead of reconstructing entries.")
parser.add_argument('-a', '--all', action='store_true',
help="Show all tags in log, included tags in corrupted commits.")
help="Optional second block address for finding the superblock.")
parser.add_argument('-l', '--log', action='store_true',
help="Show tags in log.")
parser.add_argument('-d', '--data', action='store_true',
help="Also show the raw contents of files/attrs/tags.")
parser.add_argument('-a', '--all', action='store_true',
help="Show all tags in log, included tags in corrupted commits.")
parser.add_argument('-T', '--no-truncate', action='store_true',
help="Don't truncate large amounts of data in files.")
help="Show the full contents of files/attrs/tags.")
sys.exit(main(parser.parse_args()))

View File

@@ -52,7 +52,7 @@ DEFINES = {
'LFS_LOOKAHEAD_SIZE': 16,
'LFS_ERASE_VALUE': 0xff,
'LFS_ERASE_CYCLES': 0,
'LFS_BADBLOCK_BEHAVIOR': 'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_BADBLOCK_BEHAVIOR': 'LFS_TESTBD_BADBLOCK_PROGERROR',
}
PROLOGUE = """
// prologue
@@ -182,27 +182,43 @@ class TestCase:
elif args.get('no_internal', False) and self.in_ is not None:
return False
elif self.if_ is not None:
return eval(self.if_, None, self.defines.copy())
if_ = self.if_
while True:
for k, v in sorted(self.defines.items(),
key=lambda x: len(x[0]), reverse=True):
if k in if_:
if_ = if_.replace(k, '(%s)' % v)
break
else:
break
if_ = (
re.sub('(\&\&|\?)', ' and ',
re.sub('(\|\||:)', ' or ',
re.sub('!(?!=)', ' not ', if_))))
return eval(if_)
else:
return True
def test(self, exec=[], persist=False, cycles=None,
gdb=False, failure=None, **args):
gdb=False, failure=None, disk=None, **args):
# build command
cmd = exec + ['./%s.test' % self.suite.path,
repr(self.caseno), repr(self.permno)]
# persist disk or keep in RAM for speed?
if persist:
if not disk:
disk = self.suite.path + '.disk'
if persist != 'noerase':
try:
os.remove(self.suite.path + '.disk')
with open(disk, 'w') as f:
f.truncate(0)
if args.get('verbose', False):
print('rm', self.suite.path + '.disk')
print('truncate --size=0', disk)
except FileNotFoundError:
pass
cmd.append(self.suite.path + '.disk')
cmd.append(disk)
# simulate power-loss after n cycles?
if cycles:
@@ -215,7 +231,7 @@ class TestCase:
ncmd.extend(['-ex', 'r'])
if failure.assert_:
ncmd.extend(['-ex', 'up 2'])
elif gdb == 'start':
elif gdb == 'main':
ncmd.extend([
'-ex', 'b %s:%d' % (self.suite.path, self.code_lineno),
'-ex', 'r'])
@@ -235,33 +251,37 @@ class TestCase:
mpty = os.fdopen(mpty, 'r', 1)
stdout = []
assert_ = None
while True:
try:
line = mpty.readline()
except OSError as e:
if e.errno == errno.EIO:
break
raise
stdout.append(line)
if args.get('verbose', False):
sys.stdout.write(line)
# intercept asserts
m = re.match(
'^{0}([^:]+):(\d+):(?:\d+:)?{0}{1}:{0}(.*)$'
.format('(?:\033\[[\d;]*.| )*', 'assert'),
line)
if m and assert_ is None:
try:
while True:
try:
with open(m.group(1)) as f:
lineno = int(m.group(2))
line = next(it.islice(f, lineno-1, None)).strip('\n')
assert_ = {
'path': m.group(1),
'line': line,
'lineno': lineno,
'message': m.group(3)}
except:
pass
line = mpty.readline()
except OSError as e:
if e.errno == errno.EIO:
break
raise
stdout.append(line)
if args.get('verbose', False):
sys.stdout.write(line)
# intercept asserts
m = re.match(
'^{0}([^:]+):(\d+):(?:\d+:)?{0}{1}:{0}(.*)$'
.format('(?:\033\[[\d;]*.| )*', 'assert'),
line)
if m and assert_ is None:
try:
with open(m.group(1)) as f:
lineno = int(m.group(2))
line = (next(it.islice(f, lineno-1, None))
.strip('\n'))
assert_ = {
'path': m.group(1),
'line': line,
'lineno': lineno,
'message': m.group(3)}
except:
pass
except KeyboardInterrupt:
raise TestFailure(self, 1, stdout, None)
proc.wait()
# did we pass?
@@ -279,11 +299,17 @@ class ValgrindTestCase(TestCase):
return not self.leaky and super().shouldtest(**args)
def test(self, exec=[], **args):
exec = exec + [
verbose = args.get('verbose', False)
uninit = (self.defines.get('LFS_ERASE_VALUE', None) == -1)
exec = [
'valgrind',
'--leak-check=full',
] + (['--undef-value-errors=no'] if uninit else []) + [
] + (['--track-origins=yes'] if not uninit else []) + [
'--error-exitcode=4',
'-q']
'--error-limit=no',
] + (['--num-callers=1'] if not verbose else []) + [
'-q'] + exec
return super().test(exec=exec, **args)
class ReentrantTestCase(TestCase):
@@ -294,7 +320,7 @@ class ReentrantTestCase(TestCase):
def shouldtest(self, **args):
return self.reentrant and super().shouldtest(**args)
def test(self, exec=[], persist=False, gdb=False, failure=None, **args):
def test(self, persist=False, gdb=False, failure=None, **args):
for cycles in it.count(1):
# clear disk first?
if cycles == 1 and persist != 'noerase':
@@ -360,10 +386,11 @@ class TestSuite:
# code lineno?
if 'code' in case:
case['code_lineno'] = code_linenos.pop()
# give our case's config a copy of our "global" config
for k, v in config.items():
if k not in case:
case[k] = v
# merge conditions if necessary
if 'if' in config and 'if' in case:
case['if'] = '(%s) && (%s)' % (config['if'], case['if'])
elif 'if' in config:
case['if'] = config['if']
# initialize test case
self.cases.append(TestCase(case, filter=filter,
suite=self, caseno=i+1, lineno=lineno, **args))
@@ -654,6 +681,10 @@ def main(**args):
if filtered != sum(len(suite.perms) for suite in suites):
print('filtered down to %d permutations' % filtered)
# only requested to build?
if args.get('build', False):
return 0
print('====== testing ======')
try:
for suite in suites:
@@ -678,18 +709,17 @@ def main(**args):
perm=perm, path=perm.suite.path, lineno=perm.lineno,
returncode=perm.result.returncode or 0))
if perm.result.stdout:
for line in (perm.result.stdout
if not perm.result.assert_
else perm.result.stdout[:-1]):
if perm.result.assert_:
stdout = perm.result.stdout[:-1]
else:
stdout = perm.result.stdout
for line in stdout[-5:]:
sys.stdout.write(line)
if perm.result.assert_:
sys.stdout.write(
"\033[01m{path}:{lineno}:\033[01;31massert:\033[m "
"{message}\n{line}\n".format(
**perm.result.assert_))
else:
for line in perm.result.stdout:
sys.stdout.write(line)
sys.stdout.write('\n')
failed += 1
@@ -728,7 +758,9 @@ if __name__ == "__main__":
parser.add_argument('-p', '--persist', choices=['erase', 'noerase'],
nargs='?', const='erase',
help="Store disk image in a file.")
parser.add_argument('-g', '--gdb', choices=['init', 'start', 'assert'],
parser.add_argument('-b', '--build', action='store_true',
help="Only build the tests, do not execute.")
parser.add_argument('-g', '--gdb', choices=['init', 'main', 'assert'],
nargs='?', const='assert',
help="Drop into gdb on test failure.")
parser.add_argument('--no-internal', action='store_true',
@@ -741,4 +773,6 @@ if __name__ == "__main__":
help="Run non-leaky tests under valgrind to check for memory leaks.")
parser.add_argument('-e', '--exec', default=[], type=lambda e: e.split(' '),
help="Run tests with another executable prefixed on the command line.")
parser.add_argument('-d', '--disk',
help="Specify a file to use for persistent/reentrant tests.")
sys.exit(main(**vars(parser.parse_args())))

View File

@@ -1,9 +1,10 @@
# allocator tests
# note for these to work there are many constraints on the device geometry
# note for these to work there are a number constraints on the device geometry
if = 'LFS_BLOCK_CYCLES == -1'
[[case]] # parallel allocation test
define.FILES = 3
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-4)) / FILES)'
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-6)) / FILES)'
code = '''
const char *names[FILES] = {"bacon", "eggs", "pancakes"};
lfs_file_t files[FILES];
@@ -46,7 +47,7 @@ code = '''
[[case]] # serial allocation test
define.FILES = 3
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-4)) / FILES)'
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-6)) / FILES)'
code = '''
const char *names[FILES] = {"bacon", "eggs", "pancakes"};
@@ -85,7 +86,7 @@ code = '''
[[case]] # parallel allocation reuse test
define.FILES = 3
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-4)) / FILES)'
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-6)) / FILES)'
define.CYCLES = [1, 10]
code = '''
const char *names[FILES] = {"bacon", "eggs", "pancakes"};
@@ -140,7 +141,7 @@ code = '''
[[case]] # serial allocation reuse test
define.FILES = 3
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-4)) / FILES)'
define.SIZE = '(((LFS_BLOCK_SIZE-8)*(LFS_BLOCK_COUNT-6)) / FILES)'
define.CYCLES = [1, 10]
code = '''
const char *names[FILES] = {"bacon", "eggs", "pancakes"};
@@ -322,6 +323,90 @@ code = '''
lfs_unmount(&lfs) => 0;
'''
[[case]] # what if we have a bad block during an allocation scan?
in = "lfs.c"
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_BADBLOCK_BEHAVIOR = 'LFS_TESTBD_BADBLOCK_READERROR'
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
// first fill to exhaustion to find available space
lfs_file_open(&lfs, &file, "pacman", LFS_O_WRONLY | LFS_O_CREAT) => 0;
strcpy((char*)buffer, "waka");
size = strlen("waka");
lfs_size_t filesize = 0;
while (true) {
lfs_ssize_t res = lfs_file_write(&lfs, &file, buffer, size);
assert(res == (lfs_ssize_t)size || res == LFS_ERR_NOSPC);
if (res == LFS_ERR_NOSPC) {
break;
}
filesize += size;
}
lfs_file_close(&lfs, &file) => 0;
// now fill all but a couple of blocks of the filesystem with data
filesize -= 3*LFS_BLOCK_SIZE;
lfs_file_open(&lfs, &file, "pacman", LFS_O_WRONLY | LFS_O_CREAT) => 0;
strcpy((char*)buffer, "waka");
size = strlen("waka");
for (lfs_size_t i = 0; i < filesize/size; i++) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
// also save head of file so we can error during lookahead scan
lfs_block_t fileblock = file.ctz.head;
lfs_unmount(&lfs) => 0;
// remount to force an alloc scan
lfs_mount(&lfs, &cfg) => 0;
// but mark the head of our file as a "bad block", this is force our
// scan to bail early
lfs_testbd_setwear(&cfg, fileblock, 0xffffffff) => 0;
lfs_file_open(&lfs, &file, "ghost", LFS_O_WRONLY | LFS_O_CREAT) => 0;
strcpy((char*)buffer, "chomp");
size = strlen("chomp");
while (true) {
lfs_ssize_t res = lfs_file_write(&lfs, &file, buffer, size);
assert(res == (lfs_ssize_t)size || res == LFS_ERR_CORRUPT);
if (res == LFS_ERR_CORRUPT) {
break;
}
}
lfs_file_close(&lfs, &file) => 0;
// now reverse the "bad block" and try to write the file again until we
// run out of space
lfs_testbd_setwear(&cfg, fileblock, 0) => 0;
lfs_file_open(&lfs, &file, "ghost", LFS_O_WRONLY | LFS_O_CREAT) => 0;
strcpy((char*)buffer, "chomp");
size = strlen("chomp");
while (true) {
lfs_ssize_t res = lfs_file_write(&lfs, &file, buffer, size);
assert(res == (lfs_ssize_t)size || res == LFS_ERR_NOSPC);
if (res == LFS_ERR_NOSPC) {
break;
}
}
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
// check that the disk isn't hurt
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "pacman", LFS_O_RDONLY) => 0;
strcpy((char*)buffer, "waka");
size = strlen("waka");
for (lfs_size_t i = 0; i < filesize/size; i++) {
uint8_t rbuffer[4];
lfs_file_read(&lfs, &file, rbuffer, size) => size;
assert(memcmp(rbuffer, buffer, size) == 0);
}
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
'''
# Below, I don't like these tests. They're fragile and depend _heavily_
# on the geometry of the block device. But they are valuable. Eventually they
# should be removed and replaced with generalized tests.
@@ -329,7 +414,7 @@ code = '''
[[case]] # chained dir exhaustion test
define.LFS_BLOCK_SIZE = 512
define.LFS_BLOCK_COUNT = 1024
if = 'LFS_BLOCK_SIZE == 512 and LFS_BLOCK_COUNT == 1024'
if = 'LFS_BLOCK_SIZE == 512 && LFS_BLOCK_COUNT == 1024'
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -400,7 +485,7 @@ code = '''
[[case]] # split dir test
define.LFS_BLOCK_SIZE = 512
define.LFS_BLOCK_COUNT = 1024
if = 'LFS_BLOCK_SIZE == 512 and LFS_BLOCK_COUNT == 1024'
if = 'LFS_BLOCK_SIZE == 512 && LFS_BLOCK_COUNT == 1024'
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -445,7 +530,7 @@ code = '''
[[case]] # outdated lookahead test
define.LFS_BLOCK_SIZE = 512
define.LFS_BLOCK_COUNT = 1024
if = 'LFS_BLOCK_SIZE == 512 and LFS_BLOCK_COUNT == 1024'
if = 'LFS_BLOCK_SIZE == 512 && LFS_BLOCK_COUNT == 1024'
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -510,7 +595,7 @@ code = '''
[[case]] # outdated lookahead and split dir test
define.LFS_BLOCK_SIZE = 512
define.LFS_BLOCK_COUNT = 1024
if = 'LFS_BLOCK_SIZE == 512 and LFS_BLOCK_COUNT == 1024'
if = 'LFS_BLOCK_SIZE == 512 && LFS_BLOCK_COUNT == 1024'
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;

View File

@@ -1,6 +1,17 @@
# bad blocks with block cycles should be tested in test_relocations
if = 'LFS_BLOCK_CYCLES == -1'
[[case]] # single bad blocks
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_BADBLOCK_BEHAVIOR = 'LFS_TESTBD_BADBLOCK_NOPROG'
define.LFS_ERASE_VALUE = [0x00, 0xff, -1]
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_PROGERROR',
'LFS_TESTBD_BADBLOCK_ERASEERROR',
'LFS_TESTBD_BADBLOCK_READERROR',
'LFS_TESTBD_BADBLOCK_PROGNOOP',
'LFS_TESTBD_BADBLOCK_ERASENOOP',
]
define.NAMEMULT = 64
define.FILEMULT = 1
code = '''
@@ -64,144 +75,16 @@ code = '''
}
'''
[[case]] # single persistent blocks (can't erase)
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_BADBLOCK_BEHAVIOR = 'LFS_TESTBD_BADBLOCK_NOERASE'
define.NAMEMULT = 64
define.FILEMULT = 1
code = '''
for (lfs_block_t badblock = 2; badblock < LFS_BLOCK_COUNT; badblock++) {
lfs_testbd_setwear(&cfg, badblock-1, 0) => 0;
lfs_testbd_setwear(&cfg, badblock, 0xffffffff) => 0;
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
for (int i = 1; i < 10; i++) {
for (int j = 0; j < NAMEMULT; j++) {
buffer[j] = '0'+i;
}
buffer[NAMEMULT] = '\0';
lfs_mkdir(&lfs, (char*)buffer) => 0;
buffer[NAMEMULT] = '/';
for (int j = 0; j < NAMEMULT; j++) {
buffer[j+NAMEMULT+1] = '0'+i;
}
buffer[2*NAMEMULT+1] = '\0';
lfs_file_open(&lfs, &file, (char*)buffer,
LFS_O_WRONLY | LFS_O_CREAT) => 0;
size = NAMEMULT;
for (int j = 0; j < i*FILEMULT; j++) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, &cfg) => 0;
for (int i = 1; i < 10; i++) {
for (int j = 0; j < NAMEMULT; j++) {
buffer[j] = '0'+i;
}
buffer[NAMEMULT] = '\0';
lfs_stat(&lfs, (char*)buffer, &info) => 0;
info.type => LFS_TYPE_DIR;
buffer[NAMEMULT] = '/';
for (int j = 0; j < NAMEMULT; j++) {
buffer[j+NAMEMULT+1] = '0'+i;
}
buffer[2*NAMEMULT+1] = '\0';
lfs_file_open(&lfs, &file, (char*)buffer, LFS_O_RDONLY) => 0;
size = NAMEMULT;
for (int j = 0; j < i*FILEMULT; j++) {
uint8_t rbuffer[1024];
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(buffer, rbuffer, size) => 0;
}
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
}
'''
[[case]] # single unreadable blocks (can't read)
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_BADBLOCK_BEHAVIOR = 'LFS_TESTBD_BADBLOCK_NOREAD'
define.NAMEMULT = 64
define.FILEMULT = 1
code = '''
for (lfs_block_t badblock = 2; badblock < LFS_BLOCK_COUNT/2; badblock++) {
lfs_testbd_setwear(&cfg, badblock-1, 0) => 0;
lfs_testbd_setwear(&cfg, badblock, 0xffffffff) => 0;
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
for (int i = 1; i < 10; i++) {
for (int j = 0; j < NAMEMULT; j++) {
buffer[j] = '0'+i;
}
buffer[NAMEMULT] = '\0';
lfs_mkdir(&lfs, (char*)buffer) => 0;
buffer[NAMEMULT] = '/';
for (int j = 0; j < NAMEMULT; j++) {
buffer[j+NAMEMULT+1] = '0'+i;
}
buffer[2*NAMEMULT+1] = '\0';
lfs_file_open(&lfs, &file, (char*)buffer,
LFS_O_WRONLY | LFS_O_CREAT) => 0;
size = NAMEMULT;
for (int j = 0; j < i*FILEMULT; j++) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, &cfg) => 0;
for (int i = 1; i < 10; i++) {
for (int j = 0; j < NAMEMULT; j++) {
buffer[j] = '0'+i;
}
buffer[NAMEMULT] = '\0';
lfs_stat(&lfs, (char*)buffer, &info) => 0;
info.type => LFS_TYPE_DIR;
buffer[NAMEMULT] = '/';
for (int j = 0; j < NAMEMULT; j++) {
buffer[j+NAMEMULT+1] = '0'+i;
}
buffer[2*NAMEMULT+1] = '\0';
lfs_file_open(&lfs, &file, (char*)buffer, LFS_O_RDONLY) => 0;
size = NAMEMULT;
for (int j = 0; j < i*FILEMULT; j++) {
uint8_t rbuffer[1024];
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(buffer, rbuffer, size) => 0;
}
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
}
'''
[[case]] # region corruption (causes cascading failures)
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_ERASE_VALUE = [0x00, 0xff, -1]
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_TESTBD_BADBLOCK_NOERASE',
'LFS_TESTBD_BADBLOCK_NOREAD',
'LFS_TESTBD_BADBLOCK_PROGERROR',
'LFS_TESTBD_BADBLOCK_ERASEERROR',
'LFS_TESTBD_BADBLOCK_READERROR',
'LFS_TESTBD_BADBLOCK_PROGNOOP',
'LFS_TESTBD_BADBLOCK_ERASENOOP',
]
define.NAMEMULT = 64
define.FILEMULT = 1
@@ -266,11 +149,15 @@ code = '''
'''
[[case]] # alternating corruption (causes cascading failures)
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_ERASE_VALUE = [0x00, 0xff, -1]
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_TESTBD_BADBLOCK_NOERASE',
'LFS_TESTBD_BADBLOCK_NOREAD',
'LFS_TESTBD_BADBLOCK_PROGERROR',
'LFS_TESTBD_BADBLOCK_ERASEERROR',
'LFS_TESTBD_BADBLOCK_READERROR',
'LFS_TESTBD_BADBLOCK_PROGNOOP',
'LFS_TESTBD_BADBLOCK_ERASENOOP',
]
define.NAMEMULT = 64
define.FILEMULT = 1
@@ -337,10 +224,13 @@ code = '''
# other corner cases
[[case]] # bad superblocks (corrupt 1 or 0)
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_ERASE_VALUE = [0x00, 0xff, -1]
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_TESTBD_BADBLOCK_NOERASE',
'LFS_TESTBD_BADBLOCK_NOREAD',
'LFS_TESTBD_BADBLOCK_PROGERROR',
'LFS_TESTBD_BADBLOCK_ERASEERROR',
'LFS_TESTBD_BADBLOCK_READERROR',
'LFS_TESTBD_BADBLOCK_PROGNOOP',
'LFS_TESTBD_BADBLOCK_ERASENOOP',
]
code = '''
lfs_testbd_setwear(&cfg, 0, 0xffffffff) => 0;

View File

@@ -155,7 +155,7 @@ code = '''
'''
[[case]] # reentrant many directory creation/rename/removal
define.N = [5, 25]
define.N = [5, 11]
reentrant = true
code = '''
err = lfs_mount(&lfs, &cfg);

View File

@@ -3,7 +3,7 @@
# still pass with other inline sizes but wouldn't be testing anything.
define.LFS_CACHE_SIZE = 512
if = 'LFS_CACHE_SIZE == 512'
if = 'LFS_CACHE_SIZE % LFS_PROG_SIZE == 0 && LFS_CACHE_SIZE == 512'
[[case]] # entry grow test
code = '''

288
tests/test_evil.toml Normal file
View File

@@ -0,0 +1,288 @@
# Tests for recovering from conditions which shouldn't normally
# happen during normal operation of littlefs
# invalid pointer tests (outside of block_count)
[[case]] # invalid tail-pointer test
define.TAIL_TYPE = ['LFS_TYPE_HARDTAIL', 'LFS_TYPE_SOFTTAIL']
define.INVALSET = [0x3, 0x1, 0x2]
in = "lfs.c"
code = '''
// create littlefs
lfs_format(&lfs, &cfg) => 0;
// change tail-pointer to invalid pointers
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
lfs_dir_commit(&lfs, &mdir, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_HARDTAIL, 0x3ff, 8),
(lfs_block_t[2]){
(INVALSET & 0x1) ? 0xcccccccc : 0,
(INVALSET & 0x2) ? 0xcccccccc : 0}})) => 0;
lfs_deinit(&lfs) => 0;
// test that mount fails gracefully
lfs_mount(&lfs, &cfg) => LFS_ERR_CORRUPT;
'''
[[case]] # invalid dir pointer test
define.INVALSET = [0x3, 0x1, 0x2]
in = "lfs.c"
code = '''
// create littlefs
lfs_format(&lfs, &cfg) => 0;
// make a dir
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "dir_here") => 0;
lfs_unmount(&lfs) => 0;
// change the dir pointer to be invalid
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
// make sure id 1 == our directory
lfs_dir_get(&lfs, &mdir,
LFS_MKTAG(0x700, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_NAME, 1, strlen("dir_here")), buffer)
=> LFS_MKTAG(LFS_TYPE_DIR, 1, strlen("dir_here"));
assert(memcmp((char*)buffer, "dir_here", strlen("dir_here")) == 0);
// change dir pointer
lfs_dir_commit(&lfs, &mdir, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_DIRSTRUCT, 1, 8),
(lfs_block_t[2]){
(INVALSET & 0x1) ? 0xcccccccc : 0,
(INVALSET & 0x2) ? 0xcccccccc : 0}})) => 0;
lfs_deinit(&lfs) => 0;
// test that accessing our bad dir fails, note there's a number
// of ways to access the dir, some can fail, but some don't
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "dir_here", &info) => 0;
assert(strcmp(info.name, "dir_here") == 0);
assert(info.type == LFS_TYPE_DIR);
lfs_dir_open(&lfs, &dir, "dir_here") => LFS_ERR_CORRUPT;
lfs_stat(&lfs, "dir_here/file_here", &info) => LFS_ERR_CORRUPT;
lfs_dir_open(&lfs, &dir, "dir_here/dir_here") => LFS_ERR_CORRUPT;
lfs_file_open(&lfs, &file, "dir_here/file_here",
LFS_O_RDONLY) => LFS_ERR_CORRUPT;
lfs_file_open(&lfs, &file, "dir_here/file_here",
LFS_O_WRONLY | LFS_O_CREAT) => LFS_ERR_CORRUPT;
lfs_unmount(&lfs) => 0;
'''
[[case]] # invalid file pointer test
in = "lfs.c"
define.SIZE = [10, 1000, 100000] # faked file size
code = '''
// create littlefs
lfs_format(&lfs, &cfg) => 0;
// make a file
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "file_here",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
// change the file pointer to be invalid
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
// make sure id 1 == our file
lfs_dir_get(&lfs, &mdir,
LFS_MKTAG(0x700, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_NAME, 1, strlen("file_here")), buffer)
=> LFS_MKTAG(LFS_TYPE_REG, 1, strlen("file_here"));
assert(memcmp((char*)buffer, "file_here", strlen("file_here")) == 0);
// change file pointer
lfs_dir_commit(&lfs, &mdir, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_CTZSTRUCT, 1, sizeof(struct lfs_ctz)),
&(struct lfs_ctz){0xcccccccc, lfs_tole32(SIZE)}})) => 0;
lfs_deinit(&lfs) => 0;
// test that accessing our bad file fails, note there's a number
// of ways to access the dir, some can fail, but some don't
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "file_here", &info) => 0;
assert(strcmp(info.name, "file_here") == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
lfs_file_open(&lfs, &file, "file_here", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, buffer, SIZE) => LFS_ERR_CORRUPT;
lfs_file_close(&lfs, &file) => 0;
// any allocs that traverse CTZ must unfortunately must fail
if (SIZE > 2*LFS_BLOCK_SIZE) {
lfs_mkdir(&lfs, "dir_here") => LFS_ERR_CORRUPT;
}
lfs_unmount(&lfs) => 0;
'''
[[case]] # invalid pointer in CTZ skip-list test
define.SIZE = ['2*LFS_BLOCK_SIZE', '3*LFS_BLOCK_SIZE', '4*LFS_BLOCK_SIZE']
in = "lfs.c"
code = '''
// create littlefs
lfs_format(&lfs, &cfg) => 0;
// make a file
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "file_here",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (int i = 0; i < SIZE; i++) {
char c = 'c';
lfs_file_write(&lfs, &file, &c, 1) => 1;
}
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
// change pointer in CTZ skip-list to be invalid
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
// make sure id 1 == our file and get our CTZ structure
lfs_dir_get(&lfs, &mdir,
LFS_MKTAG(0x700, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_NAME, 1, strlen("file_here")), buffer)
=> LFS_MKTAG(LFS_TYPE_REG, 1, strlen("file_here"));
assert(memcmp((char*)buffer, "file_here", strlen("file_here")) == 0);
struct lfs_ctz ctz;
lfs_dir_get(&lfs, &mdir,
LFS_MKTAG(0x700, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_STRUCT, 1, sizeof(struct lfs_ctz)), &ctz)
=> LFS_MKTAG(LFS_TYPE_CTZSTRUCT, 1, sizeof(struct lfs_ctz));
lfs_ctz_fromle32(&ctz);
// rewrite block to contain bad pointer
uint8_t bbuffer[LFS_BLOCK_SIZE];
cfg.read(&cfg, ctz.head, 0, bbuffer, LFS_BLOCK_SIZE) => 0;
uint32_t bad = lfs_tole32(0xcccccccc);
memcpy(&bbuffer[0], &bad, sizeof(bad));
memcpy(&bbuffer[4], &bad, sizeof(bad));
cfg.erase(&cfg, ctz.head) => 0;
cfg.prog(&cfg, ctz.head, 0, bbuffer, LFS_BLOCK_SIZE) => 0;
lfs_deinit(&lfs) => 0;
// test that accessing our bad file fails, note there's a number
// of ways to access the dir, some can fail, but some don't
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "file_here", &info) => 0;
assert(strcmp(info.name, "file_here") == 0);
assert(info.type == LFS_TYPE_REG);
assert(info.size == SIZE);
lfs_file_open(&lfs, &file, "file_here", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, buffer, SIZE) => LFS_ERR_CORRUPT;
lfs_file_close(&lfs, &file) => 0;
// any allocs that traverse CTZ must unfortunately must fail
if (SIZE > 2*LFS_BLOCK_SIZE) {
lfs_mkdir(&lfs, "dir_here") => LFS_ERR_CORRUPT;
}
lfs_unmount(&lfs) => 0;
'''
[[case]] # invalid gstate pointer
define.INVALSET = [0x3, 0x1, 0x2]
in = "lfs.c"
code = '''
// create littlefs
lfs_format(&lfs, &cfg) => 0;
// create an invalid gstate
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
lfs_fs_prepmove(&lfs, 1, (lfs_block_t [2]){
(INVALSET & 0x1) ? 0xcccccccc : 0,
(INVALSET & 0x2) ? 0xcccccccc : 0});
lfs_dir_commit(&lfs, &mdir, NULL, 0) => 0;
lfs_deinit(&lfs) => 0;
// test that mount fails gracefully
// mount may not fail, but our first alloc should fail when
// we try to fix the gstate
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "should_fail") => LFS_ERR_CORRUPT;
lfs_unmount(&lfs) => 0;
'''
# cycle detection/recovery tests
[[case]] # metadata-pair threaded-list loop test
in = "lfs.c"
code = '''
// create littlefs
lfs_format(&lfs, &cfg) => 0;
// change tail-pointer to point to ourself
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
lfs_dir_commit(&lfs, &mdir, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_HARDTAIL, 0x3ff, 8),
(lfs_block_t[2]){0, 1}})) => 0;
lfs_deinit(&lfs) => 0;
// test that mount fails gracefully
lfs_mount(&lfs, &cfg) => LFS_ERR_CORRUPT;
'''
[[case]] # metadata-pair threaded-list 2-length loop test
in = "lfs.c"
code = '''
// create littlefs with child dir
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "child") => 0;
lfs_unmount(&lfs) => 0;
// find child
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_block_t pair[2];
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
lfs_dir_get(&lfs, &mdir,
LFS_MKTAG(0x7ff, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_DIRSTRUCT, 1, sizeof(pair)), pair)
=> LFS_MKTAG(LFS_TYPE_DIRSTRUCT, 1, sizeof(pair));
lfs_pair_fromle32(pair);
// change tail-pointer to point to root
lfs_dir_fetch(&lfs, &mdir, pair) => 0;
lfs_dir_commit(&lfs, &mdir, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_HARDTAIL, 0x3ff, 8),
(lfs_block_t[2]){0, 1}})) => 0;
lfs_deinit(&lfs) => 0;
// test that mount fails gracefully
lfs_mount(&lfs, &cfg) => LFS_ERR_CORRUPT;
'''
[[case]] # metadata-pair threaded-list 1-length child loop test
in = "lfs.c"
code = '''
// create littlefs with child dir
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "child") => 0;
lfs_unmount(&lfs) => 0;
// find child
lfs_init(&lfs, &cfg) => 0;
lfs_mdir_t mdir;
lfs_block_t pair[2];
lfs_dir_fetch(&lfs, &mdir, (lfs_block_t[2]){0, 1}) => 0;
lfs_dir_get(&lfs, &mdir,
LFS_MKTAG(0x7ff, 0x3ff, 0),
LFS_MKTAG(LFS_TYPE_DIRSTRUCT, 1, sizeof(pair)), pair)
=> LFS_MKTAG(LFS_TYPE_DIRSTRUCT, 1, sizeof(pair));
lfs_pair_fromle32(pair);
// change tail-pointer to point to ourself
lfs_dir_fetch(&lfs, &mdir, pair) => 0;
lfs_dir_commit(&lfs, &mdir, LFS_MKATTRS(
{LFS_MKTAG(LFS_TYPE_HARDTAIL, 0x3ff, 8), pair})) => 0;
lfs_deinit(&lfs) => 0;
// test that mount fails gracefully
lfs_mount(&lfs, &cfg) => LFS_ERR_CORRUPT;
'''

View File

@@ -1,11 +1,13 @@
[[case]] # test running a filesystem to exhaustion
define.LFS_ERASE_CYCLES = 10
define.LFS_BLOCK_COUNT = 256 # small bd so it runs faster
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_BLOCK_CYCLES = 'LFS_ERASE_CYCLES / 2'
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_TESTBD_BADBLOCK_NOERASE',
'LFS_TESTBD_BADBLOCK_NOREAD',
'LFS_TESTBD_BADBLOCK_PROGERROR',
'LFS_TESTBD_BADBLOCK_ERASEERROR',
'LFS_TESTBD_BADBLOCK_READERROR',
'LFS_TESTBD_BADBLOCK_PROGNOOP',
'LFS_TESTBD_BADBLOCK_ERASENOOP',
]
define.FILES = 10
code = '''
@@ -31,6 +33,9 @@ code = '''
lfs_ssize_t res = lfs_file_write(&lfs, &file, &c, 1);
assert(res == 1 || res == LFS_ERR_NOSPC);
if (res == LFS_ERR_NOSPC) {
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
@@ -38,9 +43,10 @@ code = '''
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
if (err == LFS_ERR_NOSPC) {
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
}
for (uint32_t i = 0; i < FILES; i++) {
// check for errors
@@ -57,7 +63,7 @@ code = '''
}
lfs_file_close(&lfs, &file) => 0;
}
}
lfs_unmount(&lfs) => 0;
cycle += 1;
@@ -70,7 +76,7 @@ exhausted:
// check for errors
sprintf(path, "roadrunner/test%d", i);
lfs_stat(&lfs, path, &info) => 0;
}
}
lfs_unmount(&lfs) => 0;
LFS_WARN("completed %d cycles", cycle);
@@ -79,12 +85,14 @@ exhausted:
[[case]] # test running a filesystem to exhaustion
# which also requires expanding superblocks
define.LFS_ERASE_CYCLES = 10
define.LFS_BLOCK_COUNT = 256 # small bd so it runs faster
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_BLOCK_CYCLES = 'LFS_ERASE_CYCLES / 2'
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_TESTBD_BADBLOCK_NOERASE',
'LFS_TESTBD_BADBLOCK_NOREAD',
'LFS_TESTBD_BADBLOCK_PROGERROR',
'LFS_TESTBD_BADBLOCK_ERASEERROR',
'LFS_TESTBD_BADBLOCK_READERROR',
'LFS_TESTBD_BADBLOCK_PROGNOOP',
'LFS_TESTBD_BADBLOCK_ERASENOOP',
]
define.FILES = 10
code = '''
@@ -107,6 +115,9 @@ code = '''
lfs_ssize_t res = lfs_file_write(&lfs, &file, &c, 1);
assert(res == 1 || res == LFS_ERR_NOSPC);
if (res == LFS_ERR_NOSPC) {
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
@@ -114,9 +125,10 @@ code = '''
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
if (err == LFS_ERR_NOSPC) {
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
}
for (uint32_t i = 0; i < FILES; i++) {
// check for errors
@@ -133,7 +145,7 @@ code = '''
}
lfs_file_close(&lfs, &file) => 0;
}
}
lfs_unmount(&lfs) => 0;
cycle += 1;
@@ -146,7 +158,7 @@ exhausted:
// check for errors
sprintf(path, "test%d", i);
lfs_stat(&lfs, path, &info) => 0;
}
}
lfs_unmount(&lfs) => 0;
LFS_WARN("completed %d cycles", cycle);
@@ -158,21 +170,11 @@ exhausted:
# check for.
[[case]] # wear-level test running a filesystem to exhaustion
define.LFS_ERASE_CYCLES = 10
define.LFS_BLOCK_COUNT = 256 # small bd so it runs faster
define.LFS_ERASE_CYCLES = 20
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_BLOCK_CYCLES = 'LFS_ERASE_CYCLES / 2'
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_TESTBD_BADBLOCK_NOERASE',
'LFS_TESTBD_BADBLOCK_NOREAD',
]
define.FILES = 10
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "roadrunner") => 0;
lfs_unmount(&lfs) => 0;
uint32_t run_cycles[2];
const uint32_t run_block_count[2] = {LFS_BLOCK_COUNT/2, LFS_BLOCK_COUNT};
@@ -182,6 +184,11 @@ code = '''
(b < run_block_count[run]) ? 0 : LFS_ERASE_CYCLES) => 0;
}
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "roadrunner") => 0;
lfs_unmount(&lfs) => 0;
uint32_t cycle = 0;
while (true) {
lfs_mount(&lfs, &cfg) => 0;
@@ -199,6 +206,9 @@ code = '''
lfs_ssize_t res = lfs_file_write(&lfs, &file, &c, 1);
assert(res == 1 || res == LFS_ERR_NOSPC);
if (res == LFS_ERR_NOSPC) {
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
@@ -206,9 +216,10 @@ code = '''
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
if (err == LFS_ERR_NOSPC) {
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
}
for (uint32_t i = 0; i < FILES; i++) {
// check for errors
@@ -225,7 +236,7 @@ code = '''
}
lfs_file_close(&lfs, &file) => 0;
}
}
lfs_unmount(&lfs) => 0;
cycle += 1;
@@ -238,7 +249,7 @@ exhausted:
// check for errors
sprintf(path, "roadrunner/test%d", i);
lfs_stat(&lfs, path, &info) => 0;
}
}
lfs_unmount(&lfs) => 0;
run_cycles[run] = cycle;
@@ -247,22 +258,15 @@ exhausted:
}
// check we increased the lifetime by 2x with ~10% error
LFS_ASSERT(run_cycles[1] > 2*run_cycles[0]-run_cycles[0]/10);
LFS_ASSERT(run_cycles[1]*110/100 > 2*run_cycles[0]);
'''
[[case]] # wear-level test + expanding superblock
define.LFS_ERASE_CYCLES = 10
define.LFS_BLOCK_COUNT = 256 # small bd so it runs faster
define.LFS_ERASE_CYCLES = 20
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_BLOCK_CYCLES = 'LFS_ERASE_CYCLES / 2'
define.LFS_BADBLOCK_BEHAVIOR = [
'LFS_TESTBD_BADBLOCK_NOPROG',
'LFS_TESTBD_BADBLOCK_NOERASE',
'LFS_TESTBD_BADBLOCK_NOREAD',
]
define.FILES = 10
code = '''
lfs_format(&lfs, &cfg) => 0;
uint32_t run_cycles[2];
const uint32_t run_block_count[2] = {LFS_BLOCK_COUNT/2, LFS_BLOCK_COUNT};
@@ -272,6 +276,8 @@ code = '''
(b < run_block_count[run]) ? 0 : LFS_ERASE_CYCLES) => 0;
}
lfs_format(&lfs, &cfg) => 0;
uint32_t cycle = 0;
while (true) {
lfs_mount(&lfs, &cfg) => 0;
@@ -289,6 +295,9 @@ code = '''
lfs_ssize_t res = lfs_file_write(&lfs, &file, &c, 1);
assert(res == 1 || res == LFS_ERR_NOSPC);
if (res == LFS_ERR_NOSPC) {
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
@@ -296,9 +305,10 @@ code = '''
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
if (err == LFS_ERR_NOSPC) {
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
}
for (uint32_t i = 0; i < FILES; i++) {
// check for errors
@@ -315,7 +325,7 @@ code = '''
}
lfs_file_close(&lfs, &file) => 0;
}
}
lfs_unmount(&lfs) => 0;
cycle += 1;
@@ -328,7 +338,7 @@ exhausted:
// check for errors
sprintf(path, "test%d", i);
lfs_stat(&lfs, path, &info) => 0;
}
}
lfs_unmount(&lfs) => 0;
run_cycles[run] = cycle;
@@ -337,5 +347,119 @@ exhausted:
}
// check we increased the lifetime by 2x with ~10% error
LFS_ASSERT(run_cycles[1] > 2*run_cycles[0]-run_cycles[0]/10);
LFS_ASSERT(run_cycles[1]*110/100 > 2*run_cycles[0]);
'''
[[case]] # test that we wear blocks roughly evenly
define.LFS_ERASE_CYCLES = 0xffffffff
define.LFS_BLOCK_COUNT = 256 # small bd so test runs faster
define.LFS_BLOCK_CYCLES = [5, 4, 3, 2, 1]
define.CYCLES = 100
define.FILES = 10
if = 'LFS_BLOCK_CYCLES < CYCLES/10'
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "roadrunner") => 0;
lfs_unmount(&lfs) => 0;
uint32_t cycle = 0;
while (cycle < CYCLES) {
lfs_mount(&lfs, &cfg) => 0;
for (uint32_t i = 0; i < FILES; i++) {
// chose name, roughly random seed, and random 2^n size
sprintf(path, "roadrunner/test%d", i);
srand(cycle * i);
size = 1 << 4; //((rand() % 10)+2);
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC) => 0;
for (lfs_size_t j = 0; j < size; j++) {
char c = 'a' + (rand() % 26);
lfs_ssize_t res = lfs_file_write(&lfs, &file, &c, 1);
assert(res == 1 || res == LFS_ERR_NOSPC);
if (res == LFS_ERR_NOSPC) {
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
err = lfs_file_close(&lfs, &file);
assert(err == 0 || err == LFS_ERR_NOSPC);
if (err == LFS_ERR_NOSPC) {
lfs_unmount(&lfs) => 0;
goto exhausted;
}
}
for (uint32_t i = 0; i < FILES; i++) {
// check for errors
sprintf(path, "roadrunner/test%d", i);
srand(cycle * i);
size = 1 << 4; //((rand() % 10)+2);
lfs_file_open(&lfs, &file, path, LFS_O_RDONLY) => 0;
for (lfs_size_t j = 0; j < size; j++) {
char c = 'a' + (rand() % 26);
char r;
lfs_file_read(&lfs, &file, &r, 1) => 1;
assert(r == c);
}
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
cycle += 1;
}
exhausted:
// should still be readable
lfs_mount(&lfs, &cfg) => 0;
for (uint32_t i = 0; i < FILES; i++) {
// check for errors
sprintf(path, "roadrunner/test%d", i);
lfs_stat(&lfs, path, &info) => 0;
}
lfs_unmount(&lfs) => 0;
LFS_WARN("completed %d cycles", cycle);
// check the wear on our block device
lfs_testbd_wear_t minwear = -1;
lfs_testbd_wear_t totalwear = 0;
lfs_testbd_wear_t maxwear = 0;
// skip 0 and 1 as superblock movement is intentionally avoided
for (lfs_block_t b = 2; b < LFS_BLOCK_COUNT; b++) {
lfs_testbd_wear_t wear = lfs_testbd_getwear(&cfg, b);
printf("%08x: wear %d\n", b, wear);
assert(wear >= 0);
if (wear < minwear) {
minwear = wear;
}
if (wear > maxwear) {
maxwear = wear;
}
totalwear += wear;
}
lfs_testbd_wear_t avgwear = totalwear / LFS_BLOCK_COUNT;
LFS_WARN("max wear: %d cycles", maxwear);
LFS_WARN("avg wear: %d cycles", totalwear / LFS_BLOCK_COUNT);
LFS_WARN("min wear: %d cycles", minwear);
// find standard deviation^2
lfs_testbd_wear_t dev2 = 0;
for (lfs_block_t b = 2; b < LFS_BLOCK_COUNT; b++) {
lfs_testbd_wear_t wear = lfs_testbd_getwear(&cfg, b);
assert(wear >= 0);
lfs_testbd_swear_t diff = wear - avgwear;
dev2 += diff*diff;
}
dev2 /= totalwear;
LFS_WARN("std dev^2: %d", dev2);
assert(dev2 < 8);
'''

View File

@@ -148,6 +148,7 @@ code = '''
[[case]] # move file corrupt source and dest
in = "lfs.c"
if = 'LFS_PROG_SIZE <= 0x3fe' # only works with one crc per commit
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -239,6 +240,7 @@ code = '''
[[case]] # move file after corrupt
in = "lfs.c"
if = 'LFS_PROG_SIZE <= 0x3fe' # only works with one crc per commit
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -593,6 +595,7 @@ code = '''
[[case]] # move dir corrupt source and dest
in = "lfs.c"
if = 'LFS_PROG_SIZE <= 0x3fe' # only works with one crc per commit
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -692,6 +695,7 @@ code = '''
[[case]] # move dir after corrupt
in = "lfs.c"
if = 'LFS_PROG_SIZE <= 0x3fe' # only works with one crc per commit
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;

View File

@@ -1,5 +1,6 @@
[[case]] # orphan test
in = "lfs.c"
if = 'LFS_PROG_SIZE <= 0x3fe' # only works with one crc per commit
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -57,10 +58,12 @@ code = '''
[[case]] # reentrant testing for orphans, basically just spam mkdir/remove
reentrant = true
# TODO fix this case, caused by non-DAG trees
if = '!(DEPTH == 3 && LFS_CACHE_SIZE != 64)'
define = [
{FILES=6, DEPTH=1, CYCLES=50},
{FILES=26, DEPTH=1, CYCLES=50},
{FILES=3, DEPTH=3, CYCLES=50},
{FILES=6, DEPTH=1, CYCLES=20},
{FILES=26, DEPTH=1, CYCLES=20},
{FILES=3, DEPTH=3, CYCLES=20},
]
code = '''
err = lfs_mount(&lfs, &cfg);

View File

@@ -247,14 +247,14 @@ code = '''
lfs_mkdir(&lfs, "coffee/coldcoffee") => 0;
memset(path, 'w', LFS_NAME_MAX+1);
path[LFS_NAME_MAX+2] = '\0';
path[LFS_NAME_MAX+1] = '\0';
lfs_mkdir(&lfs, path) => LFS_ERR_NAMETOOLONG;
lfs_file_open(&lfs, &file, path, LFS_O_WRONLY | LFS_O_CREAT)
=> LFS_ERR_NAMETOOLONG;
memcpy(path, "coffee/", strlen("coffee/"));
memset(path+strlen("coffee/"), 'w', LFS_NAME_MAX+1);
path[strlen("coffee/")+LFS_NAME_MAX+2] = '\0';
path[strlen("coffee/")+LFS_NAME_MAX+1] = '\0';
lfs_mkdir(&lfs, path) => LFS_ERR_NAMETOOLONG;
lfs_file_open(&lfs, &file, path, LFS_O_WRONLY | LFS_O_CREAT)
=> LFS_ERR_NAMETOOLONG;
@@ -270,7 +270,6 @@ code = '''
lfs_mkdir(&lfs, "coffee/warmcoffee") => 0;
lfs_mkdir(&lfs, "coffee/coldcoffee") => 0;
lfs_mount(&lfs, &cfg) => 0;
memset(path, 'w', LFS_NAME_MAX);
path[LFS_NAME_MAX] = '\0';
lfs_mkdir(&lfs, path) => 0;

View File

@@ -147,10 +147,12 @@ code = '''
# orphan testing, except here we also set block_cycles so that
# almost every tree operation needs a relocation
reentrant = true
# TODO fix this case, caused by non-DAG trees
if = '!(DEPTH == 3 && LFS_CACHE_SIZE != 64)'
define = [
{FILES=6, DEPTH=1, CYCLES=50, LFS_BLOCK_CYCLES=1},
{FILES=26, DEPTH=1, CYCLES=50, LFS_BLOCK_CYCLES=1},
{FILES=3, DEPTH=3, CYCLES=50, LFS_BLOCK_CYCLES=1},
{FILES=6, DEPTH=1, CYCLES=20, LFS_BLOCK_CYCLES=1},
{FILES=26, DEPTH=1, CYCLES=20, LFS_BLOCK_CYCLES=1},
{FILES=3, DEPTH=3, CYCLES=20, LFS_BLOCK_CYCLES=1},
]
code = '''
err = lfs_mount(&lfs, &cfg);
@@ -207,10 +209,12 @@ code = '''
[[case]] # reentrant testing for relocations, but now with random renames!
reentrant = true
# TODO fix this case, caused by non-DAG trees
if = '!(DEPTH == 3 && LFS_CACHE_SIZE != 64)'
define = [
{FILES=6, DEPTH=1, CYCLES=50, LFS_BLOCK_CYCLES=1},
{FILES=26, DEPTH=1, CYCLES=50, LFS_BLOCK_CYCLES=1},
{FILES=3, DEPTH=3, CYCLES=50, LFS_BLOCK_CYCLES=1},
{FILES=6, DEPTH=1, CYCLES=20, LFS_BLOCK_CYCLES=1},
{FILES=26, DEPTH=1, CYCLES=20, LFS_BLOCK_CYCLES=1},
{FILES=3, DEPTH=3, CYCLES=20, LFS_BLOCK_CYCLES=1},
]
code = '''
err = lfs_mount(&lfs, &cfg);

View File

@@ -27,41 +27,55 @@ code = '''
'''
[[case]] # expanding superblock
define.BLOCK_CYCLES = [32, 33, 1]
define.LFS_BLOCK_CYCLES = [32, 33, 1]
define.N = [10, 100, 1000]
code = '''
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
for (int i = 0; i < N; i++) {
lfs_mkdir(&lfs, "dummy") => 0;
lfs_file_open(&lfs, &file, "dummy",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_stat(&lfs, "dummy", &info) => 0;
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
lfs_remove(&lfs, "dummy") => 0;
}
lfs_unmount(&lfs) => 0;
// one last check after power-cycle
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "dummy") => 0;
lfs_file_open(&lfs, &file, "dummy",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_stat(&lfs, "dummy", &info) => 0;
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
lfs_unmount(&lfs) => 0;
'''
[[case]] # expanding superblock with power cycle
define.BLOCK_CYCLES = [32, 33, 1]
define.LFS_BLOCK_CYCLES = [32, 33, 1]
define.N = [10, 100, 1000]
code = '''
lfs_format(&lfs, &cfg) => 0;
for (int i = 0; i < N; i++) {
lfs_mount(&lfs, &cfg) => 0;
// remove lingering dummy?
err = lfs_remove(&lfs, "dummy");
err = lfs_stat(&lfs, "dummy", &info);
assert(err == 0 || (err == LFS_ERR_NOENT && i == 0));
lfs_mkdir(&lfs, "dummy") => 0;
if (!err) {
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
lfs_remove(&lfs, "dummy") => 0;
}
lfs_file_open(&lfs, &file, "dummy",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_stat(&lfs, "dummy", &info) => 0;
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
lfs_unmount(&lfs) => 0;
}
@@ -69,11 +83,12 @@ code = '''
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "dummy", &info) => 0;
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
lfs_unmount(&lfs) => 0;
'''
[[case]] # reentrant expanding superblock
define.BLOCK_CYCLES = [2, 1]
define.LFS_BLOCK_CYCLES = [2, 1]
define.N = 24
reentrant = true
code = '''
@@ -85,12 +100,20 @@ code = '''
for (int i = 0; i < N; i++) {
// remove lingering dummy?
err = lfs_remove(&lfs, "dummy");
err = lfs_stat(&lfs, "dummy", &info);
assert(err == 0 || (err == LFS_ERR_NOENT && i == 0));
lfs_mkdir(&lfs, "dummy") => 0;
if (!err) {
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
lfs_remove(&lfs, "dummy") => 0;
}
lfs_file_open(&lfs, &file, "dummy",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_stat(&lfs, "dummy", &info) => 0;
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
}
lfs_unmount(&lfs) => 0;
@@ -99,5 +122,6 @@ code = '''
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "dummy", &info) => 0;
assert(strcmp(info.name, "dummy") == 0);
assert(info.type == LFS_TYPE_REG);
lfs_unmount(&lfs) => 0;
'''

View File

@@ -100,7 +100,7 @@ code = '''
lfs_file_open(&lfs, &file, "sequence",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_TRUNC) => 0;
size = lfs.cfg->cache_size;
size = lfs_min(lfs.cfg->cache_size, sizeof(buffer)/2);
lfs_size_t qsize = size / 4;
uint8_t *wb = buffer;
uint8_t *rb = buffer + size;