Compare commits

..

313 Commits

Author SHA1 Message Date
Christopher Haster
60e67ae080 Fixed implicit change-of-sign warning in lfs_dir_fetch
Warning on MDK v5.27.1
Found by geniusgogo
2019-11-26 16:42:49 -06:00
grunwald-m
64dedee2d1 prepare upstream bugfix of lfs
-> call lfs_dir_fetchmatch with ftag=-1 in order to set the invalid bit
   and never let the function match a dir
2019-11-26 11:48:53 -06:00
Christopher Haster
6b65737715 Merge pull request #308 from roykuper13/readme-example-update-block-cycles
Update readme example code in accordance to the block_cycles change
2019-10-15 10:36:42 -05:00
Christopher Haster
4ebe6030c5 Merge pull request #294 from ARMmbed/fix-max-null-tests
Fixed off-by-one null terminator in tests
2019-10-15 10:36:04 -05:00
Christopher Haster
7ae8d778f1 Merge pull request #299 from sipke/sipke/fix-types-for-16bit-machines-v2
fix types for 16bit machines v2
2019-10-15 10:35:47 -05:00
Roy Kupershmid
4d068a154d Update README example code in accordance to the block_cycles change
An addition to 38a2a8d. When executing the given example in README,
you immediately get an assertion error because block_cycles is initiated to 0.
2019-10-13 20:27:18 +03:00
Sipke Vriend
ba088aa213 lfs_dir_*: Cast error return codes to int.
For correctness, cast the lfs_stag_t variables to int when returning a negative error code.
2019-10-01 15:24:17 +10:00
Sipke Vriend
955b296bcc lfs_file_rewind: Cast error return codes to int.
For correctness, cast the lfs_stag_t variables to int when returning a negative error code.
2019-10-01 14:22:25 +10:00
Sipke Vriend
241dbc6f86 lfs_stat: Cast error return codes to int.
For correctness, cast the lfs_stag_t variables to int when returning a negative error code.
2019-10-01 14:22:01 +10:00
Sipke Vriend
8cca58f1a6 lfs_file_truncate: ensure lfs_file_seek return code is lsf_soff_t and cast error returns
To ensure 16 bit devices do not invalidly truncate lfs_file_write return codes, change
the return variable to be lfs_ssize_t which is the lfs_file_write return code and
cast to int if it is a negative error code.
2019-10-01 14:20:43 +10:00
Sipke Vriend
97f86af4e9 lfs_remove: Cast tag/error return codes to int.
For correctness, cast the lfs_stag_t variables to int when returning a negative error code.
2019-10-01 13:56:51 +10:00
Sipke Vriend
d40302c5e3 lfs_rename: Cast error return codes to int.
For correctness, cast the lfs_stag_t variables to int when returning a negative error code.
2019-10-01 13:51:52 +10:00
Sipke Vriend
0b5a78e2cd Adjust lfs_dir_find return code to ensure 32 bit value.
lfs_dir_find returns either a negative return code or a tag.
For 32 bit machines with int as 32 bits this co-incides, but for smaller
bit processors, we need to ensure a 32 bit value is returned so change
the return type to lfs_stag_t.
2019-10-01 11:52:02 +10:00
Christopher Haster
27b6cc829b Fixed off-by-one null terminator in tests
Found by mr-at-eo
2019-09-23 10:43:39 -05:00
Christopher Haster
fd204ac2fb Merge pull request #278 from roykuper13/validate-lfs-cfg-sizes
lfs: Validate lfs-cfg sizes before performing any arithmetic logics with them
2019-09-19 10:02:54 -05:00
Christopher Haster
bd99402d9a Merge pull request #281 from patrick--/fix-lfs-embud-file-resource-leak
Fix for issue #260
2019-09-19 10:02:42 -05:00
Christopher Haster
bce442a86b Merge pull request #282 from runderwo/master
Corrections for typos and grammar
2019-09-19 10:02:34 -05:00
Christopher Haster
f26e970a0e Merge pull request #286 from sipke/sipke/fix-warnings-shift-count
build: Fix warnings about shift count width difference for 16 bit com…
2019-09-19 10:02:25 -05:00
Sipke Vriend
965d29b887 build: Fix warnings about shift count width difference for 16 bit compiler
Build warnings exist on a gcc based 16 bit compiler. Cast relevant types
to fix.

littlefs/lfs.c: In function 'lfs_gstate_xororphans':
littlefs/lfs.c:355:5: warning: left shift count >= width of type
littlefs/lfs.c: In function 'lfs_dir_fetchmatch':
littlefs/lfs.c:849:17: warning: left shift count >= width of type
littlefs/lfs.c: In function 'lfs_dir_commitcrc':
littlefs/lfs.c:1278:9: warning: left shift count >= width of type
2019-09-09 13:53:50 +10:00
Ryan Underwood
f7fd7d966a Corrections for typos and grammar 2019-09-01 21:11:49 -07:00
Patrick Servello
d5aba27d60 Fix for issue #260
Certain functions within lfs_emubd.c were susceptible to file resource leaks due to certain code paths not issuing an fclose() before returning.
2019-08-31 20:47:26 -05:00
Roy Kupershmid
0c77123eee lfs: Validate lfs-cfg sizes before performing arithmetic logics with them 2019-08-31 16:57:56 +03:00
Christopher Haster
494dd6673d Merge pull request #263 from rojer/wundef
Fix build with -Wundef
2019-08-08 18:50:40 -05:00
Christopher Haster
fce2569005 Merge pull request #257 from pabigot/pr/20190803a
fix seek position corruption in truncate function
2019-08-08 18:50:28 -05:00
Christopher Haster
9d1f1211a9 Merge pull request #253 from pabigot/pr/20190730a
lfs: correct alignment restriction on lookahead buffer
2019-08-08 18:50:15 -05:00
Christopher Haster
151104c790 Changed CI to create release note for patches
This is a result of feedback that the current release notes made it too
difficult to see what changes happened on patch releases. From my
experience as well it became difficult to chase down which release a
commit landed on.

The risk is that this creates additional noise, both for the release
page and for user notifications. I am open to feedback if this causes a
problem.

Other tweaks on the CI side, these came from iteration with the same
scheme for coru and equeue:

- Changed version branch updates to be atomic (vN and vN-prefix). This
  makes it a bit easier to fix if one of the pushes fails due to a rogue
  branch with the same name.

- Added GEKY_BOT_DRAFT as a CI macro that can optionally switch between
  only creating drafts or immediately posting a release. The default is
  what I will be trying with littlefs which is to draft minor/major
  releases, but automatically create patch release.

  The real benefit of automatic releases is to use on tiny repos that
  don't really have an active maintainer. Though this is definitely no
  longer the case with littlefs, and I'm happy it has gained this much
  attention.
2019-08-08 18:50:00 -05:00
Deomid "rojer" Ryabkov
303ffb2da4 Fix build with -Wundef
Part of https://github.com/mongoose-os-libs/vfs-fs-lfs/issues/2
2019-08-08 16:54:34 +01:00
Peter A. Bigot
5bf71fa43e lfs: do not reposition seek pointer on truncate
When using lfs_file_truncate() to make a file shorter the file block and
off were incorrectly positioned at the new end, resulting in invalid
data accessed when reading.  Lift the seek pointer restoration to apply
to both increasing and reducing truncates.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-08-03 17:17:49 -05:00
Peter A. Bigot
55fb1416c7 lfs: initialize file offs field
The uninitialized value creates confusion when diagnosing anomalies.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-08-03 09:59:27 -05:00
Peter A. Bigot
dc031ce1d9 lfs: use meaningful names for magic block identifiers
The difference between 0xffffffff and 0xfffffffe is too subtle.  Use
names that reflect what the value represents.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-08-03 09:59:07 -05:00
Peter A. Bigot
f85ff1d2f8 lfs: correct alignment restriction on lookahead buffer
The buffer need only be 32-bit aligned.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-30 20:02:42 -05:00
Christopher Haster
db054684a6 Bump version to v2.1 2019-07-29 01:42:28 -05:00
Christopher Haster
7872918ec8 Fixed issue where lfs_migrate would relocate root and corrupt superblock
Found during testing, the issue was with lfs_migrate in combination with
wear leveling.

Normally, we can expect lfs_migrate to be able to respect the user-configured
block_cycles. It already has allocation information on which blocks are
used by both v1 and v2, so it should be safe to relocate blocks as
needed.

However, this fell apart when root was relocated. If lfs_migrate found a
root that needed migration, it would happily relocate the root. This
would normally be fine, except relocating the root has a side-effect of
needed to update the superblock. Which, during migration, is in a
delicate state of containing both v1's and v2's superblocks in the same
metadata pair. If the superblock ends up needing to compact, this would
clobber the v1 superblock and corrupt the filesystem during migration.

The best fix I could come up with is to specifically dissallow migrating the
root directory during migration. Fortunately this is behind the
LFS_MIGRATE macro, so the code cost for this check is not normally paid.
2019-07-29 01:42:06 -05:00
Christopher Haster
e249854858 Removed dependency on uninitialized value in lfs_file_t struct 2019-07-29 00:43:54 -05:00
Christopher Haster
501b0240a9 Merge pull request #232 from ARMmbed/debug-improvements
Debug improvements
2019-07-28 21:53:55 -05:00
Christopher Haster
e1f3b90b56 Merge remote-tracking branch 'origin/master' into debug-improvements 2019-07-28 21:53:13 -05:00
Christopher Haster
74fe46de3d Merge pull request #233 from ARMmbed/discourage-no-wear-leveling
Change block_cycles disable from 0 to -1
2019-07-28 21:35:48 -05:00
Christopher Haster
582b596ed1 Merge pull request #242 from ARMmbed/fix-2048-erase-size
Fix issues with large prog sizes (prog_size > 1KiB)
2019-07-28 21:35:22 -05:00
Christopher Haster
0d4c0b105c Fixed issue where inline files were not cleaned up
Due to the logging nature of metadata pairs, switching from inline files
(type3 = 0x201) to CTZ skip-lists (type3 = 0x202) does not explicitly
erase inline files, but instead leaves them up to compaction to omit.
To save code size, this is handled by the same logic that deduplicates
tags.

Unfortunately, this wasn't working. Due to a relatively late change in v2
the struct's type field was changed to no longer be a part of determining a
tag's "uniqueness". A part of this should have been the modification of
directory traversal filtering to respect type-dependent uniqueness, but
I missed this.

The fix is to add in correct type-dependent filtering. Also there was
some clean up necessary around removing delete tags during compaction
and outlining files.

Note that while this appears to conflict with the possibility of
combining inline + ctz files, we still have the device-side-only
LFS_TYPE_FROM tag that can be repurposed for 256 additional inline
"chunks".

Found by Johnxjj
2019-07-28 21:34:17 -05:00
Christopher Haster
4850e01e14 Changed rdonly/wronly mistakes to assert
Previously these returned LFS_ERR_BADF. But attempting to modify a file
opened read-only, or reading a write-only flie, is a user error and
should not occur in normal use.

Changing this to an assert allows the logic to be omitted if the user
disables asserts to reduce the code footprint (not suggested unless the
user really really knows what they're doing).
2019-07-28 21:32:06 -05:00
Christopher Haster
4ec4425272 Fixed overlapping memcpy in emubd
Found by DanielLyubin
2019-07-28 21:26:24 -05:00
Christopher Haster
31e28fddb7 Merge pull request #237 from Ar2rL/reverse_finalize_close
Protect (LFS_ASSERT) file operations against using not opened or closed files.
2019-07-28 21:26:03 -05:00
Christopher Haster
3806d88285 Fixed seek-related typos in lfs.h
- lfs_file_rewind == lfs_file_seek(lfs, file, 0, LFS_SEEK_SET)
- lfs_file_seek returns the _new_ position of the file
2019-07-28 21:25:18 -05:00
Christopher Haster
de5972699a Fixed license header in lfs.c
Found by pabigot
2019-07-28 21:25:00 -05:00
Christopher Haster
0d8ffd6b86 Merge pull request #239 from pabigot/pr/20190723a
lfs: correct documentation on lookahead-related values
2019-07-28 21:24:39 -05:00
Christopher Haster
c0af471bc1 Merge pull request #227 from haneefmubarak/patch-1
removed <dirent.h> preventing compile on some archs
2019-07-28 21:24:22 -05:00
Christopher Haster
e8c023aab0 Changed FUSE branch to v2 (previously v2-alpha) 2019-07-28 20:43:12 -05:00
Christopher Haster
38a2a8d2a3 Minor improvement to documentation over block_cycles
Suggested by haneefmubarak
2019-07-28 20:42:13 -05:00
Christopher Haster
51fabc672b Switched to using hex for blocks and ids in debug output
This is a minor quality of life change to help debugging, specifically
when debugging test failures.

Before, the test framework used hex, while the log output used decimal.
This was slightly annoying to convert between.

Why not output lengths/offset in hex? I don't have a big reason. I find
it easier to reason about lengths in decimal and ids (such as addresses
or block numbers) in hex. But this may just be me.
2019-07-26 20:09:24 -05:00
Christopher Haster
19838371fb Fixed issue where sed buffering (QUIET=1) caused Travis timeout 2019-07-26 19:51:20 -05:00
Christopher Haster
312326c4e4 Added a better solution for large prog sizes
A current limitation of the lfs tag is the 10-bit (1024) length field.
This field is used to indicate padding for commits and effectively
limits the size of commits to 1KiB. Because commits must be prog size
aligned, this is a problem on devices with prog size > 1024.

[----                   6KiB erase block                   ----]
[-- 2KiB prog size --|-- 2KiB prog size --|-- 2KiB prog size --]
[ 1KiB commit |  ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ]

This can be increased to 12-bit (4096), but for NAND devices this is
still to small to completely solve the issue.

The previous workaround was to just create unaligned commits. This can
occur naturally if littlefs is used on portable media as the prog size
does not have to be consistent on different drivers. If littlefs sees
an unaligned commit, it treats the dir as unerased and must compact the
dir if it creates any new commits.

Unfortunately this isn't great. It effectively means that every small
commit forced an erase on devices with prog size > 1024. This is pretty
terrible.

[----                   6KiB erase block                   ----]
[-- 2KiB prog size --|-- 2KiB prog size --|-- 2KiB prog size --]
[ 1KiB commit |------------------- wasted ---------------------]

A different solution, implemented here, is to use multiple crc tags
to pad the commit until the remaining space fits in the padding. This
effectively looks like multiple empty commits and has a small runtime
cost to parse these tags, but otherwise does no harm.

[----                   6KiB erase block                   ----]
[-- 2KiB prog size --|-- 2KiB prog size --|-- 2KiB prog size --]
[ 1KiB commit | noop | 1KiB commit | noop | 1KiB commit | noop ]

It was a bit tricky to implement, but now we can effectively support
unlimited prog sizes since there's no limit to the number of commits
in a block.

found by kazink and joicetm
2019-07-26 19:51:15 -05:00
Christopher Haster
ef1c926940 Increased testing to include geometries that can't be fully tested
This is primarily to get better test coverage over devices with very
large erase/prog/read sizes. The unfortunate state of the tests is
that most of them rely on a specific block device size, so that
ENOSPC and ECORRUPT errors occur in specific situations.

This should be improved in the future, but at least for now we can
open up some of the simpler tests to run on these different
configurations.

Also added testing over both 0x00 and 0xff erase values in emubd.

Also added a number of small file tests that expose issues prevalent
on NAND devices.
2019-07-26 19:50:17 -05:00
Christopher Haster
72e3bb4448 Refactored a handful of things in tests
- Now test errors have correct line reporting! #line directives
  are passed to the compiler that reference the relevant line in
  the test case shell script.

  --- Multi-block directory ---
  ./tests/test_dirs.sh:109: assert failed with 0, expected 1
      lfs_unmount(&lfs) => 1

- Cleaned up the number of implicit global variables provided to
  tests. A lot of these were infrequently used and made it difficult
  to remember what was provided. This isn't an MCU, so there's very
  little cost to stack allocations when needed.

- Minimized the results.py script (previously stats.py) output to
  match minimization of test output.
2019-07-26 11:11:34 -05:00
Christopher Haster
649640c605 Fixed workaround for erase sizes >1024 B
Introduced in 0b76635, the workaround for erases sizes >1024 is to
commit with an unaligned CRC tag. Upon reading an unaligned CRC,
littlefs should treat the metadata pair as "requires erased". While
necessary for portability, this also lets us workaround the lack of
handling of erases sizes >1024.

Unfortunately, this workaround wasn't implemented correctly (by me)
in the case that the metadata-pair does not immediately compact. This
is solved here by added the erase check to lfs_dir_commit.

Note this is still only a part of a workaround which should be replaced.
One potential solution is to pad the commit with multiple smaller CRC
tags until we reach the next prog_size boundary.

found by kazink
2019-07-24 14:45:21 -05:00
Peter A. Bigot
eb013e6dd6 lfs: correct documentation on lookahead-related values
The size of the lookahead buffer is required to be a multiple of 8 bytes
in anticipation of a future improvement.  The buffer itself need only be
aligned to support access through a uint32_t pointer.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-23 11:05:04 -05:00
Ar2rL
7e1bad3eee Set LFS_F_OPENED flag at places required by lfs internal logic. 2019-07-21 14:36:40 +02:00
Ar2rL
72a3758958 Use LFS_F_OPENED flag to protect against use of not opened or closed file. 2019-07-21 11:34:53 +02:00
Ar2rL
df2e676562 Add necessary flag to mark file as being opened. 2019-07-21 11:34:14 +02:00
Christopher Haster
53a6e04712 Changed block_cycles disable from 0 to -1
As it is now, block_cycles = 0 disables wear leveling. This was a
mistake as 0 is the "default" value for several other config options.
It's even worse when migrating from v1 as it's easy to miss the addition
of block_cycles and end up with a filesystem that is not actually
wear-leveling.

Clearly, block_cycles = 0 should do anything but disable wear-leveling.

Here, I've changed block_cycles = 0 to assert. Forcing users to set a
value for block_cycles (500 is suggested). block_cycles can be set to -1
to explicitly disable wear leveling if desired.
2019-07-17 17:05:20 -05:00
Christopher Haster
1aaf1cb6c0 Minor improvements to testing framework
- Moved scripts into scripts folder
- Removed what have been relatively unhelpful assert printing
2019-07-16 20:53:39 -05:00
Christopher Haster
52a90b8dcc Added asserts on positive return values from block device functions
This has been a large source of porting errors, partially due to my
fault in not having enough porting documentation, which is also
planned.

In the short term, asserts should at least help catch these types of
errors instead of just letting the filesystem collapse after recieving
an odd error code.
2019-07-16 15:55:29 -05:00
Christopher Haster
e279c8ff90 Tweaked debug output
- Changed "No more free space" to be an error as suggested by davidefer
- Tweaked output to be more parsable (no space between lfs and warn)
2019-07-16 15:40:26 -05:00
Christopher Haster
6a1ee91490 Added trace statements through LFS_YES_TRACE
To use, compile and run with LFS_YES_TRACE defined:
make CFLAGS+=-DLFS_YES_TRACE=1 test_format

The name LFS_YES_TRACE was chosen to match the LFS_NO_DEBUG and
LFS_NO_WARN defines for the similar levels of output. The YES is
necessary to avoid a conflict with the actual LFS_TRACE macro that
gets emitting. LFS_TRACE can also be defined directly to provide
a custom trace formatter.

Hopefully having trace statements at the littlefs C API helps
debugging and reproducing issues.
2019-07-16 15:14:32 -05:00
Haneef Mubarak
2e92f7a49b actually removed <dirent.h> 2019-07-12 11:46:18 -07:00
Haneef Mubarak
2588948d70 removed <dirent.h> preventing compile on some archs 2019-07-11 15:46:17 -07:00
Christopher Haster
abd90cb84c Fixed 32-bit/64-bit Ubuntu multilib issue in Travis 2019-07-01 19:34:06 -05:00
Christopher Haster
b73ac594f2 Fixed issues with reading and caching inline files
Kind of a two-fold issue. One, the programming to the middle of inline
files was causing the cache to get updated to a half programmed state.
While fine, as all programs do occur in order in a block, this is less
efficient when writing to inline files as it would cause the inline file
to need to be reread even if it fits in the cache.

Two, the rereading of the inline file was broken and passed the file's
tag all the way to where a user would expect an error. This was easy to
fix but adds to the reasons we should have test coverage information.

Found by ebinans
2019-07-01 15:11:53 -05:00
Christopher Haster
614f7b1e68 Fixed accidental truncate after seek on inline files
The cause was mistakenly setting file->ctz.size directly instead of
file->pos, which file->ctz.size gets overwritten with later in
lfs_file_flush.

Also added better seek test cases specifically for inline files. This
should also catch most of the inline corner cases related to
lfs_file_size/lfs_file_tell.

Found by ebinans
2019-07-01 15:11:53 -05:00
Christopher Haster
a9a61a3e78 Added redundant compaction to lfs_format/lfs_migrate
This ensures that both blocks in the superblock pair are written with
the superblock info. While this does use an additional erase cycle, it
prevents older versions of littlefs from accidentally being picked up
in the case that the disk is mounted on a system that doesn't support
the newer version.

This does bring back the risk of picking up old littlefs versions on
a disk that has been formatted with a filesystem that doesn't use
block 2 (such as FAT), but this risk already exists, and moving between
versions of littlefs is more likely with the recent v1 -> v2 update.

Suggested by rojer
2019-07-01 15:11:38 -05:00
Christopher Haster
36973d8fd5 Fixed missing cache flush in lfs_migrate
The data written to the prog cache would make littlefs internally
consistent, but because this was never written to disk, the filesystem
would become unmountable.

Unfortunately, this wasn't found during testing because caches automatically
flush if data is written up to a program boundary (maybe this was a mistake?).

Found by rojer
2019-07-01 15:11:38 -05:00
Christopher Haster
f06dc5737f Merge pull request #201 from nickray/python2-markings
Mark all Python 2 scripts as Python 2
2019-07-01 15:11:16 -05:00
Nicolas Stalder
3fb242f3ae Mark all Python 2 scripts as Python 2 2019-06-07 04:09:44 +02:00
Christopher Haster
ef77195a64 Fixed limit of inline files based on LFS_ATTR_MAX
The maximum limit of inline files and attributes are unrelated, but were
not at a point in littlefs v2 development. This should be checking
against the bit-field limit in the littlefs tag.

Found by lsilvaalmeida
2019-05-23 16:43:23 -05:00
Christopher Haster
12e464e9c3 Fixed issue with writes following a truncate
The problem was not setting the file state correctly after the truncate.
To truncate < size, we end up using the cache to traverse the ctz
skip-list far away from where our file->pos is.

We can leave the last block in the cache in case we're going to append
to the file, but if we do this we need to set up file->block+file->off
to tell use where we are in the file, and set the LFS_F_READING flag to
indicate that our cache contains read data.

Note this is different than the LFS_F_DIRTY, which we need also. The
purpose of the flags are as follows:
- LFS_F_DIRTY - file ctz skip-list branch is out of sync with
  filesystem, need to update metadata
- LFS_F_READING - file cache is in use for reading, need to drop cache
- LFS_F_WRITING - file cache is in use for writing, need to write out
  cache to disk

The difference between flags is subtle but important because read/prog
caches are handled differently. Prog caches have asserts in place to
catch programs without erases (the infamous pcache->block == 0xffffffff
assert).

Though maybe the names deserve an update...

Found by ebinans
2019-05-23 16:43:10 -05:00
Christopher Haster
9899c7fe48 Fixed read cache amount based on hint and offset
Found by apmorton
2019-05-23 16:42:47 -05:00
Christopher Haster
bc7bed740b Merge pull request #181 from rojer/lfs1_crc
Make lfs1_crc static so it doesn't conflict with prefixed LFS1 code
2019-05-23 16:40:09 -05:00
Christopher Haster
cf9afdddff Merge pull request #179 from rojer/wundef
Fix compilation with -Wundef
2019-05-23 16:39:57 -05:00
Deomid "rojer" Ryabkov
2533a0f6d6 Make lfs1_crc static so it doesn't conflict with prefixed LFS1 code
When LFS1 code is present and LFS_MIGRATE is enabled
2019-05-16 17:51:22 +01:00
Deomid "rojer" Ryabkov
2a7f0ed11b Fix compilation with -Wundef 2019-05-14 18:18:29 +01:00
Christopher Haster
f35fb8c148 Fixed migration test condition for prefix branches
Both the littlefs-fuse and littlefs-migration test jobs depend on
the external littlefs-fuse repo. But unfortunately, the automatic
patching to update the external repo with the version under test
does not work with the prefix branches.

In this case we can just skip these tests, they've already been tested
multiple times to get to this point.
2019-04-16 18:29:44 -05:00
Christopher Haster
0a1f706ca2 Merge pull request #160 from FreddieChopin/no-cache-bypass
Don't bypass cache in `lfs_cache_prog()` and `lfs_cache_read()`
2019-04-16 17:59:28 -05:00
Freddie Chopin
fdd239fe21 Don't bypass cache in lfs_cache_prog() and lfs_cache_read()
In some cases specific alignment of buffer passed to underlying device
is required. For example SDMMC in STM32F7 (when used with DMA) requires
the buffers to be aligned to 16 bytes. If you enable data cache in
STM32F7, the alignment of buffer passed to any driver which uses DMA
should generally be at least 32 bytes.

While it is possible to provide sufficiently aligned "read", "prog" and
per-file caches to littlefs, the cases where caches are bypassed are
hard to control when littlefs is hidden under some additional layers.
For example if you couple littlefs with stdio and use it via `FILE*`,
then littlefs functions will operate on internal `FIlE*` buffer, usually
allocated dynamically, so in these specific cases - with insufficient
alignment (8 bytes on ARM Cortex-M).

The easy path was taken - remove all cases of cache bypassing.

Fixes #158
2019-04-12 15:21:25 -05:00
Christopher Haster
780ef2fce4 Fixed buffer overflow due to mistaking prog_size for cache_size
found by ajaybhargav
2019-04-12 08:44:00 -05:00
Christopher Haster
73ea008b74 Merge pull request #151 from Krakonos/master
Fixed documentation for return lfs_dir_read return value.
2019-04-12 17:07:25 -05:00
Christopher Haster
c849748453 Merge pull request #150 from ajaybhargav/truncate-fix
Fix: length more than LFS_FILE_MAX should return error
2019-04-12 17:06:58 -05:00
Christopher Haster
25a843aab7 Fixed .travis.yml to use explicit branch names for migration testing
This lets us actually update the littlefs-fuse repo instead of being
bound to master for v1.
2019-04-12 15:13:00 -05:00
Ajay Bhargav
905727b684 Fix: length more than LFS_FILE_MAX should return error
To make lfs_file_truncate inline with ftruncate function, when -ve
or size more than maximum file size is passed to function it should
return invalid parameter error. In LFS case LFS_ERR_INVAL.

Signed-off-by: Ajay Bhargav <contact@rickeyworld.info>
2019-04-12 15:09:44 -05:00
Christopher Haster
0907ba7813 Merge pull request #85 from ARMmbed/v2-alpha
v2: Metadata logging, custom attributes, inline files, and a major version bump
2019-04-10 20:49:34 -05:00
Christopher Haster
48bd2bff82 Artificially limited number of file ids per metadata block
This is an expirement to determine which field in the tag structure is
the most critical: tag id or tag size.

This came from looking at NAND storage and discussions around behaviour of
large prog_sizes. Initial exploration indicates that prog_sizes around
2KiB are not _that_ uncommon, and the 1KiB limitation is surprising.

It's possible to increase the lfs_tag size to 12-bits (4096), but at the
cost of only 8-bit ids (256).

  [----            32             ----]
a [1|-3-|-- 8 --|--  10  --|--  10  --]
b [1|-3-|-- 8 --|-- 8 --|--   12    --]

This requires more investigation, but in order to allow us to change
the tag sizes with minimal impact I've artificially limited the number
of file ids to 0xfe (255) different file ids per metadata pair. If
12-bit lengths turn out to be a bad idea, we can remove the artificial
limit without backwards incompatible changes.

To avoid breaking users already on v2-alpha, this change will refuse
_creating_ file ids > 255, but should read file ids > 255 without
issues.
2019-04-10 11:27:53 -05:00
Christopher Haster
651e14e796 Cleaned up a couple of warnings
- Shifting signed 32-bit value by 31 bits is undefined behaviour

  This was an interesting one as on initial inspection, `uint8_t & 1`
  looks like it will result in an unsigned variable. However, due to
  uint8_t being "smaller" than int, this actually results in a signed
  int, causing an undefined shift operation.

- Identical inner 'if' condition is always true (outer condition is
  'true' and inner condition is 'true').

  This was caused by the use of `if (true) {` to avoid "goto bypasses
  variable initialization" warnings. Using just `{` instead seems to
  avoid this problem.

found by keck-in-space and armandas
2019-04-10 11:27:53 -05:00
Christopher Haster
1ff6432298 Added clarification on buffer alignment.
In v2, the lookahead_buffer was changed from requiring 4-byte alignment
to requiring 8-byte alignment. This was not documented as well as it
could be, and as FabianInostroza noted, this also implies that
lfs_malloc must provide 8-byte alignment.

To protect against this, I've also added an assert on the alignment of
both the lookahead_size and lookahead_buffer.

found by FabianInostroza and amitv87
2019-04-10 11:27:48 -05:00
Christopher Haster
c2c2ce6b97 Fixed issue with handling block device errors in lfs_file_sync
lfs_file_sync was not correctly setting the LFS_F_ERRED flag.
Fortunately this is a relatively easy fix. LFS_F_ERRED prevents
further issues from occuring when cleaning up resources with
lfs_file_close.

found by TheLoneWolfling
2019-04-09 17:41:26 -05:00
Christopher Haster
0b76635f10 Added better handling of large program sizes (> 1024)
The issue here is how commits handle padding to the nearest program
size. This is done by exploiting the size field of the LFS_TYPE_CRC
tag that completes the commit. Unfortunately, during developement, the
size field shrank in size to make room for more type information,
limiting the size field to 1024.

Normally this isn't a problem, as very rarely do program sizes exceed
1024 bytes. However, using a simulated block device, user earlephilhower
found that exceeding 1024 caused littlefs to crash.

To make this corner case behave in a more user friendly manner, I've
modified this situtation to treat >1024 program sizes as small commits
that don't match the prog size. As a part of this, littlefs also needed
to understand that non-matching commits indicate an "unerased" dir
block, which would be needed for portability (something which notably
lacks testing).

This raises the question of if the tag size field size needs to be
reconsidered, but to change that at this point would need a new major
version.

found by earlephilhower
2019-04-09 16:06:43 -05:00
Christopher Haster
a32be1d875 Merge remote-tracking branch 'origin/master' into v2-alpha 2019-04-08 15:12:36 -05:00
Christopher Haster
7e110b44c0 Added automatic version prefixing to releases
The script itself is a part of .travis.yml, using ./scripts/prefix.py
for applying prefixes to the source code.

This purpose of the automatic job is to provide a branch containing
version prefixes, to avoid name conflicts in binaries containing
different major versions of littlefs with only a git clone.

As a part of each release, two branches and a tag are created:
- vN        - moving branch
- vN-prefix - moving branch
- vN.N.N    - immutable tag

The major version branch (vM) is created on major releases, but updated
every patch release. The patch version tag (vM.M.P) is created every
patch release. Patch releases occur every time a commit is merged into
master, though multiple merges may be coalesced.

The major prefix branch (vM-prefix) is modified with the ./scripts/prefix.py
script. Note that this branch is updated as a synthetic merge commit
with the previous history of vM-prefix. The reason for this is to allow
users to easily update vM-prefix with a `git pull` as they would for
other branches.

A---B---C---D---E master, v1, v1.7.3
     \       \   \
      F-------G---H v1-prefix
2019-04-08 13:55:35 -05:00
Christopher Haster
7f7b7332e3 Added scripts/prefix.py for automatically prefixing version numbers
Example:
./scripts/prefix.py lfs2

Will convert the following:
lfs_* -> lfs2_*
LFS_* -> LFS2_*
-DLFS_* -> -DLFS2_*
2019-04-08 13:55:28 -05:00
Christopher Haster
9568f8ee2d Added v1->v2 migration into CI
Also fixed issue where migration would not handle large dirs due to v1
iteration changing the pair of the directory.
2019-04-01 22:12:08 -05:00
Christopher Haster
bdff4bc59e Updated DESIGN.md to reflect v2 changes
Now with graphs! Images are stored on the branch gh-images in an effort
to avoid binary bloat in the git history.

Also spruced up SPEC.md and README.md and ran a spellechecker over the
documentation. Favorite typo so far was dependendent, which is, in fact,
not a word.
2019-03-31 22:15:32 -05:00
Ladislav Láska
26d25608b6 Fixed documentation for return lfs_dir_read return value.
lfs_dir_read breaks the convention of returning non-zero on success,
this feature should be at least documented.
2019-03-01 10:01:02 +01:00
Christopher Haster
4ad09d6c4e Added migration from littlefs v1
This is the help the introduction of littlefs v2, which is disk
incompatible with littlefs v1. While v2 can't mount v1, what we can
do is provide an optional migration, which can convert v1 into v2
partially in-place.

At worse, we only need to carry over the readonly operations on v1,
which are much less complicated than the write operations, so the extra
code cost may be as low as 25% of the v1 code size. Also, because v2
contains only metadata changes, it's possible to avoid copying file
data during the update.

Enabling the migration requires two steps
1. Defining LFS_MIGRATE
2. Call lfs_migrate (only available with the above macro)

Each macro multiplies the number of configurations needed to be tested,
so I've been avoiding macro controlled features since there's still work
to be done around testing the single configuration that's already
available. However, here the cost would be too high if we included migration
code in the standard build. We can't use the lfs_migrate function for
link time gc because of a dependency between the allocator and v1 data
structures.

So how does lfs_migrate work? It turned out to be a bit complicated, but
the answer is a multistep process that relies on mounting v1 readonly and
building the metadata skeleton needed by v2.

1. For each directory, create a v2 directory
2. Copy over v1 entries into v2 directory, including the soft-tail entry
3. Move head block of v2 directory into the unused metadata block in v1
   directory. This results in both a v1 and v2 directory sharing the
   same metadata pair.
4. Finally, create a new superblock in the unused metadata block of the
   v1 superblock.

Just like with normal metadata updates, the completion of the write to
the second metadata block marks a succesful migration that can be
mounted with littlefs v2. And all of this can occur atomically, enabling
complete fallback if power is lost of an error occurs.

Note there are several limitations with this solution.

1. While migration doesn't duplicate file data, it does temporarily
   duplicate all metadata. This can cause a device to run out of space if
   storage is tight and the filesystem as many files. If the device was
   created with >~2x the expected storage, it should be fine.

2. The current implementation is not able to recover if the metadata
   pairs develop bad blocks. It may be possilbe to workaround this, but
   it creates the problem that directories may change location during
   the migration. The other solutions I've looked at are complicated and
   require superlinear runtime. Currently I don't think it's worth
   fixing this limitation.

3. Enabling the migration requires additional code size. Currently this
   looks like it's roughly 11% at least on x86.

And, if any failure does occur, no harm is done to the original v1
filesystem on disk.
2019-02-27 19:58:07 -06:00
Christopher Haster
7d8f8ced03 Enabled -Wextra
This only required adding NULLs where commit statements were not fully
initialized.

Unfortunately we still need -Wno-missing-field-initializers because
of a bug in GCC that persists on Travis.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60784

Found by apmorton
2019-02-27 01:35:44 -06:00
Christopher Haster
a0644794ca Fixed several small issues
- Fixed uninitialized values found by valgrind.
- Fixed uninitialized value in lfs_dir_fetchmatch when handling revision
  counts.
- Fixed mess left by lfs_dir_find when attempting to find the root
  directory in lfs_rename and lfs_remove.
- Fixed corner case with definitions of lfs->cfg->block_cycles.
- Added test cases around different forms of the root directory.

I think all of these were found by TheLoneWolfling, so props!
2019-02-12 00:01:28 -06:00
Christopher Haster
512930c856 Updated SPEC.md to reflect v2 changes 2019-02-10 22:11:50 -06:00
Christopher Haster
10dfc36f08 Fixed issue with long names causing unbounded recursion
This was caused by any commit containing entries large enough to
_always_ force a compaction. This would cause littlefs to think that it
would need to split infinitely because there was no base case.

The fix here is pretty simple: treat any commit with only a single entry
as unsplittable. This forces littlefs to first try overcompacting
(fitting more in a block than what has optimal runtime), and then
failing that return LFS_ERR_NOSPC for higher layers to handle.

found by TheLoneWolfling
2019-01-31 14:59:19 -06:00
Christopher Haster
95c1a6339d Fixed corner case in block_cycles eviction logic
The problem was when we allocate a dir-pair, it's possible for the
revision count to immediately overflow and the dir-pair be evicted and
returned to the unused blocks without being written even once. In the
case that block_cycles = 1, this made it impossible to ever create a
dir-pair, even in lfs_format.

I've also added a bit of logic to lfs_dir_alloc that will prevent
any immediate evictions because of the revision count.

found by TheLoneWolfling
2019-01-29 22:43:19 -06:00
Christopher Haster
173c212151 Added scripts/prefix.py for automatically prefixing version numbers
Example:
./scripts/prefix.py lfs2

Will convert the following:
lfs_* -> lfs2_*
LFS_* -> LFS2_*
-DLFS_* -> -DLFS2_*
2019-01-29 22:42:06 -06:00
Christopher Haster
d3a2cf48d4 Merge pull request #135 from johnlunney/patch-1
Add missing word (and reflow text)
2019-01-28 15:48:19 -06:00
johnl
22b0456623 Add missing word (and reflow text) 2019-01-26 21:38:23 +01:00
Christopher Haster
8cca1b6a86 Fixed several small issues found during wider testing
- Fixed cache tarnishing issue where flush did not clean up read caches
- Removed extra alloc acks which would prevent file relocations from
  resolving on an exhausted filesystem
- Removed unsigned comparison < 0 from changed in file seek
- Fixed bug in lfs_dir_getslice with using gtag's size
- Removed warnings around PRIu32 used 16-bit types in debug info
2019-01-22 21:02:46 -06:00
Christopher Haster
5fb8fa9f06 Fixed issue with global state updates being lost during relocates
Caught during power resilience testing, this was a bug that only occurs
when we need to compact in the middle of a move commit and we find that
the destination block is bad, forcing a relocate.

This series of events would cause littlefs to clear the "gpending" state
in preparation for fixing the move atomically, but this fix never gets
written out because of the relocate.

The fix here is to separate the update to the "gdelta" and "gpending"
state, marking "gdelta" in preparation for the move, but waiting to
update "gpending" until after our commit completes. This keeps our disk
state in sync without prematurely dropping moves.
2019-01-22 21:02:46 -06:00
Christopher Haster
916b308558 Fixed excessive waste from overly large inline files
Before this, there were some safety limits, but there was no real
default limit to the size of inline files other than the amount of RAM
available. On PCs, this meant that inline files were free to fill up
directory blocks to a little under the block size.

However this is very wasteful in terms of storage space. Because of
splitting limits to keep the compact runtime reasonable, each byte of an
inline files uses 4x the amount.

Fortunately we can find an optimal inline limit:

Inline file waste for n bytes = 3n
CTZ file waste for n bytes    = B - n

Where B = block size

Solving for n = B/4

So the optimal inline limit is B/4. However, this assumes a perfect inline
file and no metadata. We can decrease this to B/8 to give a bit more
breathing room for directory+file metadata.
2019-01-22 21:02:39 -06:00
Christopher Haster
e1f9d2bc09 Added support for RAM-independent reading of inline files
One of the new features in LittleFS is "inline files", which is the
inlining of small files in the parent directory. Inline files have a big
limitation in that they no longer have a dedicated scratch area to write
out data before commit-time. This is fine as long as inline files are
small enough to fit in RAM.

However, this dependency on RAM creates an uncomfortable situation for
portability, with larger devices able to create larger files than
smaller devices. This problem is especially important on embedded
systems, where RAM is at a premium.

Recently, I realized this RAM requirement is necessary for _writing_
inline files, but not for _reading_ inline files. By allowing fetches of
specific slices of inline files it's possible to read inline files
without the RAM to back it.

However however, this creates a conflict with COW semantics. Normally,
when a file is open twice, it is referenced by a COW data structure that
can be updated independently. Inlines files that fit in RAM also allows
independent updates, but the moment an inline file can't fit in
RAM, any updates to that directory block could corrupt open files
referencing the inline file. The fact that this behaviour is only
inconsistent for inline files created on a different device with more
RAM creates a potential nightmare for user experience.

Fortunately, there is a workaround for this. When we are commiting to a
directory, any open files needs to live in a COW structure or in RAM.
While we could move large inline files to COW structures at open time,
this would break the separation of read/write operations and could lead
to write errors at read time (ie ENOSPC). But since this is only an
issue for commits, we can defer the move to a COW structure to any
commits to that directory. This means when committing to a directory we
need to find any _open_ large inline files and evict them from the
directory, leaving the file with a new COW structure even if it was
opened read only.

While complicated, the end result is inline files that can use the
MAX RAM that is available, but can be read with MIN RAM, even with
multiple write operations happening to the underlying directory block.
This prevents users from needing to learn the idiosyncrasies of inline
files to use the filesystem portably.
2019-01-22 20:59:59 -06:00
Christopher Haster
51b2c7e4b6 Changed custom attribute descriptors to used arrays
While linked-lists do have some minor benefits, arrays are more
idiomatic in C and may provide a more intuitive API.

Initially the linked-list approach was more beneficial than it is now,
since it allowed custom attributes to be chained to internal linked
lists of attributes. However, this was dropped because exposing the
internal attribute list in this way created a rather messy user
interface that required strictly encoding the attributes with the
on-disk tag format.

Minor downside, users can no longer introduce custom attributes in
different layers (think OS vs app). Minor upside, the code size and
stack usage was reduced a bit.

Fortunately, this API can always be changed in the future without
breaking anything (except maybe API compatibility).
2019-01-13 23:56:53 -06:00
Christopher Haster
66d751544d Modified global state format to work with new tag format
The main difference here is a change from encoding "hasorphans" and
"hasmove" bits in the tag itself. This worked with the old format, but
in the new format the space these bits take up must be consistent for
each tag type. The tradeoff is that the new tag format allows for up to
256 different global states which may be useful in the future (for
example, a global free list).

The new format encodes this info in the data blob, using an additional
word of storage. This word is actually formatted the same as though it
was a tag, which simplified internal handling and may allow other tag
types in the future.

Format for global state:
[----                          96 bits                         ----]
[1|- 11 -|- 10 -|- 10 -|---                 64                  ---]
 ^    ^      ^      ^                        ^- move dir pair
 |    |      |      \-------------------------- unused, must be 0s
 |    |      \--------------------------------- move id
 |    \---------------------------------------- type, 0xfff for move
 \--------------------------------------------- has orphans

This also included another iteration over globals (renamed to gstate)
with some simplifications to how globals are handled.
2019-01-13 23:56:50 -06:00
Christopher Haster
b989b4a89f Cleaned up tag encoding, now with clear chunk field
Before, the tag format's type field was limited to 9-bits. This sounds
like a lot, but this field needed to encode up to 256 user-specified
types. This limited the flexibility of the encoded types. As time went
on, more bits in the type field were repurposed for various things,
leaving a rather fragile type field.

Here we make the jump to full 11-bit type fields. This comes at the cost
of a smaller length field, however the use of the length field was
always going to come with a RAM limitation. Rather than putting pressure
on RAM for inline files, the new type field lets us encode a chunk
number, splitting up inline files into multiple updatable units. This
actually pushes the theoretical inline max from 8KiB to 256KiB! (Note
that we only allow a single 1KiB chunk for now, chunky inline files
is just a theoretical future improvement).

Here is the new 32-bit tag format, note that there are multiple levels
of types which break down into more info:

[----            32             ----]
[1|--  11   --|--  10  --|--  10  --]
 ^.     ^     .     ^          ^- entry length
 |.     |     .     \------------ file id chunk info
 |.     \-----.------------------ type info (type3)
 \.-----------.------------------ valid bit
  [-3-|-- 8 --]
    ^     ^- chunk info
    \------- type info (type1)

Additionally, I've split the CREATE tag into separate SPLICE and NAME
tags. This simplified the new compact logic a bit. For now, littlefs
still follows the rule that a NAME tag precedes any other tags related
to a file, but this can change in the future.
2019-01-13 23:56:01 -06:00
Christopher Haster
a548ce68c1 Switched to traversal-based compact logic
This simplifies some of the interactions between reading and writing
inside the commit logic. Unfortunately this change didn't decrease
code size as was initially hoped, but it does offer a nice runtime
improvement for the common case and should improve debugability.

Before, the compact logic required three iterations:
1. iterate through all the ids in a directory
2. scan attrs bound to each id in the directory
3. lookup attrs in the in-progress commit

The code for this, while terse and complicated, did have some nice side
effect. The directory lookup logic could be reused for looking up in the
in-progress commit, and iterating through each id allows us to know
exactly how many ids we can fit during a compact. Giving us a O(n^3)
compact and O(n^3) split.

However, this was complicated by a few things.

First, this compact logic doesn't handle deleted attrs. To work around this,
I added a marker for the last commit (or first based on your perspective)
which would indicate if a delete should be copied over. This worked but was
a bit hacky and meant deletes weren't cleaned up on the first compact.

Second, we can't actually figure out our compacted size until we
compact. This worked ok except for the fact that splits will always have a
failed compact. This means we waste an erase which could very expensive.
It is possible to work around this by keeping our work, but with only a
single prog cache this was very tricky and also somewhat hacky.

Third, the interactions between reading and writing to the same block
were tricky and error-prone. They should mostly be working now, but
seeing this requirement go away does not make me sad.

The new compact logic fixes these issues by moving the complexity into a
general-purpose lfs_dir_traverse function which has much fewer side
effects on the system. We can even use it for dry-runs to precompute our
estimated size.

How does it work?
1. iterate through all attr in the directory
2. for each attr, scan the rest of the directory to figure out the
   attr's history, this will change the attr based on dir modifications
   and may even exit early if the attr was deleted.

The end result is a traversal function that gives us the resulting state
of each attr in only O(n^2). To make this complete, we allow a bounded
recursion into mcu-side move attrs, although this ends up being O(n^3)
unlike moves in the original solution (however moves are less common.

This gives us a nice traversal function we can use for compacts and
moves, handles deletes, and is overall simpler to reason about.

Two minor hiccups:
1. We need to handle create attrs specially, since this algorithm
   doesn't care or id order, which can cause problems since attr
   insertion are order sensitive. We can fix this by simply looking up
   each create (since there is only one per file) in order at the
   beginning of our traversal. This is oddly complimentary to the move
   logic, which also handles create attrs separately.

2. We no longer know exactly how many ids we can write to a dir during
   splits. However, since we can do a dry-run traversal, we can use that
   to simply binary search for the mid-point.

This gives us a O(n^2) compact and O(n^2 log n) split, which is a nice
minor improvement (remember n is bounded by block size).
2018-12-28 11:17:51 -06:00
Christopher Haster
dc507a7b5f Changed required alignment of lookahead_size to 64 bits
This is to prepare for future compatibility with other implementations
of the allocator's lookahead that are under consideration. The most
promising design so far is a sort of segments-list data structure that
stores pointer+size pairs, requiring 64-bits of alignment.

Changing this now takes advantage of the major version to avoid a
compatibility break in the future. If we end up not changing the
allocator or don't need 64-bit alignment we can easily drop this
requirement without breaking anyone's code.
2018-10-22 17:58:47 -05:00
Christopher Haster
5b26c68ae2 Tweaked tag endianness to catch power-loss after <1 word is written
There was an interesting subtlety with the existing layout of tags that
could become a problem in the future. Basically, littlefs avoids writing to
any region of storage it is not absolutely sure has been erased
beforehand. This is a part of limiting the number of assumptions about
storage. It's possible a storage technology can't support writes without
erases in a way that is undetectable at write time (Maybe changing a bit
without an erase decreases the longevity of the information stored on
the bit).

But the existing layout had a very tiny corner case where this wasn't
true. Consider the location of the valid bit in the tag struct:

[1|---  31  ---]
 ^--- valid bit

The responsibility of this bit is to indicate if an attempt has been
made to write the following commit. If it is not set (the specific value
is dependent on a previous read and identified by the preceeding commit),
the assumption is that it is safe to write to the next region because it
has been erased previously. If it is set, we check if the next commit is
valid, if it isn't (because of CRC failure, likely due to power-loss), we
discard the commit. But because an attempt has been made to write to
that storage, we must then do a compaction to move to the other block in
the metadata-pair.

This plan looks good on paper, but what does it look like on storage?
The problem is that words in littlefs are in little-endian. So on
storage the tag actually looks like this:

[- 8 -|- 8 -|- 8 -|1|- 7 -]
                   ^-- valid bit

This means that we don't actually set the valid bit before writing the
tag! We write the lower bytes first. If we lose power, we may have
written 3 bytes without this fact being detectable.

We could restructure the tag structure to store the valid bit lower,
however because none of the fields are 7 bits, this would make the
extraction more costly, and we then lose the ability to check this
valid bit with a sign comparison.

The simple solution is to just store the tag in big-endian. A small
benefit is that this will actually have a negative code cost on
big-endian machines.

This mixture of endiannesses is frustrating, however it is a pragmatic
solution with only a 20-byte code size cost.
2018-10-22 17:58:32 -05:00
Christopher Haster
4a1b8ae222 Fixed issues found by more aggressive rename tests
- Fixed underflow issue caused by search id shortcuts that would result
  in early termination from lfs_dir_get
- Fixed issue where entry file delete would toss out the best id during
  lfs_dir_fetchmatch
- Fixed globals going out of date when canceling in same metadata-pair
- Fixed early removal of metadata-pair when attribute list contains
  creates after deletes bring dir->count to zero
2018-10-21 11:25:48 -05:00
Christopher Haster
c8a39c4b23 Merge remote-tracking branch 'origin/master' into v2-rebase-part2 2018-10-20 21:02:25 -05:00
Christopher Haster
ec4d8b68ad Changed release script to generate drafts 2018-10-20 12:34:41 -05:00
Christopher Haster
c7894a61e1 Added a handful of links to related projects
Interesting open-source projects that I've ran into around embedded
storage. May be interesting to others in the embedded space.

Added mklfs, SPIFFS, and Dhara.

Also a thanks to jolivepetrus for posting the mklfs tool he put
together.
2018-10-20 12:34:41 -05:00
Christopher Haster
195075819e Added 2GiB file size limit and EFBIG reporting
On disk, littlefs uses 32-bit integers to track file size. This sets a
theoretical limit of 4GiB for files.

However, the API passes file sizes around as signed numbers, with
negative values representing error codes. This means that not all of the
APIs will work with file sizes > 2GiB.

Because of related complications over in FUSE land, I've added the LFS_FILE_MAX
constant and proper error reporting if file writes/seeks exceed the 2GiB limit.
In v2 this will join the other constants that get stored in the
superblock to help portability. Since littlefs is targeting
microcontrollers, it's likely this will be a sufficient solution.

Note that it's still possible to enable partial-support for 4GiB files
by defining LFS_FILE_MAX during compilation. This will work for most of
the APIs, except lfs_file_seek, lfs_file_tell, and lfs_file_size.

We can also consider improving support for 4GiB files, by making seek a
bit more complicated and adding a lfs_file_stat function. I'll leave
this for a future improvement if there's interest.

Found by cgrozemuller
2018-10-20 12:34:23 -05:00
Christopher Haster
97d8d5e96a Fixed issue where a rename causes a split and pushes dir out of sync
The issue happens when a rename causes a split in the destination pair.
If the destination pair is the same as the source pair, this triggers the
logic to keep both pairs in sync. Unfortunately, this logic didn't work,
because the source entry still resides in the old source pair, unlike
the destination pair, which is now in the new pair created by the split.

The best fix for now is to refetch the source pair after the changes to the
destination pair. This isn't the most efficient solution, but fortunately
this bug has already been fixed in the revamped move logic in littlefs v2
(currently in progress).

Found by ohoc
2018-10-20 12:34:11 -05:00
Christopher Haster
795dd8c7ab Fixed mkdir when inserting into a non-end block
This was an oversight on my part when adding strict ordering to
directories. Unfortunately now we can't take advantage of the atomic
creation of tail+dir entries. Now we need to first create the tail, then
create the actually directory entry. If we lose power, the orphan is
cleaned up like orphans created during remove.

Note that we still take advantage of the atomic tail+dir entries if we
are an end block. This is actually because this corner case is
complicated to _not_ do atomically, needing to update the directory we
just committed to.
2018-10-18 10:00:49 -05:00
Christopher Haster
97a7191814 Fixed issue with creating files named "littlefs"
A rather humorous issue, we accidentally ended up mixing our file
namespace with our superblocks. This meant if we created a file named
"littlefs" it would reference the superblock and all sorts of things
would break.

Fixing this also highlighted another issue, the fact that the superblock
always needs to come before any file entries in the directory. I didn't
account for this in the initial B-tree design, but we need a higher
ordering for superblocks + children + files than just name. To fix this
I added ordering information in the 2 bits currently unused in the tag
type. Though note that the size of these fields are flexible.

9-bit type field:
[---      9      ---]
[1|- 3 -|- 2 -|- 3 -]
 ^   ^     ^     ^- type-specific info
 |   |     \------- ordering info
 |   \------------- subtype
 \----------------- user bit
2018-10-18 10:00:49 -05:00
Christopher Haster
aeca7667b3 Switched to strongly ordered directories
Instead of storing files in an arbitrary order, we now store files in
ascending lexicographical order by filename.

Although a big change, this actually has little impact on how littlefs
works internally. We need to support file insertion, and compare file
names to find our position. But since we already need to scan the entire
directory block, this adds relatively little overhead.

What this does allow, is the potential to add B-tree support in the
future in a backwards compatible manner.

How could you add B-trees to littlefs?
1. Add an optional "child" tag with a pointer that allows you to skip to
   a position in the metadata-pair list that composes the directory
2. When splitting a metadata-pair (sound familiar?), we either insert a
   second child tag in our parent, or we create a new root containing
   the child tags.
3. Each layer needs a bit stored in the tail-pointer to indicate if
   we're going to the next layer. This can be created trivially when we
   create a new root.
4. During lookup we keep two pointers containing the bounds of our
   search. We may need to iterate through multiple metadata-pairs in our
   linked-list, but this gives us a O(log n) lookup cost in a balanced
   tree.
5. During deletion we also delete any children pointers. Note that
   children pointers must come before the actual file entry.

This gives us a B-tree implementation that is compatible with the
current directory layout (assuming the files are ordered). This means
that B-trees could be supported by a host PC and ignored on a small
device. And during power-loss, we never end up with a broken filesystem,
just a less-than-optimal tree.

Note that we don't handle removes, so it's possible for a tree to become
unbalanced. But worst case that's the same as the current linked-list
implementation.

All we need to do now is keep directories ordered. If we decide to drop
B-tree support in the future or the B-tree implementation turns out
inherently flawed, we can just drop the ordered requirement without
breaking compatibility and recover the code cost.
2018-10-18 10:00:49 -05:00
Christopher Haster
7af8b81b81 Changed lookahead configuration unit to bytes instead of bits
The fact that the lookahead buffer uses bits instead of bytes is an
internal detail. Poking this through to the user API has caused a decent
amount of confusion. Most buffers are provided as bytes and the
inconsistency here can be surprising.

The use of bytes instead of bits also makes us forward compatible in
the case that we want to change the lookahead internal representation
(hint segment list).

Additionally, we change the configuration name to lookahead_size. This
matches other configurations, such as cache_size and read_size, while
also notifying the user that something important changed at compile time
(by breaking).
2018-10-18 10:00:49 -05:00
Christopher Haster
ad96fca18f Changed attr_max to be specific to custom attributes
While technically, both system and user attributes share the same disk
limitations, that's not what attr_max represents when considered from
the user's perspective. To the user, attr_max applies only to custom
attributes. This means attr_max should not impact other configurable
limitations, such as inline files, and the ordering should be
reconsidered with what the user finds most important.
2018-10-18 10:00:49 -05:00
Christopher Haster
f010d2add1 Fixed issue with reads ignoring the pcache
The downside of smarter caching is that now there are more complicated
corner cases to consider. Here we weren't considering our pcaches when
aligning reads to the rcache. This meant if things were unaligned, we
would read a cache-line that overlaps the pcache and then proceed to
ignore whatever we overlapped.

This fix is to determine the limit of an rcache read not from cache
alignment but from the available caches, which we check anyways to find
cached data.
2018-10-18 10:00:49 -05:00
Christopher Haster
d7e4abad0b Edited tag structure to balance size vs id count
This is a minor tweak that resulted from looking at some other use cases
for the littlefs data-structure on disk. Consider an implementation that
does not need to buffer inline-files in RAM. In this case we should have
as large a tag size field as possible. Unfortunately, we don't have much
space to work with in the 32-bit tag struct, so we have to make some
compromises. These limitations could be removed with a 64-bit tag
struct, at the cost of code size.

32-bit tag structure:
[---       32       ---]
[1|- 9 -|- 9 -|-- 13 --]
 ^   ^     ^      ^- entry length
 |   |     \-------- file id
 |   \-------------- tag type
 \------------------ valid bit
2018-10-18 10:00:49 -05:00
Christopher Haster
cafe6ab466 Fixed issue with splitting metadata-pairs in full filesystem
Depending on your perspective, this may not be a necessary operation,
given that a nearly-full filesystem is already prone to ENOSPC errors,
especially a COW filesystem. However, splitting metadata-pairs can
happen in really unfortunate situations, such as removing files.

The solution here is to allow "overcompaction", that is, a compaction
without bounds checking to allow splitting. This unfortunately pushes
our metadata-pairs past their reasonable limit of saturation, which
means writes get exponentially costly. However it does allow littlefs to
continue working in extreme situations.
2018-10-18 10:00:49 -05:00
Christopher Haster
29b881017d Revisited xored-globals and related logic
Added separate bit for "hasmove", which means we don't need to check
the move id, and allows us to add more sync-related global states in
the future, as long as they never happen simultaneously (such as
orphans and moves).

Also refactored some of the logic and removed the union in the global
structure, which didn't really add anything of value.
2018-10-18 10:00:49 -05:00
Christopher Haster
cf87ba5375 Combined superblock scan and fetch of xored-globals during mount
Conceptually these are two separate operations. However, they are both
only needed during mount, both require iteration over the linked-list of
metadata-pairs, and both are independent from each other.

Combining these into one gives us a nice code savings.

Additionally, this greatly simplifies the lookup of the root directory.
Initially we used a flag to indicate which superblock was root, since we
didn't want to fetch more pairs than we needed to. But since we're going
to fetch all metadata-pairs anyways, we can just use the last superblock
we find as the indicator of our root directory.
2018-10-18 10:00:49 -05:00
Christopher Haster
7bacf9b1e0 Removed xored-globals from the mdir struct
The xored-globals have a very large footprint. In the worst case, the
xored-globals are stored on each metadata-pair, twice in memory. They
must be very small, but are also very useful, so at risk of growing
in the future (hint global free-list?).

Initially we also stored a copy in each mdir structure, since this
avoided extra disk access to look up the globals when we need to modify
the global state on a metadata-pair. But we can easily just fetch the
globals when needed.

This is more costly in terms of runtime, but reduces RAM impact of
globals, which was previously needed for each open dir and file.
2018-10-18 10:00:49 -05:00
Christopher Haster
5eeeb9d6ac Revisited some generic concepts, callbacks, and some reorganization
- Callbacks for get/match, this does have a code cost, but allows more
  code reuse, which almost balances out the code cost, but also reduces
  maintenance and increased flexibility. Also callbacks may be able to
  be gc-ed in some cases.
- Consistent struct vs _t usage, _t for external-facing struct that
  shouldn't be messed with outside the library. structs for external and
  internal structs where anyone with access is allowed to modify.
- Reorganized several high-level function groups
- Inlined structures that didn't need separate definitions in header
2018-10-18 10:00:49 -05:00
Christopher Haster
617dd87621 Added deletion to custom attributes
This follows from enabling tag deletion, however does require some
consideration with the APIs.

Now we can remove custom attributes, as well as determine if an attribute
exists or not.
2018-10-18 10:00:49 -05:00
Christopher Haster
c67a41af7a Added support for deleting attributes
littlefs has a mechanism for deleting file entries, but it doesn't have
a mechanism for deleting individual tags. This _is_ sufficient for a
filesystem, but limits our flexibility. Deleting attributes would be
useful in the custom attribute API and for future improvements (hint the
child pointers in B-trees).

However, deleteing attributes is tricky. We can't just omit the
attribute, since we can only add new tags. Additionally, we need a way
to track what attributes have been deleted during compaction, which
currently relies on writing out attributes to disk.

The solution here is pretty nifty. First we have to come up with a way
to represent a "deleted" attribute. Rather than adding an additional
bit to the already squished tag structure, we use a -1 length field,
specifically 0xfff. Now we can commit a delete attribute, and this
deleted tag acts as a place holder during compacts.

However our delete tag will never leave our metadata log. We need some
way to discard our delete tag if we know it's the only representation of
that tag on the metadata log. Ah! We know it's the only tag if it's in
the first commit on the metadata log. So we add an additional bit to the
CRC entry to indicate if we're on the first commit, and use that to
decide if we need to keep delete tags around.

Now we have working tag deletion.

Interestingly enough, tag deletion is actually indirectly more efficient
than entry deletion, since compacting entries requires multiple passes,
whereas tag deletion gets cleaned up lazily. However we can't adopt the
same strategy in entry deletion because of the compact ordering of
entries. Tag deletion works because tag types are unique and static.
Managing entry deletion in this manner would require static id
allocation, which would cause problems when creating files, running out
of space, and disallow arbitrary insertions of files.
2018-10-18 10:00:49 -05:00
Christopher Haster
6046d85e6e Added support for entry insertion
Currently unused, the insertion of new file entries in arbitrary
locations in a metadata-pair is very easy to add into the existing
metadata logging.

The only tricky things:
1. Name tags must strictly precede any tags related to a file. We can
   pull this off during a compact, but must make two passes. One for the
   name tag, one for the file. Though a benefit of this is that now our
   scans during moves can exit early upon finding the name tag.

1. We need to handle name tags appearing out of order. This makes name
   tags symmetric to deletes, although it doesn't seem like we can
   leverage this fact very well. Note this also means we need to make
   the superblock tag a type of name tag.
2018-10-18 10:00:49 -05:00
Christopher Haster
6db5202bdc Modified valid bit to provide an early check on all tags
The valid bit present in tags is a requirement to properly detect the
end of commits in metadata logs. The way it works is that the CRC entry is
allowed to specify what is needed from the next tag's valid bit. If it's
incorrect, we've reached the end of the commit. We then set the valid bit to
indicate when we tried to program a new commit. If we lose power, this
commit will still be thrown out by a bad checksum.

However, the valid bit is unused outside of the CRC entry. Here we turn on the
valid bit for all tags, which means we have a decent chance of exiting early
if we hit a half-written commit. We still need to guarantee detection of
the valid bit on commits following the CRC entry, so we allow the CRC
entry to flip the expected valid bit.

The only tricky part is what valid bit we expect by default, since this
is used on the first commit on a metadata log. Here we default to a 1,
which gives us the fastest exit on blocks that erase to 0. This is
because blocks that erase to 1s will implicitly flip the valid bit of
the next tag, allowing us to exit on the next tag.

If we defaulted to 0, we could exit faster on disks that erase to 1, but
would need to scan the entire block on disks that erase to 0 before we
realize a CRC commit is never coming.
2018-10-18 10:00:49 -05:00
Christopher Haster
a43f9b3cd5 Modified lfs_dir_compact to avoid redundant erases during split
The commit machine in littlefs has three stages: commit, compact, and
then split. First we try to append our commit to the metadata log, if
that fails we try to compact the metadata log to remove duplicates and make
room for the commit, if that still fails we split the metadata into two
metadata-pairs and try again. Each stage is less efficient but also less
frequent.

However, in the case that we're filling up a directory with new files,
such as the bootstrap process in setting up a new system, we must pass
through all three stages rather quickly in order to get enough
metadata-pairs to hold all of our files. This means we'll compact,
split, and then need to compact again. This creates more erases than is
needed in the optimal case, which can be a big cost on disks with an
expensive erase operation.

In theory, we can actually avoid this redundant erase by reusing the
data we wrote out in the first attempt to compact. In practice, this
trick is very complicated to pull off.

1. We may need to cache a half-completed program while we write out the
   new metadata-pair. We need to write out the second pair first in
   order to get our new tail before we complete our first metadata-pair.
   This requires two pcaches, which we don't have

   The solution here is to just drop our cache and reconstruct what if
   would have been. This needs to be perfect down to the byte level
   because we don't have knowledge of where our cache lines are.

2. We may have written out entries that are then moved to the new
   metadata-pair.

   The solution here isn't pretty but it works, we just add a delete
   tag for any entry that was moved over.

In the end the solution ends up a bit hacky, with different layers poked
through the commit logic in order to manage writes at the byte level
from where we manage splits. But it works fairly well and saves erases.
2018-10-18 10:00:49 -05:00
Christopher Haster
478dcdddef Revisited caching rules to optimize bus transactions
The littlefs driver has always had this really weird quirk: larger cache
sizes can significantly harm performance. This has probably been one of
the most surprising pieces of configuraing and optimizing littlefs.

The reason is that littlefs's caches are kinda dumb (this is somewhat
intentional, as dumb caches take up much less code space than smart
caches). When littlefs needs to read data, it will load the entire cache
line. This means that even when we only need a small 4 byte piece of
data, we may need to read a full 512 byte cache. And since
microcontrollers may be reading from storage over relatively slow bus
protocols, the time to send data over the bus may dominate other
operations.

Now that we have separate configuration options for "cache_size" and
"read_size", we can start making littlefs's caches a bit smarter. They
aren't going to be perfect, because code size is still a priority, but
there are some small improvements we can do:

1. Program caches write to prog_size aligned units, but eagerly cache as
   much as possible. There's no downside to using the full cache in
   program operations.

2. Add a hint parameter to cached reads. This internal API allows callers
   to tell the cache how much data they expect to need. This avoids
   excess bus traffic, and now we can even bypass the cache if the
   caller provides enough of a buffer.

   We can still fall back to reading full cache-lines in the cases where
   we don't know how much data we need by providing the block size as
   the hint. We do this for directory fetches and for file reads.

This has immediate improvements for both metadata-log traversal and CTZ
skip-list traversal, since these both only need to read 4-byte pointers
and can always bypass the cache, allowing reuse elsewhere.
2018-10-18 10:00:49 -05:00
Christopher Haster
4db96d4d44 Changed unwritable superblock to ENOSPC for consistency
While ECORRUPT is not a wrong error code, it doesn't match other
instances of hitting a corrupt block during write. During writes, if
blocks are detected as corrupt their data is evicted and moved to a new
clean block. This means that at the end of a disk's lifetime, exhaustion
errors will be reported as ENOSPC when littlefs can't find any new block
to store the data.

This has the benefit of matching behaviour when a new file is written
and no more blocks can be found, due to either a small disk or corrupted
blocks on disk. To littlefs it's like the disk shrinks in size over
time.
2018-10-18 10:00:48 -05:00
Christopher Haster
a2532a34cd Fixed inline files when inline_max == cache_size
The initial implementation of inline files was thrown together fairly
quicky, however it has worked well so far and there hasn't been much
reason to change it.

One shortcut was to trick file writes into thinking they are writing to
imaginary blocks. This works well and reuses most of the file code
paths, as long as we don't flush the imaginary block out to disk.

Initially we did this by limiting inline_max to cache_max-1, ensuring
that the cache never fills up and gets flushed. This was a rather dirty
hack, the better solution, implemented here, is to handle the
representation of an "imaginary" block correctly all the way down into
the cache layer.

So now for files specifically, the value -1 represents a null pointer,
and the value -2 represents an "imaginary" block. This may become a
problem if the number of blocks approaches the max, however this -2
value is never written to disk and can be changed in the future without
breaking compatibility.
2018-10-18 10:00:48 -05:00
Christopher Haster
d5e800575d Collapsed recursive deorphans into a single pass
Because a block can go bad at any time, if we're unlucky, we may end up
generating multiple orphans in a single metadata write. This is
exacerbated by the early eviction in dynamic wear-leveling.

We can't track _all_ orphans, because that would require unbounded
storage and significantly complicate things, but there are a handful of
intentional orphans we do track because they are easy to resolve without
the O(n^2) deorphan scan. These are anytime we intentionally remove a
metadata-pair.

Initially we cleaned up orphans as they occur with whatever knowledge we
do have, and just accepted the extra O(n^2) deorphan scans in the
unlucky case. However we can do a bit better by being lazy and leaving
deorphaning up to the next metadata write. This needs to work with the known
orphans while still setting the orphan flag on disk correctly. To
accomplish this we replace the internal flag with a small counter.

Note, this means that our internal representation of orphans differs
from what's on disk. This is annoying but not the end of the world.
2018-10-18 10:00:48 -05:00
Christopher Haster
21217d75ad Dropped lfs_fs_getattr for the more implicit lfs_getattr("/")
This was a pretty simple oversight on my part. Conceptually, there's no
difference between lfs_fs_getattr and lfs_getattr("/"). Any operations
on directories can be applied "globally" by referring to the root
directory.

Implementation wise, this actually fixes the "corner case" of storing
attributes on the root directory, which is broken since the root
directory doesn't have a related entry. Instead we need to use the root
superblock for this purpose.

Fewer functions means less code to document and maintain, so this is a
nice benefit. Now we just have a single lfs_getattr/setattr/removeattr set
of functions along with the ability to access attributes atomically in
lfs_file_opencfg.
2018-10-18 10:00:48 -05:00
Christopher Haster
38011f4cd0 Fixed minor memory leak
- Fixed memory leak
- Change lfs_globals_zero to use memset as this
  made leak checking more effective
- Checked for leaks with valgrind
2018-10-18 10:00:48 -05:00
Christopher Haster
126ef8b07f Added allocation randomization for dynamic wear-leveling
This implements the second step of full dynamic wear-leveling, block
allocation randomization. This is the key part the uniformly distributes
wear across the filesystem, even through reboots.

The entropy actually comes from the filesystem itself, by xoring
together all of the CRCs in the metadata-pairs on the filesystem. While
this sounds like a ridiculous operation, it's easy to do when we already
scan the metadata-pairs at mount time.

This gives us a random number we can use for block allocation.
Unfortunately it's not a great general purpose random generator as the
output only changes every filesystem write. Fortunately that's exactly
when we need our allocator.

---

Additionally, the randomization created a mess for the testing
framework. Fortunately, this method of randomization is deterministic.
A very useful property for reproducing bugs.
2018-10-18 09:55:47 -05:00
Christopher Haster
e4a0d586d5 Added building blocks for dynamic wear-leveling
Initially, littlefs relied entirely on bad-block detection for
wear-leveling. Conceptually, at the end of a devices lifespan, all
blocks would be worn evenly, even if they weren't worn out at the same
time. However, this doesn't work for all devices, rather than causing
corruption during writes, wear reduces a devices "sticking power",
causing bits to flip over time. This means for many devices, true
wear-leveling (dynamic or static) is required.

Fortunately, way back at the beginning, littlefs was designed to do full
dynamic wear-leveling, only dropping it when making the retrospectively
short-sighted realization that bad-block detection is theoretically
sufficient. We can enable dynamic wear-leveling with only a few tweaks
to littlefs. These can be implemented without breaking backwards
compatibility.

1. Evict metadata-pairs after a certain number of writes. Eviction in
   this case is identical to a relocation to recover from a bad block.
   We move our data and stick the old block back into our pool of
   blocks.

   For knowing when to evict, we already have a revision count for each
   metadata-pair which gives us enough information. We add the
   configuration option block_cycles and evict when our revision count
   is a multiple of this value.

2. Now all blocks participate in COW behaviour. However we don't store
   the state of our allocator, so every boot cycle we reuse the first
   blocks on storage. This is very bad on a microcontroller, where we
   may reboot often. We need a way to spread our usage across the disk.

   To pull this off, we can simply randomize which block we start our
   allocator at. But we need a random number generator that is different
   on each boot. Fortunately we have a great source of entropy, our
   filesystem. So we seed our block allocator with a simple hash of the
   CRCs on our metadata-pairs. This can be done for free since we
   already need to scan the metadata-pairs during mount.

What we end up with is a uniform distribution of wear on storage. The
wear is not perfect, if a block is used for metadata it gets more wear,
and the randomization may not be exact. But we can never actually get
perfect wear-leveling, since we're already resigned to dynamic
wear-leveling at the file level.

With the addition of metadata logging, we end up with a really
interesting two-stage wear-leveling algorithm. At the low-level,
metadata is statically wear-leveled. At the high-level, blocks are
dynamically wear-leveled.

---

This specific commit implements the first step, eviction of metadata
pairs. Entertwining this into the already complicated compact logic was
a bit annoying, however we can combine the logic for superblock
expansion with the logic for metadata-pair eviction.
2018-10-18 09:30:45 -05:00
Christopher Haster
20b669a23d Fixed issue with big-endian CTZ lists intertwined in commit logic
Found while testing big-endian support. Basically, if littlefs is really
really unlucky, the block allocator could kick in while committing a
file's CTZ reference. If this happens, the block allocator will need to
traverse all CTZ skip-lists in memory, including the skip-list we're
committing. This means we can't convert the CTZ's endianness in place,
and need to make a copy on big-endian systems.

We rely on dead-code elimination from the compiler to make the
conditional behaviour for big-endian vs little-endian system a noop
determined by the lfs_tole32 intrinsic.
2018-10-16 20:53:25 -05:00
Christopher Haster
10f45ac02f Changed lfs_crc to match more common API
In looking at the common CRC APIs out there, this seemed the most
common. At least more common than the current modified-in-place pointer
API. It also seems to have a slightly better code footprint. I'm blaming
pointer optimization issues.

One downside is that lfs_crc can't report errors, however it was already
assumed that lfs_crc can not error.
2018-10-16 20:53:19 -05:00
Christopher Haster
3b3981eb74 Fixed testing issues introduced by expanding superblocks
This was mostly tweaking test cases to be accommodating for variable
sized superblock-lists. Though there were a few bugs that needed fixing:
- Changed compact to use source dir for move since the original dir
  could have changed as a result of an expand.
- Created copy of current directory so we don't overwrite ourselves
  during an internal commit update.

Also made sure all of the test suites provide reproducable results when
ran independently (the entry tests were behaving differently based on
which tests were ran before).

(Some where legitimate test failures)
2018-10-16 20:18:24 -05:00
Christopher Haster
d8f930eeab Modified CTZ struct type to make space for erased files in the future
In v1, littlefs didn't trust blocks that were been previously erased and
conservatively erased any blocks before writing to them. This was a part
of the design since the beginning because of the complexity of managing
erased blocks when we can lose power at any time.

However, we theoretically could keep track of files that have been
properly erased by marking them with an "erased bit". A file marked this
way could be opened and appended to without needing to COW the last
block. The requirement would be that the "erased bit" is cleared during
a write, since a power-loss would require that littlefs no longer trust
the erased state of the file.

This commit just shuffles the struct types around to make space for an
"erased bit" in the struct type field to be added in the future. This
ordering also makes more sense, since there will likely be more file
representations than directory representations on disk.
2018-10-16 20:07:19 -05:00
Christopher Haster
7c70068b89 Added root entry and expanding superblocks
Expanding superblocks has been on my wishlist for a while. The basic
idea is that instead of maintaining a fixed offset blocks {0, 1} to the
the root directory (1 pointer), we maintain a dynamically sized
linked-list of superblocks that point to the actual root. If the number
of writes to the root exceeds some value, we increase the size of the
superblock linked-list.

This can leverage existing metadata-pair operations. The revision count for
metadata-pairs provides some knowledge on how much wear we've put on the
superblock, and the threaded linked-list can also be reused for this
purpose. This means superblock expansion is both optional and cheap to
implement.

Expanding superblocks helps both extremely small and extremely large filesystem
(extreme being relative of course). On the small end, we can actually
collapse the superblock into the root directory and drop the hard requirement
of 4-blocks for the superblock. On the large end, our superblock will
now last longer than the rest of the filesystem. Each time we expand,
the number of cycles until the superblock dies is increased by a power.

Before we were stuck with this layout:
level  cycles  limit    layout
1      E^2     390 MiB  s0 -> root

Now we expand every time a fixed offset is exceeded:
level  cycles  limit    layout
0      E       4 KiB    s0+root
1      E^2     390 MiB  s0 -> root
2      E^3     37 TiB   s0 -> s1 -> root
3      E^4     3.6 EiB  s0 -> s1 -> s2 -> root
...

Where the cycles are the number of cycles before death, and the limit is
the worst-case size a filesystem where early superblock death becomes a
concern (all writes to root using this formula: E^|s| = E*B, E = erase
cycles = 100000, B = block count, assuming 4096 byte blocks).

Note we can also store copies of the superblock entry on the expanded
superblocks. This may help filesystem recover tools in the future.
2018-10-16 19:30:56 -05:00
Christopher Haster
c3e36bd2a7 Standardized naming for internal functions
- lfs_pairblah -> lfs_pair_blah
- lfs_ctzblah -> lfs_ctz_blah
- lfs_tagblah -> lfs_tag_blah
- lfs_globalblah -> lfs_global_blah
- lfs_commitblah -> lfs_commit_blah
2018-10-16 11:35:39 -05:00
Christopher Haster
6d0a6fc462 Merge remote-tracking branch 'origin/master' into v2-alpha 2018-10-16 11:33:00 -05:00
Christopher Haster
3186e89b14 Changed littlefs-fuse target for testing purposes
This is a downside caused by relying on and external repo for testing,
but also storing the CI configuration inside this repo. Fortunately we
can use a temporary v2-alpha branch in the FUSE repo mirroring the
v2-alpha branch for testing.
2018-10-16 09:42:46 -05:00
Christopher Haster
dbcbe4e088 Changed name of upper-limits from blah_size to blah_max
This standardizes the naming between the LFS_BLAH_MAX macros and the
blah_max configuration in the lfs_config structure.
2018-10-16 09:42:46 -05:00
Christopher Haster
213530c376 Changed LFS_ERR_CORRUPT to match EILSEQ instead of EBADE
LFS_ERR_CORRUPT is unfortunately not a well defined error code. It's
very important in the context of littlefs, but missing from the standard
error codes defined in Linux.

After some discussions with other developers, it was encouraged to use
the encoding for EILSEQ over EBADE for representing on disk corrupt, as
EILSEQ implies that there is something wrong with the data.

I've changed this now to take advantage of the breaking changes in v2 to
avoid a risky change to a return value.
2018-10-16 09:40:05 -05:00
Christopher Haster
a88230ae6a Updated custom attribute documentation and tweaked nonexistant attributes
Because of limitations in how littlefs manages attributes on disk,
littlefs views zero-length attributes and missing attributes as the same
thing. The simpliest implementation of attributes mirrors this behaviour
transparently for the user.
2018-10-16 09:20:44 -05:00
Christopher Haster
f369f80540 Added tests for global state stealing
State stealing is a tricky part of managing the xored-globals. When
removing a metadata-pair from the metadata chain, whichever
metadata-pair does the removing is also responsible for stealing the
removed metadata-pair's global delta and incorporating it into it's own
global delta. Otherwise the global state would become corrupted.
2018-10-16 09:18:18 -05:00
Christopher Haster
1941bbda76 Cleaned up config options
- Updated documentation where needed
- Added asserts which take into account relationships with the new
  cache_size configuration
- Restructured ordering to be consistent for the three main
  configurables: LFS_ATTR_MAX, LFS_NAME_MAX, and LFS_INLINE_MAX
2018-10-16 09:07:22 -05:00
Christopher Haster
3cfa08602a Introduced cache_size as alternative to hardware read/write sizes
The introduction of an explicit cache_size configuration allows
customization of the cache buffers independently from the hardware
read/write sizes.

This has been one of littlefs's main handicaps. Without a distinction
between cache units and hardware limitations, littlefs isn't able to
read or program _less_ than the cache size. This leads to the
counter-intuitive case where larger cache sizes can actually be harmful,
since larger read/prog sizes require sending more data over the bus if
we're only accessing a small set of data (for example the CTZ skip-list
traversal).

This is compounded with metadata logging, since a large program size
limits the number of commits we can write out in a single metadata
block. It really doesn't make sense to link program size + cache
size here.

With a separate cache_size configuration, we can be much smarter about
what we actually read/write from disk.

This also simplifies cache handling a bit. Before there were two
possible cache sizes, but these were rarely used. Note that the
cache_size is NOT written to the superblock and can be freely changed
without breaking backwards compatibility.
2018-10-16 08:32:01 -05:00
Christopher Haster
97f35c3e05 Simplified the internal xored-globals implementation
There wasn't much use (and inconsistent compiler support) for storing
small values next to the unaligned lfs_global_t struct. So instead, I've
rounded the struct up to the nearest word to try to take advantage of
the alignment in xor and memset operations.

I've also moved the global fetching into lfs_mount, since that was the
only use of the operation. This allows for some variable reuse in the
mount function.
2018-10-16 08:28:14 -05:00
Christopher Haster
35f68d28cc Squished in-flight files/dirs into single list
This is an effort to try to consolidate the handling of in-flight files
and dirs opened by the user (and possibly opened internally). Both files
and dirs have metadata state that need to be kept in sync by the commit
logic.

This metadata state is mostly contained in the lfs_mdir_t type, which is
present in both the lfs_file_t and lfs_dir_t. Unfortunately both of
these structs have some relatively unrelated metadata that needs to be
kept in sync:
- Files store an id representing the open file
- Dirs store an id during iteration

While these take up the same space, they unfortunately need to be
managed differently by the commit logic.

The best solution I can come up with is to simple store a general
purpose list and tag both structures with LFS_TYPE_REG and LFS_TYPE_DIR
respectively. This is kinda funky, but wins out over duplicated the
commit logic.
2018-10-16 08:18:21 -05:00
Christopher Haster
bd1e0c4059 Cleaned up several TODOs
Other than removed outdated TODOs, there are several tweaks:
- Standardized naming of fs-level functions (mostly internal names)
- Tweaked low-level use of subtype to hopefully take advantage of
  redundant code removal
- Moved root-handling into lfs_dir_getinfo
- Updated DEBUG statements around move/orphan fixes
- Removed trailing 1s in type fields
- Removed unused code
2018-10-16 08:12:27 -05:00
Christopher Haster
01d837e08d Removed redundant lfs_scan in lfs_init
Interestingly enough, lfs_scan is only needed during mount, as the state
of the filesystem is well know during format.
2018-10-16 08:05:17 -05:00
Christopher Haster
112fefc068 Added back big-endian support again on the new metadata structures
The only interesting thing to note is that we now have to also support
le16 due to storing the id outside of tags in the globals structure.
2018-10-16 08:03:30 -05:00
Christopher Haster
64df0a5e20 Added orphan bit to xored-globals
Unfortunately for us, even with the new ability to store global state,
orphans can not be handled as gracefully as moves. This is due to the
fact that directory operations can create an unbounded number of
orphans. It's usually small, the fact that it's unbounded means we can't
store the orphan info in xored-globals.

However, one thing we can do to leverage the xored-global state is store
a bit indicating if _any_ orphans are present. This means in the common
case we can completely avoid the deorphan step, while only using a
single bit of the global state, which is effectively free since we can
store it in the globals tag itself.

If a littlefs drive does not want to consider the orphan bit, it's free
to use the previous behaviour of always checking for orphans on first
write.
2018-10-16 07:48:15 -05:00
Christopher Haster
1a58ba799c Fixed ENOSPC issues with zero-granularity blocks
Result of testing on zero-granularity blocks, where the prog size and
read size equals the block size. This represents SD cards and other
traditional forms of block storage where we don't really get a benefit
from the metadata logging.

Unfortunately, since updates in both are tested by the same script,
we can't really use simple bash commands. Added a more complex
script to simulate corruption. Fortunately this should be more robust
than the previous solutions.

The main fixes were around corner cases where the commit logic fell
apart when it didn't have room to complete commits, but these were
fixable in the current design.
2018-10-16 07:41:56 -05:00
Christopher Haster
105907ba66 Cleaned up config usage in file logic
The main change here was to drop the in-place twiddling of custom
attributes to match the internal attribute structures. The original
thought was that this could allow the compiler to garbage collect more
of the custom attribute logic when not used, but since this occurs in
the common lfs_file_opencfg function, gc can't really happen.

Not twiddling the user's structure is the polite thing to do, opens up
the ability to store the lfs_attr structure in ROM, and avoids surprising
the user if they attempt to use the structure for their own purposes.

This means we can make the lfs_attr structure const and rely on the list
in the lfs_file_config structure, similar to how we rely on the global
lfs_config structure.

Some other tweaks:
- Dropped the global file_buffer, replaced entirely by per-file buffers.
- Updated LFS_INLINE_MAX and LFS_ATTR_MAX to correct values
- Added workaround for compiler bug related to zero initializer:
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53119
2018-10-16 07:31:24 -05:00
Christopher Haster
df1b607351 Removed the implicit lfs_t parameter to lfs_traverse
This is a very minor thing but it has been bugging me. On one hand, all
a callback ever needs is a single pointer for context. On the other
hand, you could make the argument that in the context of littlefs, the
lfs_t struct represents global state and should always be available to
callbacks passed to littlefs.

In the end I'm sticking with only a single context pointer, since this
is satisfies the minimum requirements and has the highest chance of
function reuse. If a user needs access to the lfs_t struct, it can be
passed by reference in the context provided to the callback.

This also matches callbacks used in other languages with more emphasis
on objects and classes. Usually the callback doesn't get a reference to
the caller.
2018-10-16 07:25:22 -05:00
Christopher Haster
225706044e Fixed test bugs around handling corruption
The main thing to consider was how lfs_dir_fetchwith reacts to
corruption it finds and to make sure falling back to old values works
correctly.

Some of the tricky bits involved making sure we could fall back to both old
commits and old metadata blocks while still handling things like
synthetic moves correctly.
2018-10-16 07:15:59 -05:00
Christopher Haster
3e246da52c Fixed the orphan test to handle logging metadata-pairs
The main issue here was that the old orphan test relied on deleting the
block that contained the most recent update. In the new design this
doesn't really work since updates get appended to metadata-pairs
incrementally.

This is fixed by instead using the truncate command on the appropriate
block. We're now passing orphan tests.
2018-10-16 07:04:44 -05:00
Christopher Haster
15d156082c Added support for custom attributes leveraging the new metadata logging
Now that littlefs has been rebuilt almost from the ground up with the
intention to support custom attributes, adding in custom attribute
support is relatively easy.

The highest bit in the 9-bit type structure indicates that an attribute
is a user-specified custom attribute. The user then has a full 8-bits to
specify the attribute type. Other than that, custom attributes are
treated the same as system-level attributes.

Also made some tweaks to custom attributes:
- Adopted the opencfg for file-level attributes provided by dpgeorge
- Changed setattrs/getattrs to the simpler setattr/getattr functions
  users will probably be more familiar with. Note that multiple
  attributes can still be committed atomically with files, though not
  with directories.
- Changed LFS_ATTRS_MAX -> LFS_ATTR_MAX since there's no longer a global
  limit on the sum of attribute sizes, which was rather confusing.
  Though they are still limited by what can fit in a metadata-pair.
2018-10-16 06:41:59 -05:00
Christopher Haster
3914cdf39f Pulled in fixes for additional path corner cases
Pulled in 015b86b. Merging this now avoids duplicate effort restructuring
the path lookup logic.
2018-10-16 06:36:00 -05:00
Christopher Haster
392b2ac79f Refactored the updates of in-flight files/dirs
Updated to account for changes as a result of commits/compacts. And
changed instances of iteration over both files and dirs to use a single
nested loop.

This does rely implicitly on the structure layout of dirs/files and
their location in lfs_t, which isn't great. But it gets the job done
with less code duplication.
2018-10-16 06:00:18 -05:00
Christopher Haster
d9a24d0a2b Fixed move handling when caught in a relocate
This was a surprisingly tricky issue. One of the subtle requirements for
the new move handling to work is that the block containing the move does
not change until the move is resolved. Initially, this seemed easy to
implement, given that a move is always immediately followed by its
resolution.

However, the extra metadata-pair operations needed to maintain integrity
present a challenge. At any commit, a directory block may end up moved
as a side effect of relocation due to a bad block.

The fix here is to move the move resolution directly into the commit
logic. This means that any commit to a block containing a move will be
implicitly resolved, leaving the later attempt at move resolution as a
noop.

This fix required quite a bit of restructuring, but as a nice
side-effect some of the complexity around moves actually went away.
Additionally, the new move handling is surprisingly powerful at
combining moves with nearby commits. And we now get same-metadata-pair
renames for free! A win for procrasination on that minor feature.
2018-10-16 05:28:00 -05:00
Christopher Haster
5d24e656f1 Cleaned up commit logic and function organization
Restrctured function organization to make a bit more sense, and made
some small refactoring tweaks, specifically around the commit logic and
global related functions.
2018-10-16 05:19:38 -05:00
Christopher Haster
d3f3711560 Cleaned up attributes and related logic
The biggest change here is to make littlefs less obsessed with the
lfs_mattr_t struct. It was limiting our flexibility and can be entirely
replaced by passing the tag + data explicitly. The remaining use of
lfs_mattr_t is specific to the commit logic, where it replaces the
lfs_mattrlist_t struct.

Other changes:
- Added global lfs_diskoff struct for embedding disk references inside
  the lfs_mattr_t.
- Reordered lfs_mattrlist_t to squeeze out some code savings
- Added commit_get for explicit access to entries from unfinished
  metadata-pairs
- Parameterized the "stop_at_commit" flag instead of hackily storing it
  in the lfs_mdir_t temporarily
- Changed return value of lfs_pred to error-only with ENOENT representing
  a missing predecessor
- Adopted const where possible
2018-10-16 05:03:44 -05:00
Christopher Haster
5fc53bd726 Changed internal functions to return tags over pointers
One neat (if gimmicky) trick, is that each tag has a valid bit in the
highest bit position of the 32-bit word. This is used to determine when
to stop a fetch operation, but after fetch, the bit is free to use in
the driver. This means we can create a typed-union of sorts with error
codes and tags, returning both as the return value from a function.

Say what you will about this trick, it does have a significant impact on
code size. I suspect this is primarily due to the compiler having a hard
time optimizing around pointer access.
2018-10-16 04:50:35 -05:00
Christopher Haster
2b35c36b67 Renamed tag functions and macros
- lfs_tagverb -> lfs_tag_verb
- lfs_mktag -> LFS_MKTAG (it's a macro now)
- LFS_STRUCT_THING -> LFS_THINGSTRUCT
2018-10-16 04:47:20 -05:00
Christopher Haster
7c88bc96b6 Restructured get/traverse functions
While it makes sense to reuse as many code paths as possible, it turns
out that the logic behind the traversal of littlefs's metadata-pairs is
so simple that it's actually cheaper to duplicate the traversal code
where needed.

This means instead of the code path move -> traverse -> movescan -> get
-> traverse -> getscan, we can use the relatively flatter code path of
move -> get.
2018-10-16 04:40:32 -05:00
Christopher Haster
e3b867897a Modified results from find-like functions to use tags
Tags offer all of the necessary info from the find functions (which
makes sense, this is the structure that stores the info on disk).
Passing around a single tag instead of separate id and type fields
simplifies the internal functions while leverages the tag's compactness.
2018-10-16 04:37:23 -05:00
Christopher Haster
67d9f88b0e Combined get functions into one
Unfortunately, the three different sets of get functions were not
contributing very much, and having three different get functions means
we may be wasting code on redundant code paths.

By dropping the user of the lfs_mattr_t struct in favor of a buffer, we
can combine the three code paths with a bit of tweaking.
2018-10-16 04:34:07 -05:00
Christopher Haster
7ad9700d9e Integrated findscan into fetch as a built in side effect
Now that littlefs's fetchwith operations have stabilized a bit, there's
actually only a single fetchwith operation, the findscan function.
Given that there's no need for the internal functions to be a forward
compatible API, we can integrate the findscan behaviour directly into
fetchwith and avoid the (annoyingly) costly generalization overhead.

As an added benefit, we can easily add additional tag modifications
during fetch, such as the synthetic moves needed to resolve in-flight
move operations without disk modifications.
2018-10-16 04:25:24 -05:00
Christopher Haster
fe31f79b5f Consolidated find/parent scanning functions
An interesting observation about the find and parent scanning functions
is that at their core, they're both actually doing the same operation.
They search a metadata-pair during fetch for an entry uses the entry data
instead of the entry tag. This means we can combine these functions and
get a decent code savings.

It's a little bit trickier because pair ordering isn't guaranteed. But
to work around that we can simply search for both pair orderings. It's a
bit more expensive but may be worth the code savings. A fancier
implementation in the future can avoid the 2x lfs_parent scans.
2018-10-16 04:14:48 -05:00
Christopher Haster
fd121dc2e2 Dropped "has id" bit encoding in favor of invalid id
I've been trying to keep tag types organized with an encoding that hints
if a tag uses its id field for file ids. However this seem to have been
a mistake. Using a null id of 0x3ff greatly simplified quite a bit of
the logic around managing file related tags.

The downside is one less id we can use, but if we look at the encoding
cost, donating one full bit costs us 2^9 id permutations vs 1 id
permutation. So even if we had a perfect encoding it's in our favor to
use a null id. The cost of null ids is code size, but with the
complexity around figuring out if a type used it's id or not it just
works out better to use a null id.
2018-10-15 18:49:01 -05:00
Christopher Haster
b7bd34f461 Restructured types to use a more flexible bit encoding
Recall that the 32-bit tag structure contains a 9-bit type. The type
structure then decomposes into a bit more information:
[---   9   ---]
[1|- 4 -|- 4 -]
 ^   ^     ^- specific type
 |   \------- subtype
 \----------- user bit

The main change is an observation from moving type info to the name tag
from the struct tag. Since we don't need the type info in the struct
tag, we can significantly simplify the type structure.
2018-10-15 18:34:26 -05:00
Christopher Haster
c1103efb53 Changed type info to be retrieved from name tag instead of struct tag
Originally, I had type info encoded in the struct tag. This initially
made sense because the type info only directly impacts the struct tag.
However this was a case of focusing too much on the details instead of
the bigger picture.

A more file operations need to figure out the type of a file, but it's
only actually a small number of file operations that need to interact
with the file's structure. For the common case, providing the type of
the file early shortens operations by a full tag access.

Additionally, but storing the type in the file name tag, this opens up
the struct tag to use those bits for storing more struct descriptions.
2018-10-15 18:27:28 -05:00
Christopher Haster
d7b0652936 Removed old move logic, now passing move tests
The introduction of xored-globals required quite a bit of work to
integrate. But now that that is working, we can strip out the old move
logic.

It's worth noting that the xored-globals integration with commits is
relatively complex and subtle.
2018-10-15 16:03:18 -05:00
Christopher Haster
2ff32d2dfb Fixed bug where globals were poisoning move commits
The issue lies in the reuse of the id field for globals. Before globals,
the only tags with a non-null (0x3ff) id field were names, structs, and
other file-specific metadata. But globals are also using this field for
the indirect delete, since otherwise the globals structure would be very
unaligned (74-bits long).

To make matters worse, the id field for globals contains the delta used
to reconstruct the globals at mount time. Which means the id field could
take on very absurd values and break the dir fetch logic if we're not
careful.

Solution is to use the scope portion of the type field where necessary,
although unforunately this does add some code cost.
2018-10-15 15:56:04 -05:00
Christopher Haster
b46fcac585 Fixed issues with finding wrong ids after bad commits
Unfortunately, the behaviour needed of lfs_dir_fetchwith is as subtle as
it is important. When fetching from a block corrupted by power-loss,
lfs_dir_fetch must be able to rewind any state it picks up to before the
corruption. This is not limited to the directory state, but includes
find results and other side-effects.

This gets a bit complicated when trying to generalize littlefs's
fetchwith mechanics. Being able to scan a directory block during a fetch
greatly impacts the runtime of littlefs operations, but if the state is
generic how do we know what to rollback to?

The fix here is to leave the management of rolling back state to the
fetchwith match functions, and transparently pass a CRC tag to indicate
the temporary state can be saved.
2018-10-13 19:46:38 -05:00
Christopher Haster
cebf7aa0fe Switched back to simple deorphan-step on directory remove
Originally I tried to reuse the indirect delete to accomplish truely
atomic directory removes, however this fell apart when it came to
implementing directory removes as a side-effect of renames.

A single indirect-delete simply can't handle renames with removes as
a side effects. When copying an entry to its destination, we need to
atomically delete both the old entry, and the source of our copy. We
can't delete both with only a single indirect-delete. It is possible to
accomplish this with two indirect-deletes, but this is such an uncommon
case that it's really not worth supporting efficiently due to how
expensive globals are.

I also dropped indirect-deletes for normal directory removes. I may add
it back later, but at the moment it's extra code cost for that's not
traveled very often.

As a result, restructured the indirect delete handling to be a bit more
generic, now with a multipurpose lfs_globals_t struct instead of the
delete specific lfs_entry_t struct.

Also worked on integrating xored-globals, now with several primitive
global operations to manage fetching/updating globals on disk.
2018-10-13 19:35:45 -05:00
Christopher Haster
3ffcedb95b Restructured tags to better support xored-globals
32-bit tag structure:
[---        32       ---]
[1|- 9 -|- 10 -|-- 12 --]
 ^   ^     ^       ^- entry length
 |   |     \--------- file id
 |   \--------------- tag type
 \------------------- valid

In this tag, the type decomposes into some more information:
[---      9      ---]
[1|- 2 -|- 3 -|- 3 -]
 ^   ^     ^     ^- struct
 |   |     \------- type
 |   \------------- scope
 \----------------- user

The change in this encoding is the addition of a global scope:
LFS_SCOPE_STRUCT = 0 00 xxx xxx
LFS_SCOPE_ENTRY  = 0 01 xxx xxx
LFS_SCOPE_DIR    = 0 10 xxx xxx
LFS_SCOPE_FS     = 0 11 xxx xxx
LFS_SCOPE_USER   = 1 xx xxx xxx
2018-10-13 19:12:35 -05:00
Christopher Haster
e39f7e99d1 Introduced xored-globals logic to fix fundamental problem with moves
This was a big roadblock for a while: with the new feature of inlined
files, the existing move logic was fundamentally flawed.

To pull off atomic moves between two different metadata-pairs, littlefs
uses a simple, if a bit clumsy trick.
1. Marks entry as "moving"
2. Copies entry to new metadata-pair
3. Deletes old entry

If power is lost before the move operation is completed, we will find the
"moving" tag. This means there may or may not be an incomplete move on
the filesystem. In this case, we simply search for the moved entry, if
we find it, we remove the old entry, otherwise we just remove the
"moving" tag.

This worked perfectly, until we introduced inlined files. See, unlike
the existing directory and ctz entries, inlined files have no guarantee
they are unique. There is nothing we can search for that will allow us
to find a moved file unless we assign entries globally-unique ids. (note
that moves are fundamentally rename operations, so searching for names
does not make sense).

---

Solving this problem required completely restructuring how littlefs
handled moves and pulled out a really old idea that had been left in the
cutting room floor back when littlefs was going through many
designs: xored-globals.

The problem xored-globals solves is the need to maintain some global state
via commits to these distributed, independent metadata-pairs. The idea
is that we can use some sort of symmetric operation, such as xor, to
introduces deltas of the global state that can be committed atomically
along with any other info to these metadata-pairs.

This means that to figure out our global state, we xor together the global
delta stored in every metadata-pair.

Which means any commit can update the global state atomically, opening
up a whole new set atomic possibilities.

There is a couple of downsides. These globals may end up with deltas on
every single metadata-pair, effectively duplicating the data for each
block. Additionally, these globals need to have multiple copies in RAM.
This means and globals need to be a bounded size and very small, since even
small globals will have a large footprint.

---

On top of xored-globals, it's trivial to fix our move logic. Here we've
added an indirect delete tag which allows us to atomically specify a
delete of any entry on the filesystem.

Our move operation is now:
1. Copy entry to new metadata-pair and atomically xor globals to
   indirectly delete our original entry.
2. Delete the original entry and xor globals to remove the indirect
   delete.

Extra exciting is that this now takes our relatively clumsy move
operation into a sexy guaranteed O(1) move operation with no searching
necessary (though we do need to xor globals during mount).

Also reintroduced entry struct, now with a specific purpose to describe
the metadata-pair + id combo needed by indirect deletes to locate an
entry.
2018-10-13 18:35:33 -05:00
Christopher Haster
116c1e76de Adopted EISDIR as internal error for root path as argument
Unfortunately, it's hard to make directory lookups for root not a
special case, and we don't like special cases when we're trying to keep
code size small.

Since there are a handful of code paths where opening root should return
EISDIR (such as lfs_file_open("/")), using EISDIR to note that the
argument is in fact a path to the root.

This is needed because we no longer look up an entries contents in
lfs_dir_find for free, since entries are more ephemeral.
2018-10-13 18:25:08 -05:00
Christopher Haster
f458da4b7c Added the internal meta-directory structure
Similarly to the internal "meta-attributes", I was finding quite a bit
of need for an internal structure that mirrors the user-facing directory
structure for when I need to do an operation on a metadata-pair, but
don't need all of the state associated with a fully iterable directory
chain.

lfs_mdir_t - meta-directory, describes a single metadata-pair
lfs_dir_t  - directory, describes an iterable directory chain

While it may seem complex to have all these structures lying around,
they only complicate the code at compile time. To the machine, any
number of nested structures all looks the same.
2018-10-13 18:20:44 -05:00
Christopher Haster
eaa9220aad Renamed lfs_entry_t -> lfs_mattr_t
Attributes are used to describe more than just entries, so calling these
list of attributes "entries" was inaccurate. However, the name
"attributes" would conflict with "user attributes", user-facing
attributes with a very similar purpose. "user attributes" must be kept
distinct due to differences in binary layout (internal attributes can
use a more compact tag+buffer representation, but expecting users to
jump through hoops to get their data to look like that isn't very
user-friendly).

Decided to go with "mattr" as shorthand for "meta-attributes", similar
to "metadata".
2018-10-13 18:14:38 -05:00
Christopher Haster
9278b17537 Trimmed old names and functions from the code base
I've found using temporary names to duplicate functions temporarily
(lfs_dir_commit + lfs_dir_commit_) is a great way to introduce sweeping
changes while keeping the code base functional and (mostly) passing
tests.

It does mean at some point I need to deduplicate all these functions.
2018-10-13 18:13:12 -05:00
Christopher Haster
85a9638d9f Fixed issues discovered around testing moves
lfs_dir_fetchwith did not recover from failed dir fetches correctly,
added a temporary dir variable to hold dir contents while being
populated, allowing us to fall back to a known good dir state if a
commit is corrupted.

There is a RAM cost, but the upside is that our lfs_dir_fetchwith
actually works.

Also added better handling of move ids during some get functions.
2018-10-13 18:08:28 -05:00
Christopher Haster
483d41c545 Passing all of the basic functionality tests
Integration with the new journaling metadata has now progressed to the
point where all of the basic functionality tests are passing. This
includes:
- test_format
- test_dirs
- test_files
- test_seek
- test_truncate
- test_interspersed
- test_paths

Some of the fixes:
- Modified move to correctly change entry ids
- Called lfs_commit_move directly from compact, avoiding commit parsing
  logic during a compact
- Opened up commit filters to be passed down from compact for moves
- Added correct drop logic to lfs_dir_delete
- Updated lfs_dir_seek to use ids instead of offsets
- Caught id updates manually where possible (this needs to be fixed)
2018-10-13 17:58:56 -05:00
Christopher Haster
11a3c8d062 Continued progress with reintroducing testing on the new metadata logging
Now with some tweaks to commit/compact, and a committers for entrylists and
moves specifically. No longer relying on a commitwith callback, the
types of commits are now infered from their tags.

This means we can now commit things atomically with special commits,
such as moves. Now lfs_rename can move entries to new names correctly.
2018-10-13 17:47:01 -05:00
Christopher Haster
0bdaeb7f8b More testing progress, combined dir/commit traversal
Passing more tests now with the journalling change, but still have more
work to do.

The most humorous bug was a bug where during the three step move
process, the entry move logic would dumbly copy over any tags associated
with the moving entry, including the tag used to temporarily mark the
entry as "moving".

Also combined dir and commit traversal using a "stop_at_commit" flag in
directory struct as a short-term hack to combine the code paths.
2018-10-13 17:44:37 -05:00
Christopher Haster
0405ceb171 Cleaned up enough things to pass basic file testing 2018-10-13 13:41:05 -05:00
Christopher Haster
a3c67d9697 Reorganized the internal operations to make more sense
Also refactored lfs_dir_compact a bit, adding begin and end as arguments
since they simplify a bit of the logic and can be found out much easier
earlier in the commit logic.

Also changed add -> append and drop -> delete and cleaned up some of the
logic around there.
2018-10-13 13:38:04 -05:00
Christopher Haster
0695862b38 Completed transition of files with journalling metadata
This was the simpler part of transitioning since file operations only
interact with metadata at sync time.

Also switched from array to linked-list of entries.
2018-10-13 13:33:29 -05:00
Christopher Haster
fe553e8af4 More progress integrating journaling
- Integrated into lfs_file_t_, duplicating functions where necessary
- Added lfs_dir_fetchwith_ as common parent to both lfs_dir_fetch_ and
  lfs_dir_find_
- Added similar parent with lfs_dir_commitwith_
- Made matching find/get operations with getbuffer/getentry and
  findbuffer/findentry
- lfs_dir_alloc now populates tail, since almost all directory block
  allocations need to populate tail
2018-10-13 13:31:47 -05:00
Christopher Haster
87f3e01a17 Progressed integration of journaling metadata pairs
- Integrated journaling into lfs_dir_t_ struct and operations,
  duplicating functions where necessary
- Added internal lfs_tag_t and lfs_stag_t
- Consolidated lfs_region and lfs_entry structures
2018-10-13 13:31:42 -05:00
Christopher Haster
8070abec34 Added rudimentary framework for journaling metadata pairs
This is a big change stemming from the fact that resizable entries
were surprisingly complicated to implement and came in with a sizable
code cost.

The theory is that the journalling has a comparable cost to resizable
entries. Both need to handle overflowing blocks, and managing offsets is
comparable to managing attribute IDs. But by jumping all the way to full
journaling, we can statically wear-level the metadata written to
metadata pairs.

The idea of journaling littlefs's metadata has been mentioned several times in
discussions and fits well into how littlefs works. You could even view the
existing metadata log as a log of size 2.

The downside of this approach is that changing the metadata in this way
would break compatibility from the existing layout on disk. Something
that resizable entries does not do.

That being said, adopting journaling at the metadata layer offers a big
improvement to littlefs's performance and wear-leveling, with very
little cost (maybe even none or negative after resizable entries?).
2018-10-13 13:22:53 -05:00
Christopher Haster
61f454b008 Added tests for resizable entries and custom attributes
Also found some bugs. Should now have a good amount of confidence in
these features.
2018-10-09 23:02:57 -05:00
Christopher Haster
ea4ded420c Fixed big-endian support again
This is what I get for not runing CI on a local development branch.
2018-10-09 23:02:57 -05:00
Christopher Haster
2a8277bd4d Added test coverage for filesystems with no inline files 2018-10-09 23:02:57 -05:00
Christopher Haster
746b90965c Added lfs_fs_size for finding a count of used blocks
This has existed for some time in the form of the lfs_traverse
function, through which a user could provide a simple callback that
would just count the number of blocks lfs_traverse finds. However,
this approach is relatively unconventional and has proven to be confusing
for most users.
2018-10-09 23:02:57 -05:00
Christopher Haster
93244a3734 Added file-level and fs-level custom attribute APIs
In the form of lfs_file_setattr, lfs_file_getattr, lfs_fs_setattr,
lfs_fs_getattr.

This enables atomic updates of custom attributes as described in
6c754c8, and provides a custom attribute API that allows custom attributes
to be stored on the filesystem itself.
2018-10-09 23:02:50 -05:00
Christopher Haster
636c0ed3d1 Modified commit regions to work better with custom attributes
Mostly just removed LFS_FROM_DROP and changed the DSL grammar a bit to
allow drops to occur naturally through oldsize -> newsize diff expressed
in the region struct. This prevents us from having to add a drop every
time we want to update an entry in-place.
2018-10-09 23:02:09 -05:00
Christopher Haster
6c754c8023 Added support for atomically committing custom attributes
Although it's simple and probably what most users expect, the previous
custom attributes API suffered from one problem: the inability to update
attributes atomically.

If we consider our timestamp use case, updating a file would require:
1. Update the file
2. Update the timestamp

If a power loss occurs during this sequence of updates, we could end up
with a file with an incorrect timestamp.

Is this a big deal? Probably not, but it could be a surprise only found
after a power-loss. And littlefs was developed with the _specifically_
to avoid suprises during power-loss.

The littlefs is perfectly capable of bundling multiple attribute updates
in a single directory commit. That's kind of what it was designed to do.
So all we need is a new committer opcode for list of attributes, and
then poking that list of attributes through the API.

We could provide the single-attribute functions, but don't, because the
fewer functions makes for a smaller codebase, and these are already the
more advanced functions so we can expect more from users. This also
changes semantics about what happens when we don't find an attribute,
since erroring would throw away all of the other attributes we're
processing.

To atomically commit both custom attributes and file updates, we need a
new API, lfs_file_setattr. Unfortunately the semantics are a bit more
confusing than lfs_setattr, since the attributes aren't written out
immediately.
2018-10-09 23:02:09 -05:00
Christopher Haster
6ffc8d3480 Added simple custom attributes
A much requested feature (mostly because of littlefs's notable lack of
timestamps), this commits adds support for user-specified custom
attributes.

Planned (though underestimated) since v1, custom attributes provide a
route for OSs and applications to provide their own metadata in
littlefs, without limiting portability.

However, unlike custom attributes that can be found on much more
powerful PC filesystems, these custom attributes are very limited,
intended for only a handful of bytes for very important metadata. Each
attribute has only a single byte to identify the attribute, and the
size of all attributes attached to a file is limited to 64 bytes.

Custom attributes can be accessed through the lfs_getattr, lfs_setattr,
and lfs_removeattr functions.
2018-10-09 23:02:09 -05:00
Christopher Haster
65ea6b3d0f Bumped versions, cleaned up some TODOs and missing comments 2018-10-09 23:02:09 -05:00
Christopher Haster
6774276124 Expanded inline files up to a limit of 1023 bytes
One of the big benefits of inline files is that small files no longer need to
take up a full block. This opens up an opportunity to provide much better
support for storage devices with only a handful of very large blocks. Such as
the internal flash found on most microcontrollers.

After investigating some use cases for a filesystem on internal flash,
it has become apparent that the 255-byte limit is going to be too
restrictive to be useful in many cases. Most uses I found needed files
~4-64 bytes in size, but it wasn't uncommon to find files ~512 bytes in
length.

To try to remedy this, I've pushed the 255 byte limit up to 1023 bytes,
by stealing some bits from the previously-unused attributes's size.
Unfortunately this limits attributes to 63 bytes in total and has a
minor code cost, but I'm not sure even 1023 bytes will be sufficient for
a lot of cases.

The littlefs will probably never be as efficient with internal flash as
other filesystems such as SPIFFS, it just wasn't designed for this sort of
limited geometry. However, this feature has been heavily requested, even
with limitations, because of the opportunity for code reuse on
microcontrollers with both internal and external flash.
2018-10-09 23:02:09 -05:00
Christopher Haster
6362afa8d0 Added disk-backed limits on the name/attrs/inline sizes
Being a portable, microcontroller-scale embedded filesystem, littlefs is
presented with a relatively unique challenge. The amount of RAM
available is on completely different scales from machine to machine, and
what is normally a reasonable RAM assumption may break completely on an
embedded system.

A great example of this is file names. On almost every PC these days, the limit
for a file name is 255 bytes. It's a very convenient limit for a number
of reasons. However, on microcontrollers, allocating 255 bytes of RAM to
do a file search can be unreasonable.

The simplest solution (and one that has existing in littlefs for a
while), is to let this limit be redefined to a smaller value on devices
that need to save RAM. However, this presents an interesting portability
issue. If these devices are plugged into a PC with relatively infinite
RAM, nothing stops the PC from writing files with full 255-byte file
names, which can't be read on the small device.

One solution here is to store this limit on the superblock during format
time. When mounting a disk, the filesystem implementation is responsible for
checking this limit in the superblock. If it's larger than what can be
read, raise an error. If it's smaller, respect the limit on the
superblock and raise an error if the user attempts to exceed it.

In this commit, this strategy is adopted for file names, inline files,
and the size of all attributes, since these could impact the memory
consumption of the filesystem. (Recording the attribute's limit is
iffy, but is the only other arbitrary limit and could be used for disabling
support of custom attributes).

Note! This changes makes it very important to configure littlefs
correctly at format time. If littlefs is formatted on a PC without
changing the limits appropriately, it will be rejected by a smaller
device.
2018-10-09 23:02:09 -05:00
Christopher Haster
955545839b Added internal lfs_dir_set, an umbrella to dir append/update/remove operations
This move was surprisingly complex, but offers the ultimate opportunity for
code reuse in terms of resizable entries. Instead of needing to provide
separate functions for adding and removing entries, adding and removing
entries can just be viewed as changing an entry's size to-and-from zero.

Unfortunately, it's not _quite_ that simple, since append and remove
hide some relatively complex operations for when directory blocks
overflow or need to be cleaned up.

However, with enough shoehorning, and a new committer type that allows
specifying recursive commit lists (is this now a push-down automata?),
it does seem to be possible to shove all of the entry update logic into
a single function.

Sidenote, I switched back to an enum-based DSL, since the addition of a
recursive region opcode breaks the consistency of what needs to be
passed to the DSL callback functions. It's much simpler to handle each
opcode explicitly inside a recursive lfs_commit_region function.
2018-10-09 23:02:09 -05:00
Christopher Haster
ad74825bcf Added internal lfs_dir_get to consolidate logic for reading dir entries
It's a relatively simple function but offers some code reuse as well as
making the dir entry operations a bit more readable.
2018-10-09 23:02:09 -05:00
Christopher Haster
d0e0453651 Changed how we write out superblock to use append
Making the superblock look like "just another entry" allows us to treat
the superblock like "just another entry" and reuse a decent amount of
logic that would otherwise only be used a format and mount time. In this
case we can use append to write out the superblock like it was creating
a new entry on the filesystem.
2018-10-09 23:02:09 -05:00
Christopher Haster
701e4fa438 Fixed a handful of bugs as result of testing 2018-10-09 23:02:09 -05:00
Christopher Haster
d8cadecba6 Better implementation of inline files, now with overflowing
Now when a file overflows the max inline file size, it will be correctly
written out to a proper block. Additionally, tweaked corner cases around
inline file, however this still needs significant testing.

A real neat part that surprised me is that littlefs _already_ contains
the logic for writing out inline files: in lfs_file_relocate! With a bit
of tweaking, littlefs can pull off both the overflow from inline to
normal files _and_ the relocating of bad blocks in files with the same
piece of logic.
2018-10-09 23:02:09 -05:00
Christopher Haster
836e23895a Shoehorned in hacky implementation of inline files
Proof-of-concept implementation of inline files that stores the file's
content directly in its parent's directory pair.

Inline files are indicated by a different type stored in an entry's
struct field, and take advantage of resizable entries. Where a normal
file's entry would normally hold the reference to the CTZ skip-list, an
inline file's entry contains the contents of the actual file.

Unfortunately, storing the inline file on disk is the easy part. We also
need to manage inline files in the internals of littlefs and provide the
same operations that we do on normal files, all while reusing as much
code as possible to avoid a significant increase in code cost.

There is a relatively simple, though maybe a bit hacky, solution here. If a
file fits entirely in a cache line, the file logic never actually has to go to
disk. This means we can just give the file a "pretend" block (hopefully
one that would assert if ever written to), and carry out file operations
as normal, as long as we catch the file before it exceeds the cache line
and write out the file to an actual disk.
2018-10-09 23:02:09 -05:00
Christopher Haster
fb23044872 Fixed big-endian support for entry structures 2018-10-09 23:02:09 -05:00
Christopher Haster
9273ac708b Added size field to entry structure
The size field is redundant, since an entry's size can be determined
from the nlen+elen+alen+4. However, as you may have guessed from that
expression, calculating the size this way is a bit roundabout and
inefficient. Despite its redundancy, it's cheaper to store the size in the
entry, though with a minor RAM cost.

Note, extra care must now be taken to make sure these size and len fields
don't fall out of sync.
2018-10-09 23:02:09 -05:00
Christopher Haster
03b262b1e8 Separated out version of dir remove/append for non-entries
This allows updates to directories without needing to allocate an entry
struct for every call.
2018-10-09 23:02:09 -05:00
Christopher Haster
362b0bbe45 Minor improvement to from-memory commits
Tweaked the commit callback to pass the arguments for from-memory
commits explicitly, with non-from-memory commits still being able to
hijack the opaque data pointer for additional state.

The from-memory commits make up the vast majority of commits in
littlefs, so this small change has a noticable impact.
2018-10-09 23:02:09 -05:00
Christopher Haster
e4a0cd942d Take advantage of empty space early in dir search
Before, when appending new entries to a directory, we try to find empty space
in the last block of a directory chain. This has a nice side-effect that
the order of directory entries is maintained. However, this isn't strictly
necessary.

We're already scanning the directory chain in order, so other than changes to
directory order, there's no downside to taking advantage of any free
space we come across.
2018-10-09 23:02:09 -05:00
Christopher Haster
f30ab677a4 Traded enum-based DSL for full callback-based DSL
Now, instead of passing an enum for mem/disk commits, we pass a function
pointer that can specify any behaviour.

This has the benefit of opening up the possibility to pass any sort of
commit logic to the committers, and unused logic can be garbage-collected
by the compiler if unused. The downside is that unfortunately compilers have
a harder time optimizing around functions pointers than enums, and
fitting the state into structs for the callbacks may be costly.
2018-10-09 23:02:09 -05:00
Christopher Haster
ca3d6a52d2 Made implicity tag updates explicit
Before, tags were implicitly updated by the dir update functions, which
have a strong understanding of the entry struct. However, most of the
time the tag was already a part of the entry struct being committed.

By making tag updates explicit, this does add cost to commits that
now have to pass tag updates explicitly, but it reduces cost where that
tag and entry update can be combined into one commit region.

It also simplifies the dir update functions.
2018-10-09 23:02:09 -05:00
Christopher Haster
692f0c542e Naive implementation of resizable entries
Now, with the off, diff, and len parameters in each commit entry, we can build
up directory commits that resize entries. This adds complexity but opens
up the directory blocks to be much more flexible.

The main concern is that resizing entries can push around neighboring entries
in surprising ways, such as pushing them into new directory blocks when a
directory splits. This can break littlefs's internal logic in how it tracks
in-flight entries. The most problematic example being open files.

Fortunately, this is helped by a global linked-list of all files and
directories opened by the filesystem. As entries change size, the state
of open files/dirs may be updated as needed. Note this already needed to
exist for the ability to remove files/dirs, which has the same issue.
2018-10-09 23:02:09 -05:00
Christopher Haster
e3daee2621 Changed dir append to mirror commit DSL
Expiremental implementation. This opens up the opportunity to use the same
commit description for both commits and appends, which effectively do the same
thing.

This should lead to better code reuse.
2018-10-09 23:02:09 -05:00
Christopher Haster
73d29f05b2 Adopted a tiny LISP-like DSL for some extra flexibility
Really all this means is that the internal commit function was changed
from taking an array of "commit structures" to a linked-list of "commit
structures". The benefit of a linked-list is that layers of commit
functions can pull off some minor modifications to the description of
the commit. Most notably, commit functions can add additional entries
that will be atomically written out and CRCed along with the initial
commit.

Also a minor benefit, this is one less parameter when committing a
directory with zero entries.
2018-10-09 23:02:09 -05:00
Christopher Haster
4c35c8655a Added different sources for commits, now with disk->disk moves
Previously, commits could only come from memory in RAM. This meant any
entries had to be buffered in their entirety before they could be moved
to a different directory pair. By adding parameters for specifying
commits from existing entries stored on disk, we allow any sized entries
to be moved between directory pairs with a fixed RAM cost.
2018-10-09 23:02:09 -05:00
Christopher Haster
49698e431f Separated type/struct fields in dir entries
The separation of data-structure vs entry type has been implicit for a
while now, and even taken advantage of to simplify the traverse logic.

Explicitely separating the data-struct and entry types allows us to
introduce new data structures (inlined files).
2018-10-09 23:02:01 -05:00
Christopher Haster
0bb1f7af17 Modified release script to create notes only on minor releases
Before, release notes with a list of changes were created every
patch release. Unfortunately, it looks like this will create a lot of
noise on github, with a notification every patch release, which may be
as often as every time a PR is merged.

Rather than creating all of this noise for relatively uninteresting
changes, the script will now stick to simple tags, and create the
release notes only on minor releases.

I think this is what several of you were originally suggesting,
sorry about the journey, at least I learned a lot.
2018-09-29 12:31:27 -05:00
Christopher Haster
447d89cbd8 Merge pull request #109 from OTAkeys/pr/fix-sign-compare
Fix -Wsign-compare error
2018-09-29 12:29:54 -05:00
Vincent Dupont
28d2d96a83 Fix -Wsign-compare error 2018-09-29 11:33:19 -05:00
Christopher Haster
cb62bf2188 Fixed release script issue with fetching recent tags
Fetching all tags was triggering the pagination system inside the github
API. This prevent version tags from being found.

Modified to use the version tag prefix in the ref lookup, however this
still may cause an issue if there are still enough patch releases to trigger
pagination.

Simpleish solution is to grab the link header to jump to the last page,
since pagination results appear to be in sorted order.
2018-09-27 14:46:12 -05:00
Christopher Haster
646b1b5a6c Added -Wjump-misses-init and fixed uninitialized warnings 2018-09-26 18:58:54 -05:00
Christopher Haster
1b7a15599e Merge pull request #106 from conkerkh/master
If stats file doesn't exist lfs_emubd_create will fail.
2018-09-26 18:58:34 -05:00
Christopher Haster
e5a6938faf Fixed possible infinite loop in deorphan step
Normally, the linked-list of directory pairs should terminate at a null
pointer. However, it is possible if the filesystem is corrupted, that
that this linked-list forms a cycle.

This should never happen with littlefs's power resilience, but if it does
we should recover appropriately.

Modified lfs_deorphan to notice if we have a cycle and return
LFS_ERR_CORRUPT in that situation.

Found by kneko715
2018-09-26 18:58:11 -05:00
Chris
6ad544f3f3 If stats file doesn't exist lfs_emubd_create will fail.
This will create default stats file if it doesn't exist.
2018-09-26 18:24:58 -05:00
Christopher Haster
3419284689 Fixed issue with corruption due to different cache sizes
The lfs_cache_zero function that was recently added assumed a single cache
size, which is incorrect. This would cause a buffer overflow if
read_size != prog_size.

Since lfs_cache_zero is only used for scrubbing prog caches, the fix
here is to use lfs_cache_drop instead on read caches. Info in read
caches should never make its way to disk.

Found by nstcl
2018-09-04 13:57:22 -05:00
Christopher Haster
510cd13df9 Bumped minor version to v1.6 2018-07-27 15:59:18 -05:00
Christopher Haster
f5e0539951 Fixed issue with release script non-standard version tags 2018-07-27 15:20:00 -05:00
Christopher Haster
066448055c Moved SPDX and license info into README
This makes is a bit easier to find the description about the SPDX tags,
and fixes the issue where GitHub doesn't detect the license text.
2018-07-27 14:02:38 -05:00
Christopher Haster
d66723ccfd Merge pull request #81 from ARMmbed/simple-versioning
Simplified release process based on feedback
2018-07-27 14:02:23 -05:00
Christopher Haster
0234c77102 Simplified release process based on feedback
Previously, littlefs had mutable versions. That is, anytime a new commit
landed on master, the bot would update the most recent version to
contain the patch. The idea was that this would make sure users always
had the most recent bug fixes. Immutable snapshots could be accessed
through the git hashes.

However, at this point multiple developers have pointed out that this is
confusing, with mutable versions being non-standard and surprising.

This new release process adopts SemVer in its entirety, with
incrementing patch numbers and immutable versions.

When a new commit lands on master:
1. The major/minor version is taken from lfs.h
2. The most recent patch version is looked up on GitHub and incremented
3. A changelog is built out of the commits to the previous version
4. A new release is created on GitHub

Additionally, any commits that land while CI is still running are
coalesced together. Which means multiple PRs can land in a single
release.
2018-07-25 14:21:58 -05:00
Christopher Haster
84adead98b Merge pull request #80 from FreddieChopin/fix-memory-leaks
Fix memory leaks
2018-07-19 17:30:48 -05:00
Freddie Chopin
0422c55b81 Fix memory leaks in lfs_mount and lfs_format
Squashed:
- Change lfs_deinit() return to void to simplify error handling
- Move lfs_deinit() before lfs_init()
- Fix memory leaks in lfs_init()
- Fix memory leaks in lfs_format()
- Fix memory leaks in lfs_mount()
2018-07-19 16:54:38 -05:00
Christopher Haster
11ad3a2414 Merge pull request #76 from ARMmbed/fix-corrupt-read
Add handling for corrupt as initial state of blocks
2018-07-17 20:32:33 -05:00
Christopher Haster
16318d003f Merge pull request #58 from dpgeorge/file-open-no-malloc
Added possibility to open multiple files with LFS_NO_MALLOC enabled
2018-07-17 20:31:20 -05:00
Damien George
961fab70c3 Added file config structure and lfs_file_opencfg
The optional config structure options up the possibility of adding
file-level configuration in a backwards compatible manner.

Also adds possibility to open multiple files with LFS_NO_MALLOC
enabled thanks to dpgeorge

Also bumped minor version to v1.5
2018-07-17 18:32:18 -05:00
Christopher Haster
041e90a1ca Added handling for corrupt as initial state of blocks
Before this, littlefs incorrectly assumed corrupt blocks were only the result
of our own modification. This would be fine for most cases of freshly
erased storage, but for storage with block-level ECC this wasn't always
true.

Fortunately, it's quite easy for littlefs to handle this case correctly,
as long as corrupt storage always reports that it is corrupt, which for
most forms of ECC is the case unless we perform a write on the storage.

found by rojer
2018-07-16 15:33:52 -05:00
Christopher Haster
f94d233deb Merge pull request #74 from FreddieChopin/cxx-guards
Add C++ guards to public headers
2018-07-13 10:55:16 -05:00
Freddie Chopin
577d777c20 Add C++ guards to public headers
Fixes #53
Fixes #32
2018-07-13 09:34:49 +02:00
Christopher Haster
c72d25203c Merge pull request #73 from FreddieChopin/fix-format-specifiers
Use PRIu32 and PRIx32 format specifiers to fix warnings
2018-07-12 16:54:13 -05:00
Freddie Chopin
7e67f9324e Use PRIu32 and PRIx32 format specifiers to fix warnings
When using "%d" or "%x" with uint32_t types, arm-none-eabi-gcc reports
warnings like below:

-- >8 -- >8 -- >8 -- >8 -- >8 -- >8 --

In file included from lfs.c:8:
lfs_util.h:45:12: warning: format '%d' expects argument of type 'int', but argument 4 has type 'lfs_block_t' {aka 'long unsigned int'} [-Wformat=]
     printf("lfs debug:%d: " fmt "\n", __LINE__, __VA_ARGS__)
            ^~~~~~~~~~~~~~~~
lfs.c:2512:21: note: in expansion of macro 'LFS_DEBUG'
                     LFS_DEBUG("Found partial move %d %d",
                     ^~~~~~~~~
lfs.c:2512:55: note: format string is defined here
                     LFS_DEBUG("Found partial move %d %d",
                                                      ~^
                                                      %ld

-- >8 -- >8 -- >8 -- >8 -- >8 -- >8 --

Fix this by replacing "%d" and "%x" with `"%" PRIu32` and `"%" PRIx32`.
2018-07-11 12:32:21 +02:00
Christopher Haster
5a17fa42e4 Fixed script issue with bash expansion inside makefile parameter
This was causing code sizes to be reported with several of the logging
functions still built in. A useful number, but not the minimum
achievable code size.
2018-07-10 17:18:45 -05:00
Christopher Haster
eed1eec5fd Fixed information leaks through reused caches
As a shortcut, littlefs never bother to zero any of the buffers is used.
It didn't need to because it would always write out the entirety of the
data it needed.

Unfortunately, this, combined with the extra padding used to align
buffers to the nearest prog size, would lead to uninitialized data
getting written out to disk.

This means unrelated file data could be written to different parts of
storage, or worse, information leaked from the malloc calls could be
written out to disk unnecessarily.

found by rojer
2018-07-10 11:18:46 -05:00
Christopher Haster
4a86370327 Added quality of life improvements for main.c/test.c issues
1. Added check for main.c and test.c to decide compilation target
2. Added step to remove test.c after successful test completion

The test.c file, which contains the expanded test main, is useful when
debugging why tests are failing. However, keeping the test.c file around
causes problems when a later attempt is made to compile a larger project
containing the littlefs directory.

Under (hopefully) normal operation, tests always pass. So it should be ok
to remove the test.c file after a successful test run. Hopefully this
behaviour doesn't cause too much confusion for contributors using the
tests.

On the other side of things, compiling the library with no main ends
(successfully) with the "main not found" error message. By defaulting
to lfs.a if neither test.c/main.c is avoid this in the common cases

found by armijnhemel and Sim4n6
2018-07-10 11:17:50 -05:00
Christopher Haster
ba4f17173f Merge pull request #57 from dpgeorge/fix-warnings
Fix some compiler warnings: shadowed variable and unused parameters
2018-07-02 12:01:34 -05:00
Damien George
51346b8bf4 Fixed shadowed variable warnings
- Fixed shadowed variable warnings in lfs_dir_find.
- Fixed unused parameter warnings when LFS_NO_MALLOC is enabled.
- Added extra warning flags to CFLAGS.
- Updated tests so they don't shadow the "size" variable for -Wshadow
2018-07-02 10:29:19 -05:00
Christopher Haster
93a2e0bbe5 Merge pull request #62 from ARMmbed/license-bsd-3
v1.4 - Change license to BSD-3-Clause
2018-06-21 13:10:57 -05:00
Christopher Haster
6beff502e9 Changed license to BSD-3-Clause
For better compatibility with GPL v2

With permissions from:
- aldot
- Sim4n6
- jrast
2018-06-21 11:41:43 -05:00
Christopher Haster
c5e2b335d6 Added error when opening multiple files with a statically allocated buffer
Opening multiple files simultaneously is not supported without dynamic memory,
but the previous behaviour would just let the files overwrite each other, which
could lead to bad errors down the line

found by husigeza
2018-04-30 03:37:10 -05:00
Christopher Haster
015b86bc51 Fixed issue with trailing dots in file paths
Paths such as the following were causing issues:
/tea/hottea/.
/tea/hottea/..

Unfortunately the existing structure for path lookup didn't make it very
easy to introduce proper handling in this case without duplicating the
entire skip logic for paths. So the lfs_dir_find function had to be
restructured a bit.

One odd side-effect of this is that now lfs_dir_find includes the
initial fetch operation. This kinda breaks the fetch -> op pattern of
the dir functions, but does come with a nice code size reduction.
2018-04-22 07:26:31 -05:00
Christopher Haster
9637b96069 Fixed lookahead overflow and removed unbounded lookahead pointers
As pointed out by davidefer, the lookahead pointer modular arithmetic
does not work around integer overflow when the pointer size is not a
multiple of the block count.

To avoid overflow problems, the easy solution is to stop trying to
work around integer overflows and keep the lookahead offset inside the
block device. To make this work, the ack was modified into a resetable
counter that is decremented every block allocation.

As a plus, quite a bit of the allocation logic ended up simplified.
2018-04-11 14:38:25 -05:00
Christopher Haster
89a7630d84 Fixed issue with lookahead trusting old lookahead blocks
One of the big simplifications in littlefs's implementation is the
complete lack of tracking free blocks, allowing operations to simply
drop blocks that are no longer in use.

However, this means the lookahead buffer can easily contain outdated
blocks that were previously deleted. This is usually fine, as littlefs
will rescan the storage if it can't find a free block in the lookahead
buffer, but after changes that caused littlefs to more conservatively
respect the alloc acks (e611cf5), any scanned blocks after an ack would
be incorrectly trusted.

The fix is to eagerly scan ahead in the lookahead when we allocate so
that alloc acks are better able to discredit old lookahead blocks. Since
usually alloc acks are tightly coupled to allocations of one or two blocks,
this allows littlefs to properly rescan every set of allocations.

This may still be a concern if there is a long series of worn out
blocks, but in the worst case littlefs will conservatively avoid using
blocks it's not sure about.

Found by davidefer
2018-04-09 14:37:35 -05:00
Christopher Haster
43eac3083b Renamed test_parallel tests to test_interespersed
The name test_parallel gave off the incorrect impression that these
tests are multithreaded.
2018-04-08 17:31:09 -05:00
Christopher Haster
dbc3cb1798 Fixed Travis rate-limit issue with Github requests
Using credentials avoids rate-limiting based on Travis's IP address
2018-04-08 17:31:09 -05:00
Christopher Haster
93ece2e87a Removed outdated note about moves and powerloss 2018-04-08 17:31:05 -05:00
Christopher Haster
d9c076d909 Removed the uninitialized read for invalid superblocks 2018-03-19 00:39:40 -05:00
Christopher Haster
58f3bb1f08 Merge pull request #37 from jrast/patch-1
Added a note about the callback functions
2018-03-13 00:13:44 -05:00
Christopher Haster
f72f6d6a05 Removed out of date note about endianness 2018-03-12 21:27:39 -05:00
Juerg Rast
5c4ee2109d Added a note about the callback functions
Added a short section about the callback functions, based on the answers
to issue #35 and #36
2018-03-12 21:26:40 -05:00
Christopher Haster
155224600a Fixed Travis issue with deploy stage in PRs 2018-03-12 19:57:57 -05:00
Christopher Haster
9ee112a7cb Fixed issue updating dir struct when extended dir chain
Like most of the lfs_dir_t functions, lfs_dir_append is responsible for
updating the lfs_dir_t struct if the underlying directory block is
moved. This property makes handling worn out blocks much easier by
removing the amount of state that needs to be considered during a
directory update.

However, extending the dir chain is a bit of a corner case. It's not
changing the old block, but callers of lfs_dir_append do assume the
"entry" will reside in "dir" after lfs_dir_append completes.

This issue only occurs when creating files, since mkdir does not use
the entry after lfs_dir_append. Unfortunately, the tests against
extending the directory chain were all made using mkdir.

Found by schouleu
2018-02-28 23:14:41 -06:00
Christopher Haster
d9c36371e7 Fixed handling of root as target for create operations
Before this patch, when calling lfs_mkdir or lfs_file_open with root
as the target, littlefs wouldn't find the path properly and happily
run into undefined behaviour.

The fix is to populate a directory entry for root in the lfs_dir_find
function. As an added plus, this allowed several special cases around
root to be completely dropped.
2018-02-28 23:13:02 -06:00
Christopher Haster
1476181bd1 Added LFS_CONFIG for user provided configuration of the utils
Suggested by sn00pster, LFS_CONFIG is an opt-in user provided
configuration file that will override the util implementation in
lfs_util.h. This is useful for allowing system-specific overrides
without needing to rely on git merges or other forms of patching
for updates.
2018-02-22 13:39:24 -06:00
Christopher Haster
b2124a5ae5 Fixed multiple deploy steps in Travis 2018-02-20 17:55:30 -06:00
Christopher Haster
949015ad52 Merge pull request #28 from geky/configurables
Add better general support in lfs_utils.h
2018-02-19 17:29:48 -06:00
Christopher Haster
67daf9e2c5 Added cross-compile targets for testing
Using gcc cross compilers and qemu:
- make test CC="arm-linux-gnueabi-gcc --static -mthumb" EXEC="qemu-arm"
- make test CC="powerpc-linux-gnu-gcc --static" EXEC="qemu-ppc"
- make test CC="mips-linux-gnu-gcc --static" EXEC="qemu-mips"

Also separated out Travis jobs and added some size reporting
2018-02-19 01:40:28 -06:00
Christopher Haster
a3fd2d4d6d Added more configurable utils
Note: It's still expected to modify lfs_utils.h when porting littlefs
to a new target/system. There's just too much room for system-specific
improvements, such as taking advantage of CRC hardware.

Rather, encouraging modification of lfs_util.h and making it easy to
modify and debug should result in better integration with the consuming
systems.

This just adds a bunch of quality-of-life improvements that should help
development and integration in littlefs.

- Macros that require no side-effects are all-caps
- System includes are only brought in when needed
- Malloc/free wrappers
- LFS_NO_* checks for quickly disabling things at the command line
- At least a little-bit more docs
2018-02-19 01:40:23 -06:00
Christopher Haster
a0a55fb9e5 Added conversion to/from little-endian on disk
Required to support big-endian processors, with the most notable being
the PowerPC architecture.

On little-endian architectures, these conversions can be optimized out
and have no code impact.

Initial patch provided by gmouchard
2018-02-19 01:39:08 -06:00
Christopher Haster
4f08424b51 Added software implementations of bitwise instructions
This helps significantly with supporting different compilers. Intrinsics for
different compilers can be added as they are found.

Note that for ARMCC, __builtin_ctz is not used. This was the result of a
strange issue where ARMCC only emits __builtin_ctz when passed the
--gnu flag, but __builtin_clz and __builtin_popcount are always emitted.
This isn't a big problem since the ARM instruction set doesn't have a
ctz instruction, and the npw2 based implementation is one of the most
efficient.

Also note that for littefs's purposes, we consider ctz(0) to be
undefined. This lets us save a branch in the software lfs_ctz
implementation.
2018-02-19 01:39:04 -06:00
Christopher Haster
59ce49fa4b Merge pull request #26 from Sim4n6/master
Added a .git ignore file
2018-02-08 23:28:55 -06:00
iamatacos
2f8ae344d2 Added a git ignore file with .o .d blocks dir and lfs bin 2018-02-08 02:20:51 -06:00
Christopher Haster
e611cf5050 Fix incorrect lookahead population before ack
Rather than tracking all in-flight blocks blocks during a lookahead,
littlefs uses an ack scheme to mark the first allocated block that
hasn't reached the disk yet. littlefs assumes all blocks since the
last ack are bad or in-flight, and uses this to know when it's out
of storage.

However, these unacked allocations were still being populated in the
lookahead buffer. If the whole block device fits in the lookahead
buffer, _and_ littlefs managed to scan around the whole storage while
an unacked block was still in-flight, it would assume the block was
free and misallocate it.

The fix is to only fill the lookahead buffer up to the last ack.
The internal free structure was restructured to simplify the runtime
calculation of lookahead size.
2018-02-08 01:52:39 -06:00
Christopher Haster
a25743a82a Fixed some minor error code differences
- Write on read-only file to return LFS_ERR_BADF
- Renaming directory onto file to return LFS_ERR_NOTEMPTY
- Changed LFS_ERR_INVAL in lfs_file_seek to assert
2018-02-04 14:36:36 -06:00
Christopher Haster
6716b5580a Fixed error check when truncating files to larger size 2018-02-04 14:09:55 -06:00
Christopher Haster
809ffde60f Merge pull request #24 from aldot/silence-shadow-warnings-1
Silence shadow warnings
2018-02-04 13:36:55 -06:00
Christopher Haster
dc513b172f Silenced more of aldot's warnings
Flags used:
-Wall -Wextra -Wshadow -Wwrite-strings -Wundef -Wstrict-prototypes
-Wunused -Wunused-parameter -Wunused-function -Wunused-value
-Wmissing-prototypes -Wmissing-declarations -Wold-style-definition
2018-02-04 13:15:30 -06:00
Bernhard Reutner-Fischer
aa50e03684 Commentary typo fix 2018-02-04 13:15:26 -06:00
Bernhard Reutner-Fischer
6d55755128 tests: Silence warnings in template
- no previous prototype for ‘test_assert’
- no previous prototype for ‘test_count’
- unused parameter ‘b’ in test_count
- function declaration isn’t a prototype for main
2018-02-04 13:15:17 -06:00
Bernhard Reutner-Fischer
029361ea16 Silence shadow warnings 2018-02-04 13:15:09 -06:00
Christopher Haster
fd04ed4f25 Added autogenerated release notes from commits 2018-02-02 02:35:07 -06:00
Bernhard Reutner-Fischer
3101bc92b3 Do not print command invocation if QUIET 2018-02-02 09:34:01 +01:00
Christopher Haster
d82e34c3ee Merge pull request #21 from aldot/doc-tweaks
documentation touch up, take 2
2018-02-01 15:06:24 -06:00
Bernhard Reutner-Fischer
436707c8d0 doc: Editorial tweaks 2018-02-01 14:56:43 -06:00
Bernhard Reutner-Fischer
3457252fe6 doc: Spelling fixes 2018-01-31 19:18:51 -06:00
Christopher Haster
6d8e0e21d0 Moved -Werror flag to CI only
The most useful part of -Werror is preventing code from being
merged that has warnings. However it is annoying for users who may have
different compilers with different warnings. Limiting -Werror to CI only
covers the main concern about warnings without limiting users.
2018-01-29 18:37:48 -06:00
Christopher Haster
88f678f4c6 Fixed self-assign warning in tests
Some of the tests were creating a variable `res`, however the test
system itself relies on it's own `res` variable. This worked out by
luck, but could lead to problems if the res variables were different
types.

Changed the generated variable in the test system to the less common
name `test`, which also works out to share the same prefix as other test
functions.
2018-01-29 18:37:48 -06:00
Christopher Haster
3ef4847434 Added remove step in tests to force rebuild
Found by user iamscottmoyers, this was an interesting bug with the test
system. If the new test.c file is generated fast enough, it may not have
a new timestamp and not get recompiled.

To fix, we can remove the specific files that need to be rebuilt (lfs and
test.o).
2018-01-29 18:37:41 -06:00
Christopher Haster
f694b14afb Merge pull request #16 from geky/versioning
Add version info for software library and on-disk structures
2018-01-29 01:20:23 -06:00
Christopher Haster
5a38d00dde Added deploy step in Travis to push new version as tags 2018-01-29 00:51:43 -06:00
Christopher Haster
035552a858 Add version info for software library and on-disk structures
An annoying part of filesystems is that the software library can change
independently of the on-disk structures. For this reason versioning is
very important, and must be handled separately for the software and
on-disk parts.

In this patch, littlefs provides two version numbers at compile time,
with major and minor parts, in the form of 6 macros.

LFS_VERSION        // Library version, uint32_t encoded
LFS_VERSION_MAJOR  // Major - Backwards incompatible changes
LFS_VERSION_MINOR  // Minor - Feature additions

LFS_DISK_VERSION        // On-disk version, uint32_t encoded
LFS_DISK_VERSION_MAJOR  // Major - Backwards incompatible changes
LFS_DISK_VERSION_MINOR  // Minor - Feature additions

Note that littlefs will error if it finds a major version number that
is different, or a minor version number that has regressed.
2018-01-26 14:26:25 -06:00
Christopher Haster
997c2e594e Fixed incorrect reliance on errno in emubd
When running the tests, the emubd erase function relied on the value of
errno to not change over a possible call to unlink. Annoyingly, I've
only seen this cause problems on a couple of specific Travis instances
while self-hosting littlefs on top of littlefs-fuse.
2018-01-22 19:28:29 -06:00
36 changed files with 10463 additions and 4475 deletions

9
.gitignore vendored Normal file
View File

@@ -0,0 +1,9 @@
# Compilation output
*.o
*.d
*.a
# Testing things
blocks/
lfs
test.c

View File

@@ -1,47 +1,317 @@
# Environment variables
env:
global:
- CFLAGS=-Werror
# Common test script
script:
# make sure example can at least compile
- sed -n '/``` c/,/```/{/```/d; p;}' README.md > test.c &&
CFLAGS='
# make sure example can at least compile
- sed -n '/``` c/,/```/{/```/d; p;}' README.md > test.c &&
make all CFLAGS+="
-Duser_provided_block_device_read=NULL
-Duser_provided_block_device_prog=NULL
-Duser_provided_block_device_erase=NULL
-Duser_provided_block_device_sync=NULL
-include stdio.h -Werror' make all size
-include stdio.h"
# run tests
- make test QUIET=1
# run tests
- make test QUIET=1
# run tests with a few different configurations
- CFLAGS="-DLFS_READ_SIZE=1 -DLFS_PROG_SIZE=1" make test QUIET=1
- CFLAGS="-DLFS_READ_SIZE=512 -DLFS_PROG_SIZE=512" make test QUIET=1
- CFLAGS="-DLFS_BLOCK_COUNT=1023 -DLFS_LOOKAHEAD=2048" make test QUIET=1
# run tests with a few different configurations
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=1 -DLFS_CACHE_SIZE=4"
- make test QUIET=1 CFLAGS+="-DLFS_READ_SIZE=512 -DLFS_CACHE_SIZE=512 -DLFS_BLOCK_CYCLES=16"
- make test QUIET=1 CFLAGS+="-DLFS_BLOCK_COUNT=1023 -DLFS_LOOKAHEAD_SIZE=256"
- make clean test QUIET=1 CFLAGS+="-DLFS_INLINE_MAX=0"
- make clean test QUIET=1 CFLAGS+="-DLFS_EMUBD_ERASE_VALUE=0xff"
- make clean test QUIET=1 CFLAGS+="-DLFS_NO_INTRINSICS"
# additional configurations that don't support all tests (this should be
# fixed but at the moment it is what it is)
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=1 -DLFS_BLOCK_SIZE=4096"
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=\(2*1024\) -DLFS_BLOCK_SIZE=\(64*1024\)"
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=\(8*1024\) -DLFS_BLOCK_SIZE=\(64*1024\)"
- make test_files QUIET=1
CFLAGS+="-DLFS_READ_SIZE=11 -DLFS_BLOCK_SIZE=704"
# compile and find the code size with the smallest configuration
- make clean size
OBJ="$(ls lfs*.o | tr '\n' ' ')"
CFLAGS+="-DLFS_NO_ASSERT -DLFS_NO_DEBUG -DLFS_NO_WARN -DLFS_NO_ERROR"
| tee sizes
# update status if we succeeded, compare with master if possible
- |
if [ "$TRAVIS_TEST_RESULT" -eq 0 ]
then
CURR=$(tail -n1 sizes | awk '{print $1}')
PREV=$(curl -u "$GEKY_BOT_STATUSES" https://api.github.com/repos/$TRAVIS_REPO_SLUG/status/master \
| jq -re "select(.sha != \"$TRAVIS_COMMIT\")
| .statuses[] | select(.context == \"$STAGE/$NAME\").description
| capture(\"code size is (?<size>[0-9]+)\").size" \
|| echo 0)
STATUS="Passed, code size is ${CURR}B"
if [ "$PREV" -ne 0 ]
then
STATUS="$STATUS ($(python -c "print '%+.2f' % (100*($CURR-$PREV)/$PREV.0)")%)"
fi
fi
# CI matrix
jobs:
include:
# native testing
- stage: test
env:
- STAGE=test
- NAME=littlefs-x86
# cross-compile with ARM (thumb mode)
- stage: test
env:
- STAGE=test
- NAME=littlefs-arm
- CC="arm-linux-gnueabi-gcc --static -mthumb"
- EXEC="qemu-arm"
install:
- sudo apt-get install
gcc-arm-linux-gnueabi
libc6-dev-armel-cross
qemu-user
- arm-linux-gnueabi-gcc --version
- qemu-arm -version
# cross-compile with PowerPC
- stage: test
env:
- STAGE=test
- NAME=littlefs-powerpc
- CC="powerpc-linux-gnu-gcc --static"
- EXEC="qemu-ppc"
install:
- sudo apt-get install
gcc-powerpc-linux-gnu
libc6-dev-powerpc-cross
qemu-user
- powerpc-linux-gnu-gcc --version
- qemu-ppc -version
# cross-compile with MIPS
- stage: test
env:
- STAGE=test
- NAME=littlefs-mips
- CC="mips-linux-gnu-gcc --static"
- EXEC="qemu-mips"
install:
- sudo apt-get install
gcc-mips-linux-gnu
libc6-dev-mips-cross
qemu-user
- mips-linux-gnu-gcc --version
- qemu-mips -version
# self-host with littlefs-fuse for fuzz test
- make -C littlefs-fuse
- stage: test
env:
- STAGE=test
- NAME=littlefs-fuse
if: branch !~ -prefix$
install:
- sudo apt-get install libfuse-dev
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v2
- fusermount -V
- gcc --version
before_script:
# setup disk for littlefs-fuse
- rm -rf littlefs-fuse/littlefs/*
- cp -r $(git ls-tree --name-only HEAD) littlefs-fuse/littlefs
- littlefs-fuse/lfs --format /dev/loop0
- littlefs-fuse/lfs /dev/loop0 mount
- mkdir mount
- sudo chmod a+rw /dev/loop0
- dd if=/dev/zero bs=512 count=4096 of=disk
- losetup /dev/loop0 disk
script:
# self-host test
- make -C littlefs-fuse
- ls mount
- mkdir mount/littlefs
- cp -r $(git ls-tree --name-only HEAD) mount/littlefs
- cd mount/littlefs
- ls
- make -B test_dirs test_files QUIET=1
- littlefs-fuse/lfs --format /dev/loop0
- littlefs-fuse/lfs /dev/loop0 mount
- ls mount
- mkdir mount/littlefs
- cp -r $(git ls-tree --name-only HEAD) mount/littlefs
- cd mount/littlefs
- stat .
- ls -flh
- make -B test_dirs test_files QUIET=1
# self-host with littlefs-fuse for fuzz test
- stage: test
env:
- STAGE=test
- NAME=littlefs-migration
if: branch !~ -prefix$
install:
- sudo apt-get install libfuse-dev
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v2 v2
- git clone --depth 1 https://github.com/geky/littlefs-fuse -b v1 v1
- fusermount -V
- gcc --version
before_script:
# setup disk for littlefs-fuse
- rm -rf v2/littlefs/*
- cp -r $(git ls-tree --name-only HEAD) v2/littlefs
- mkdir mount
- sudo chmod a+rw /dev/loop0
- dd if=/dev/zero bs=512 count=4096 of=disk
- losetup /dev/loop0 disk
script:
# compile v1 and v2
- make -C v1
- make -C v2
# run self-host test with v1
- v1/lfs --format /dev/loop0
- v1/lfs /dev/loop0 mount
- ls mount
- mkdir mount/littlefs
- cp -r $(git ls-tree --name-only HEAD) mount/littlefs
- cd mount/littlefs
- stat .
- ls -flh
- make -B test_dirs test_files QUIET=1
# attempt to migrate
- cd ../..
- fusermount -u mount
- v2/lfs --migrate /dev/loop0
- v2/lfs /dev/loop0 mount
# run self-host test with v2 right where we left off
- ls mount
- cd mount/littlefs
- stat .
- ls -flh
- make -B test_dirs test_files QUIET=1
# Automatically create releases
- stage: deploy
env:
- STAGE=deploy
- NAME=deploy
script:
- |
bash << 'SCRIPT'
set -ev
# Find version defined in lfs.h
LFS_VERSION=$(grep -ox '#define LFS_VERSION .*' lfs.h | cut -d ' ' -f3)
LFS_VERSION_MAJOR=$((0xffff & ($LFS_VERSION >> 16)))
LFS_VERSION_MINOR=$((0xffff & ($LFS_VERSION >> 0)))
# Grab latests patch from repo tags, default to 0, needs finagling
# to get past github's pagination api
PREV_URL=https://api.github.com/repos/$TRAVIS_REPO_SLUG/git/refs/tags/v$LFS_VERSION_MAJOR.$LFS_VERSION_MINOR.
PREV_URL=$(curl -u "$GEKY_BOT_RELEASES" "$PREV_URL" -I \
| sed -n '/^Link/{s/.*<\(.*\)>; rel="last"/\1/;p;q0};$q1' \
|| echo $PREV_URL)
LFS_VERSION_PATCH=$(curl -u "$GEKY_BOT_RELEASES" "$PREV_URL" \
| jq 'map(.ref | match("\\bv.*\\..*\\.(.*)$";"g")
.captures[].string | tonumber) | max + 1' \
|| echo 0)
# We have our new version
LFS_VERSION="v$LFS_VERSION_MAJOR.$LFS_VERSION_MINOR.$LFS_VERSION_PATCH"
echo "VERSION $LFS_VERSION"
# Check that we're the most recent commit
CURRENT_COMMIT=$(curl -f -u "$GEKY_BOT_RELEASES" \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/commits/master \
| jq -re '.sha')
[ "$TRAVIS_COMMIT" == "$CURRENT_COMMIT" ] || exit 0
# Create major branch
git branch v$LFS_VERSION_MAJOR HEAD
# Create major prefix branch
git config user.name "geky bot"
git config user.email "bot@geky.net"
git fetch https://github.com/$TRAVIS_REPO_SLUG.git \
--depth=50 v$LFS_VERSION_MAJOR-prefix || true
./scripts/prefix.py lfs$LFS_VERSION_MAJOR
git branch v$LFS_VERSION_MAJOR-prefix $( \
git commit-tree $(git write-tree) \
$(git rev-parse --verify -q FETCH_HEAD | sed -e 's/^/-p /') \
-p HEAD \
-m "Generated v$LFS_VERSION_MAJOR prefixes")
git reset --hard
# Update major version branches (vN and vN-prefix)
git push --atomic https://$GEKY_BOT_RELEASES@github.com/$TRAVIS_REPO_SLUG.git \
v$LFS_VERSION_MAJOR \
v$LFS_VERSION_MAJOR-prefix
# Build release notes
PREV=$(git tag --sort=-v:refname -l "v*" | head -1)
if [ ! -z "$PREV" ]
then
echo "PREV $PREV"
CHANGES=$(git log --oneline $PREV.. --grep='^Merge' --invert-grep)
printf "CHANGES\n%s\n\n" "$CHANGES"
fi
case ${GEKY_BOT_DRAFT:-minor} in
true) DRAFT=true ;;
minor) DRAFT=$(jq -R 'endswith(".0")' <<< "$LFS_VERSION") ;;
false) DRAFT=false ;;
esac
# Create the release and patch version tag (vN.N.N)
curl -f -u "$GEKY_BOT_RELEASES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/releases \
-d "{
\"tag_name\": \"$LFS_VERSION\",
\"name\": \"${LFS_VERSION%.0}\",
\"target_commitish\": \"$TRAVIS_COMMIT\",
\"draft\": $DRAFT,
\"body\": $(jq -sR '.' <<< "$CHANGES")
}" #"
SCRIPT
# Manage statuses
before_install:
- fusermount -V
- gcc --version
- |
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"$STAGE/$NAME\",
\"state\": \"pending\",
\"description\": \"${STATUS:-In progress}\",
\"target_url\": \"https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$TRAVIS_JOB_ID\"
}"
install:
- sudo apt-get install libfuse-dev
- git clone --depth 1 https://github.com/geky/littlefs-fuse
after_failure:
- |
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"$STAGE/$NAME\",
\"state\": \"failure\",
\"description\": \"${STATUS:-Failed}\",
\"target_url\": \"https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$TRAVIS_JOB_ID\"
}"
before_script:
- rm -rf littlefs-fuse/littlefs/*
- cp -r $(git ls-tree --name-only HEAD) littlefs-fuse/littlefs
after_success:
- |
curl -u "$GEKY_BOT_STATUSES" -X POST \
https://api.github.com/repos/$TRAVIS_REPO_SLUG/statuses/${TRAVIS_PULL_REQUEST_SHA:-$TRAVIS_COMMIT} \
-d "{
\"context\": \"$STAGE/$NAME\",
\"state\": \"success\",
\"description\": \"${STATUS:-Passed}\",
\"target_url\": \"https://travis-ci.org/$TRAVIS_REPO_SLUG/jobs/$TRAVIS_JOB_ID\"
}"
- mkdir mount
- sudo chmod a+rw /dev/loop0
- dd if=/dev/zero bs=512 count=2048 of=disk
- losetup /dev/loop0 disk
# Job control
stages:
- name: test
- name: deploy
if: branch = master AND type = push

2859
DESIGN.md

File diff suppressed because it is too large Load Diff

View File

@@ -1,165 +1,24 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Copyright (c) 2017, Arm Limited. All rights reserved.
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
1. Definitions.
- Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this
list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.
- Neither the name of ARM nor the names of its contributors may be used to
endorse or promote products derived from this software without specific prior
written permission.
"License" shall mean the terms and conditions for use, reproduction, and
distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by the copyright
owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all other entities
that control, are controlled by, or are under common control with that entity.
For the purposes of this definition, "control" means (i) the power, direct or
indirect, to cause the direction or management of such entity, whether by
contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity exercising
permissions granted by this License.
"Source" form shall mean the preferred form for making modifications, including
but not limited to software source code, documentation source, and configuration
files.
"Object" form shall mean any form resulting from mechanical transformation or
translation of a Source form, including but not limited to compiled object code,
generated documentation, and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or Object form, made
available under the License, as indicated by a copyright notice that is included
in or attached to the work (an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object form, that
is based on (or derived from) the Work and for which the editorial revisions,
annotations, elaborations, or other modifications represent, as a whole, an
original work of authorship. For the purposes of this License, Derivative Works
shall not include works that remain separable from, or merely link (or bind by
name) to the interfaces of, the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including the original version
of the Work and any modifications or additions to that Work or Derivative Works
thereof, that is intentionally submitted to Licensor for inclusion in the Work
by the copyright owner or by an individual or Legal Entity authorized to submit
on behalf of the copyright owner. For the purposes of this definition,
"submitted" means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems, and
issue tracking systems that are managed by, or on behalf of, the Licensor for
the purpose of discussing and improving the Work, but excluding communication
that is conspicuously marked or otherwise designated in writing by the copyright
owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity on behalf
of whom a Contribution has been received by Licensor and subsequently
incorporated within the Work.
2. Grant of Copyright License.
Subject to the terms and conditions of this License, each Contributor hereby
grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
irrevocable copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the Work and such
Derivative Works in Source or Object form.
3. Grant of Patent License.
Subject to the terms and conditions of this License, each Contributor hereby
grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
irrevocable (except as stated in this section) patent license to make, have
made, use, offer to sell, sell, import, and otherwise transfer the Work, where
such license applies only to those patent claims licensable by such Contributor
that are necessarily infringed by their Contribution(s) alone or by combination
of their Contribution(s) with the Work to which such Contribution(s) was
submitted. If You institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work or a
Contribution incorporated within the Work constitutes direct or contributory
patent infringement, then any patent licenses granted to You under this License
for that Work shall terminate as of the date such litigation is filed.
4. Redistribution.
You may reproduce and distribute copies of the Work or Derivative Works thereof
in any medium, with or without modifications, and in Source or Object form,
provided that You meet the following conditions:
You must give any other recipients of the Work or Derivative Works a copy of
this License; and
You must cause any modified files to carry prominent notices stating that You
changed the files; and
You must retain, in the Source form of any Derivative Works that You distribute,
all copyright, patent, trademark, and attribution notices from the Source form
of the Work, excluding those notices that do not pertain to any part of the
Derivative Works; and
If the Work includes a "NOTICE" text file as part of its distribution, then any
Derivative Works that You distribute must include a readable copy of the
attribution notices contained within such NOTICE file, excluding those notices
that do not pertain to any part of the Derivative Works, in at least one of the
following places: within a NOTICE text file distributed as part of the
Derivative Works; within the Source form or documentation, if provided along
with the Derivative Works; or, within a display generated by the Derivative
Works, if and wherever such third-party notices normally appear. The contents of
the NOTICE file are for informational purposes only and do not modify the
License. You may add Your own attribution notices within Derivative Works that
You distribute, alongside or as an addendum to the NOTICE text from the Work,
provided that such additional attribution notices cannot be construed as
modifying the License.
You may add Your own copyright statement to Your modifications and may provide
additional or different license terms and conditions for use, reproduction, or
distribution of Your modifications, or for any such Derivative Works as a whole,
provided Your use, reproduction, and distribution of the Work otherwise complies
with the conditions stated in this License.
5. Submission of Contributions.
Unless You explicitly state otherwise, any Contribution intentionally submitted
for inclusion in the Work by You to the Licensor shall be under the terms and
conditions of this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify the terms of
any separate license agreement you may have executed with Licensor regarding
such Contributions.
6. Trademarks.
This License does not grant permission to use the trade names, trademarks,
service marks, or product names of the Licensor, except as required for
reasonable and customary use in describing the origin of the Work and
reproducing the content of the NOTICE file.
7. Disclaimer of Warranty.
Unless required by applicable law or agreed to in writing, Licensor provides the
Work (and each Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
including, without limitation, any warranties or conditions of TITLE,
NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
solely responsible for determining the appropriateness of using or
redistributing the Work and assume any risks associated with Your exercise of
permissions under this License.
8. Limitation of Liability.
In no event and under no legal theory, whether in tort (including negligence),
contract, or otherwise, unless required by applicable law (such as deliberate
and grossly negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special, incidental,
or consequential damages of any character arising as a result of this License or
out of the use or inability to use the Work (including but not limited to
damages for loss of goodwill, work stoppage, computer failure or malfunction, or
any and all other commercial damages or losses), even if such Contributor has
been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability.
While redistributing the Work or Derivative Works thereof, You may choose to
offer, and charge a fee for, acceptance of support, warranty, indemnity, or
other liability obligations and/or rights consistent with this License. However,
in accepting such obligations, You may act only on Your own behalf and on Your
sole responsibility, not on behalf of any other Contributor, and only if You
agree to indemnify, defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason of your
accepting any such warranty or additional liability.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

View File

@@ -1,8 +1,11 @@
TARGET = lfs
TARGET = lfs.a
ifneq ($(wildcard test.c main.c),)
override TARGET = lfs
endif
CC = gcc
AR = ar
SIZE = size
CC ?= gcc
AR ?= ar
SIZE ?= size
SRC += $(wildcard *.c emubd/*.c)
OBJ := $(SRC:.c=.o)
@@ -14,15 +17,21 @@ TEST := $(patsubst tests/%.sh,%,$(wildcard tests/test_*))
SHELL = /bin/bash -o pipefail
ifdef DEBUG
CFLAGS += -O0 -g3
override CFLAGS += -O0 -g3
else
CFLAGS += -Os
override CFLAGS += -Os
endif
ifdef WORD
CFLAGS += -m$(WORD)
override CFLAGS += -m$(WORD)
endif
CFLAGS += -I.
CFLAGS += -std=c99 -Wall -pedantic
ifdef TRACE
override CFLAGS += -DLFS_YES_TRACE
endif
override CFLAGS += -I.
override CFLAGS += -std=c99 -Wall -pedantic
override CFLAGS += -Wextra -Wshadow -Wjump-misses-init -Wundef
# Remove missing-field-initializers because of GCC bug
override CFLAGS += -Wno-missing-field-initializers
all: $(TARGET)
@@ -33,18 +42,32 @@ size: $(OBJ)
$(SIZE) -t $^
.SUFFIXES:
test: test_format test_dirs test_files test_seek test_truncate test_parallel \
test_alloc test_paths test_orphan test_move test_corrupt
test: \
test_format \
test_dirs \
test_files \
test_seek \
test_truncate \
test_entries \
test_interspersed \
test_alloc \
test_paths \
test_attrs \
test_move \
test_orphan \
test_corrupt
@rm test.c
test_%: tests/test_%.sh
ifdef QUIET
./$< | sed -n '/^[-=]/p'
@./$< | sed -nu '/^[-=]/p'
else
./$<
endif
-include $(DEP)
$(TARGET): $(OBJ)
lfs: $(OBJ)
$(CC) $(CFLAGS) $^ $(LFLAGS) -o $@
%.a: $(OBJ)

190
README.md
View File

@@ -1,6 +1,6 @@
## The little filesystem
## littlefs
A little fail-safe filesystem designed for embedded systems.
A little fail-safe filesystem designed for microcontrollers.
```
| | | .---._____
@@ -11,17 +11,19 @@ A little fail-safe filesystem designed for embedded systems.
| | |
```
**Bounded RAM/ROM** - The littlefs is designed to work with a limited amount
of memory. Recursion is avoided and dynamic memory is limited to configurable
buffers that can be provided statically.
**Power-loss resilience** - littlefs is designed to handle random power
failures. All file operations have strong copy-on-write guarantees and if
power is lost the filesystem will fall back to the last known good state.
**Power-loss resilient** - The littlefs is designed for systems that may have
random power failures. The littlefs has strong copy-on-write guaruntees and
storage on disk is always kept in a valid state.
**Dynamic wear leveling** - littlefs is designed with flash in mind, and
provides wear leveling over dynamic blocks. Additionally, littlefs can
detect bad blocks and work around them.
**Wear leveling** - Since the most common form of embedded storage is erodible
flash memories, littlefs provides a form of dynamic wear leveling for systems
that can not fit a full flash translation layer.
**Bounded RAM/ROM** - littlefs is designed to work with a small amount of
memory. RAM usage is strictly bounded, which means RAM consumption does not
change as the filesystem grows. The filesystem contains no unbounded
recursion and dynamic memory is limited to configurable buffers that can be
provided statically.
## Example
@@ -49,7 +51,9 @@ const struct lfs_config cfg = {
.prog_size = 16,
.block_size = 4096,
.block_count = 128,
.lookahead = 128,
.cache_size = 16,
.lookahead_size = 16,
.block_cycles = 500,
};
// entry point
@@ -88,70 +92,158 @@ int main(void) {
## Usage
Detailed documentation (or at least as much detail as is currently available)
can be cound in the comments in [lfs.h](lfs.h).
can be found in the comments in [lfs.h](lfs.h).
As you may have noticed, littlefs takes in a configuration structure that
defines how the filesystem operates. The configuration struct provides the
filesystem with the block device operations and dimensions, tweakable
parameters that tradeoff memory usage for performance, and optional
static buffers if the user wants to avoid dynamic memory.
littlefs takes in a configuration structure that defines how the filesystem
operates. The configuration struct provides the filesystem with the block
device operations and dimensions, tweakable parameters that tradeoff memory
usage for performance, and optional static buffers if the user wants to avoid
dynamic memory.
The state of the littlefs is stored in the `lfs_t` type which is left up
to the user to allocate, allowing multiple filesystems to be in use
simultaneously. With the `lfs_t` and configuration struct, a user can
format a block device or mount the filesystem.
Once mounted, the littlefs provides a full set of posix-like file and
Once mounted, the littlefs provides a full set of POSIX-like file and
directory functions, with the deviation that the allocation of filesystem
structures must be provided by the user.
All posix operations, such as remove and rename, are atomic, even in event
of power-loss. Additionally, no file updates are actually commited to the
filesystem until sync or close is called on the file.
All POSIX operations, such as remove and rename, are atomic, even in event
of power-loss. Additionally, no file updates are not actually committed to
the filesystem until sync or close is called on the file.
## Other notes
All littlefs have the potential to return a negative error code. The errors
can be either one of those found in the `enum lfs_error` in [lfs.h](lfs.h),
or an error returned by the user's block device operations.
All littlefs calls have the potential to return a negative error code. The
errors can be either one of those found in the `enum lfs_error` in
[lfs.h](lfs.h), or an error returned by the user's block device operations.
It should also be noted that the current implementation of littlefs doesn't
really do anything to insure that the data written to disk is machine portable.
This is fine as long as all of the involved machines share endianness
(little-endian) and don't have strange padding requirements.
In the configuration struct, the `prog` and `erase` function provided by the
user may return a `LFS_ERR_CORRUPT` error if the implementation already can
detect corrupt blocks. However, the wear leveling does not depend on the return
code of these functions, instead all data is read back and checked for
integrity.
## Reference material
If your storage caches writes, make sure that the provided `sync` function
flushes all the data to memory and ensures that the next read fetches the data
from memory, otherwise data integrity can not be guaranteed. If the `write`
function does not perform caching, and therefore each `read` or `write` call
hits the memory, the `sync` function can simply return 0.
[DESIGN.md](DESIGN.md) - DESIGN.md contains a fully detailed dive into how
littlefs actually works. I would encourage you to read it since the
solutions and tradeoffs at work here are quite interesting.
## Design
[SPEC.md](SPEC.md) - SPEC.md contains the on-disk specification of littlefs
with all the nitty-gritty details. Can be useful for developing tooling.
At a high level, littlefs is a block based filesystem that uses small logs to
store metadata and larger copy-on-write (COW) structures to store file data.
In littlefs, these ingredients form a sort of two-layered cake, with the small
logs (called metadata pairs) providing fast updates to metadata anywhere on
storage, while the COW structures store file data compactly and without any
wear amplification cost.
Both of these data structures are built out of blocks, which are fed by a
common block allocator. By limiting the number of erases allowed on a block
per allocation, the allocator provides dynamic wear leveling over the entire
filesystem.
```
root
.--------.--------.
| A'| B'| |
| | |-> |
| | | |
'--------'--------'
.----' '--------------.
A v B v
.--------.--------. .--------.--------.
| C'| D'| | | E'|new| |
| | |-> | | | E'|-> |
| | | | | | | |
'--------'--------' '--------'--------'
.-' '--. | '------------------.
v v .-' v
.--------. .--------. v .--------.
| C | | D | .--------. write | new E |
| | | | | E | ==> | |
| | | | | | | |
'--------' '--------' | | '--------'
'--------' .-' |
.-' '-. .-------------|------'
v v v v
.--------. .--------. .--------.
| F | | G | | new F |
| | | | | |
| | | | | |
'--------' '--------' '--------'
```
More details on how littlefs works can be found in [DESIGN.md](DESIGN.md) and
[SPEC.md](SPEC.md).
- [DESIGN.md](DESIGN.md) - A fully detailed dive into how littlefs works.
I would suggest reading it as the tradeoffs at work are quite interesting.
- [SPEC.md](SPEC.md) - The on-disk specification of littlefs with all the
nitty-gritty details. May be useful for tooling development.
## Testing
The littlefs comes with a test suite designed to run on a pc using the
The littlefs comes with a test suite designed to run on a PC using the
[emulated block device](emubd/lfs_emubd.h) found in the emubd directory.
The tests assume a linux environment and can be started with make:
The tests assume a Linux environment and can be started with make:
``` bash
make test
```
## License
The littlefs is provided under the [BSD-3-Clause] license. See
[LICENSE.md](LICENSE.md) for more information. Contributions to this project
are accepted under the same license.
Individual files contain the following tag instead of the full license text.
SPDX-License-Identifier: BSD-3-Clause
This enables machine processing of license information based on the SPDX
License Identifiers that are here available: http://spdx.org/licenses/
## Related projects
[Mbed OS](https://github.com/ARMmbed/mbed-os/tree/master/features/filesystem/littlefs) -
The easiest way to get started with littlefs is to jump into [Mbed](https://os.mbed.com/),
which already has block device drivers for most forms of embedded storage. The
littlefs is available in Mbed OS as the [LittleFileSystem](https://os.mbed.com/docs/latest/reference/littlefilesystem.html)
class.
- [littlefs-fuse] - A [FUSE] wrapper for littlefs. The project allows you to
mount littlefs directly on a Linux machine. Can be useful for debugging
littlefs if you have an SD card handy.
[littlefs-fuse](https://github.com/geky/littlefs-fuse) - A [FUSE](https://github.com/libfuse/libfuse)
wrapper for littlefs. The project allows you to mount littlefs directly in a
Linux machine. Can be useful for debugging littlefs if you have an SD card
handy.
- [littlefs-js] - A javascript wrapper for littlefs. I'm not sure why you would
want this, but it is handy for demos. You can see it in action
[here][littlefs-js-demo].
[littlefs-js](https://github.com/geky/littlefs-js) - A javascript wrapper for
littlefs. I'm not sure why you would want this, but it is handy for demos.
You can see it in action [here](http://littlefs.geky.net/demo.html).
- [mklfs] - A command line tool built by the [Lua RTOS] guys for making
littlefs images from a host PC. Supports Windows, Mac OS, and Linux.
- [Mbed OS] - The easiest way to get started with littlefs is to jump into Mbed
which already has block device drivers for most forms of embedded storage.
littlefs is available in Mbed OS as the [LittleFileSystem] class.
- [SPIFFS] - Another excellent embedded filesystem for NOR flash. As a more
traditional logging filesystem with full static wear-leveling, SPIFFS will
likely outperform littlefs on small memories such as the internal flash on
microcontrollers.
- [Dhara] - An interesting NAND flash translation layer designed for small
MCUs. It offers static wear-leveling and power-resilience with only a fixed
_O(|address|)_ pointer structure stored on each block and in RAM.
[BSD-3-Clause]: https://spdx.org/licenses/BSD-3-Clause.html
[littlefs-fuse]: https://github.com/geky/littlefs-fuse
[FUSE]: https://github.com/libfuse/libfuse
[littlefs-js]: https://github.com/geky/littlefs-js
[littlefs-js-demo]:http://littlefs.geky.net/demo.html
[mklfs]: https://github.com/whitecatboard/Lua-RTOS-ESP32/tree/master/components/mklfs/src
[Lua RTOS]: https://github.com/whitecatboard/Lua-RTOS-ESP32
[Mbed OS]: https://github.com/armmbed/mbed-os
[LittleFileSystem]: https://os.mbed.com/docs/mbed-os/v5.12/apis/littlefilesystem.html
[SPIFFS]: https://github.com/pellepl/spiffs
[Dhara]: https://github.com/dlbeer/dhara

1045
SPEC.md

File diff suppressed because it is too large Load Diff

View File

@@ -1,19 +1,8 @@
/*
* Block device emulated on standard files
*
* Copyright (c) 2017 ARM Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* Copyright (c) 2017, Arm Limited. All rights reserved.
* SPDX-License-Identifier: BSD-3-Clause
*/
#include "emubd/lfs_emubd.h"
@@ -22,15 +11,58 @@
#include <stdlib.h>
#include <stdio.h>
#include <limits.h>
#include <dirent.h>
#include <sys/stat.h>
#include <unistd.h>
#include <assert.h>
#include <stdbool.h>
#include <inttypes.h>
// Emulated block device utils
static inline void lfs_emubd_tole32(lfs_emubd_t *emu) {
emu->cfg.read_size = lfs_tole32(emu->cfg.read_size);
emu->cfg.prog_size = lfs_tole32(emu->cfg.prog_size);
emu->cfg.block_size = lfs_tole32(emu->cfg.block_size);
emu->cfg.block_count = lfs_tole32(emu->cfg.block_count);
emu->stats.read_count = lfs_tole32(emu->stats.read_count);
emu->stats.prog_count = lfs_tole32(emu->stats.prog_count);
emu->stats.erase_count = lfs_tole32(emu->stats.erase_count);
for (unsigned i = 0; i < sizeof(emu->history.blocks) /
sizeof(emu->history.blocks[0]); i++) {
emu->history.blocks[i] = lfs_tole32(emu->history.blocks[i]);
}
}
static inline void lfs_emubd_fromle32(lfs_emubd_t *emu) {
emu->cfg.read_size = lfs_fromle32(emu->cfg.read_size);
emu->cfg.prog_size = lfs_fromle32(emu->cfg.prog_size);
emu->cfg.block_size = lfs_fromle32(emu->cfg.block_size);
emu->cfg.block_count = lfs_fromle32(emu->cfg.block_count);
emu->stats.read_count = lfs_fromle32(emu->stats.read_count);
emu->stats.prog_count = lfs_fromle32(emu->stats.prog_count);
emu->stats.erase_count = lfs_fromle32(emu->stats.erase_count);
for (unsigned i = 0; i < sizeof(emu->history.blocks) /
sizeof(emu->history.blocks[0]); i++) {
emu->history.blocks[i] = lfs_fromle32(emu->history.blocks[i]);
}
}
// Block device emulated on existing filesystem
int lfs_emubd_create(const struct lfs_config *cfg, const char *path) {
LFS_TRACE("lfs_emubd_create(%p {.context=%p, "
".read=%p, .prog=%p, .erase=%p, .sync=%p, "
".read_size=%"PRIu32", .prog_size=%"PRIu32", "
".block_size=%"PRIu32", .block_count=%"PRIu32"}, \"%s\")",
(void*)cfg, cfg->context,
(void*)(uintptr_t)cfg->read, (void*)(uintptr_t)cfg->prog,
(void*)(uintptr_t)cfg->erase, (void*)(uintptr_t)cfg->sync,
cfg->read_size, cfg->prog_size, cfg->block_size, cfg->block_count,
path);
lfs_emubd_t *emu = cfg->context;
emu->cfg.read_size = cfg->read_size;
emu->cfg.prog_size = cfg->prog_size;
@@ -41,7 +73,9 @@ int lfs_emubd_create(const struct lfs_config *cfg, const char *path) {
size_t pathlen = strlen(path);
emu->path = malloc(pathlen + 1 + LFS_NAME_MAX + 1);
if (!emu->path) {
return -ENOMEM;
int err = -ENOMEM;
LFS_TRACE("lfs_emubd_create -> %"PRId32, err);
return err;
}
strcpy(emu->path, path);
@@ -52,38 +86,74 @@ int lfs_emubd_create(const struct lfs_config *cfg, const char *path) {
// Create directory if it doesn't exist
int err = mkdir(path, 0777);
if (err && errno != EEXIST) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_create -> %"PRId32, err);
return err;
}
// Load stats to continue incrementing
snprintf(emu->child, LFS_NAME_MAX, "stats");
snprintf(emu->child, LFS_NAME_MAX, ".stats");
FILE *f = fopen(emu->path, "r");
if (!f) {
return -errno;
memset(&emu->stats, LFS_EMUBD_ERASE_VALUE, sizeof(emu->stats));
} else {
size_t res = fread(&emu->stats, sizeof(emu->stats), 1, f);
lfs_emubd_fromle32(emu);
if (res < 1) {
err = -errno;
LFS_TRACE("lfs_emubd_create -> %"PRId32, err);
fclose(f);
return err;
}
err = fclose(f);
if (err) {
err = -errno;
LFS_TRACE("lfs_emubd_create -> %"PRId32, err);
return err;
}
}
size_t res = fread(&emu->stats, sizeof(emu->stats), 1, f);
if (res < 1) {
return -errno;
}
err = fclose(f);
if (err) {
return -errno;
// Load history
snprintf(emu->child, LFS_NAME_MAX, ".history");
f = fopen(emu->path, "r");
if (!f) {
memset(&emu->history, 0, sizeof(emu->history));
} else {
size_t res = fread(&emu->history, sizeof(emu->history), 1, f);
lfs_emubd_fromle32(emu);
if (res < 1) {
err = -errno;
LFS_TRACE("lfs_emubd_create -> %"PRId32, err);
fclose(f);
return err;
}
err = fclose(f);
if (err) {
err = -errno;
LFS_TRACE("lfs_emubd_create -> %"PRId32, err);
return err;
}
}
LFS_TRACE("lfs_emubd_create -> %"PRId32, 0);
return 0;
}
void lfs_emubd_destroy(const struct lfs_config *cfg) {
LFS_TRACE("lfs_emubd_destroy(%p)", (void*)cfg);
lfs_emubd_sync(cfg);
lfs_emubd_t *emu = cfg->context;
free(emu->path);
LFS_TRACE("lfs_emubd_destroy -> %s", "void");
}
int lfs_emubd_read(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_emubd_read(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_emubd_t *emu = cfg->context;
uint8_t *data = buffer;
@@ -96,36 +166,49 @@ int lfs_emubd_read(const struct lfs_config *cfg, lfs_block_t block,
memset(data, 0, size);
// Read data
snprintf(emu->child, LFS_NAME_MAX, "%x", block);
snprintf(emu->child, LFS_NAME_MAX, "%" PRIx32, block);
FILE *f = fopen(emu->path, "rb");
if (!f && errno != ENOENT) {
return -errno;
int err = -errno;
LFS_TRACE("lfs_emubd_read -> %d", err);
return err;
}
if (f) {
int err = fseek(f, off, SEEK_SET);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_read -> %d", err);
fclose(f);
return err;
}
size_t res = fread(data, 1, size, f);
if (res < size && !feof(f)) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_read -> %d", err);
fclose(f);
return err;
}
err = fclose(f);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_read -> %d", err);
return err;
}
}
emu->stats.read_count += 1;
emu->stats.read_count += size;
LFS_TRACE("lfs_emubd_read -> %d", 0);
return 0;
}
int lfs_emubd_prog(const struct lfs_config *cfg, lfs_block_t block,
lfs_off_t off, const void *buffer, lfs_size_t size) {
LFS_TRACE("lfs_emubd_prog(%p, 0x%"PRIx32", %"PRIu32", %p, %"PRIu32")",
(void*)cfg, block, off, buffer, size);
lfs_emubd_t *emu = cfg->context;
const uint8_t *data = buffer;
@@ -135,11 +218,13 @@ int lfs_emubd_prog(const struct lfs_config *cfg, lfs_block_t block,
assert(block < cfg->block_count);
// Program data
snprintf(emu->child, LFS_NAME_MAX, "%x", block);
snprintf(emu->child, LFS_NAME_MAX, "%" PRIx32, block);
FILE *f = fopen(emu->path, "r+b");
if (!f) {
return (errno == EACCES) ? 0 : -errno;
int err = (errno == EACCES) ? 0 : -errno;
LFS_TRACE("lfs_emubd_prog -> %d", err);
return err;
}
// Check that file was erased
@@ -147,107 +232,183 @@ int lfs_emubd_prog(const struct lfs_config *cfg, lfs_block_t block,
int err = fseek(f, off, SEEK_SET);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_prog -> %d", err);
fclose(f);
return err;
}
size_t res = fwrite(data, 1, size, f);
if (res < size) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_prog -> %d", err);
fclose(f);
return err;
}
err = fseek(f, off, SEEK_SET);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_prog -> %d", err);
fclose(f);
return err;
}
uint8_t dat;
res = fread(&dat, 1, 1, f);
if (res < 1) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_prog -> %d", err);
fclose(f);
return err;
}
err = fclose(f);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_prog -> %d", err);
return err;
}
emu->stats.prog_count += 1;
// update history and stats
if (block != emu->history.blocks[0]) {
memmove(&emu->history.blocks[1], &emu->history.blocks[0],
sizeof(emu->history) - sizeof(emu->history.blocks[0]));
emu->history.blocks[0] = block;
}
emu->stats.prog_count += size;
LFS_TRACE("lfs_emubd_prog -> %d", 0);
return 0;
}
int lfs_emubd_erase(const struct lfs_config *cfg, lfs_block_t block) {
LFS_TRACE("lfs_emubd_erase(%p, 0x%"PRIx32")", (void*)cfg, block);
lfs_emubd_t *emu = cfg->context;
// Check if erase is valid
assert(block < cfg->block_count);
// Erase the block
snprintf(emu->child, LFS_NAME_MAX, "%x", block);
snprintf(emu->child, LFS_NAME_MAX, "%" PRIx32, block);
struct stat st;
int err = stat(emu->path, &st);
if (err && errno != ENOENT) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_erase -> %d", err);
return err;
}
if (!err && S_ISREG(st.st_mode) && (S_IWUSR & st.st_mode)) {
int err = unlink(emu->path);
err = unlink(emu->path);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_erase -> %d", err);
return err;
}
}
if (errno == ENOENT || (S_ISREG(st.st_mode) && (S_IWUSR & st.st_mode))) {
if (err || (S_ISREG(st.st_mode) && (S_IWUSR & st.st_mode))) {
FILE *f = fopen(emu->path, "w");
if (!f) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_erase -> %d", err);
return err;
}
err = fclose(f);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_erase -> %d", err);
return err;
}
}
emu->stats.erase_count += 1;
emu->stats.erase_count += cfg->block_size;
LFS_TRACE("lfs_emubd_erase -> %d", 0);
return 0;
}
int lfs_emubd_sync(const struct lfs_config *cfg) {
LFS_TRACE("lfs_emubd_sync(%p)", (void*)cfg);
lfs_emubd_t *emu = cfg->context;
// Just write out info/stats for later lookup
snprintf(emu->child, LFS_NAME_MAX, "config");
snprintf(emu->child, LFS_NAME_MAX, ".config");
FILE *f = fopen(emu->path, "w");
if (!f) {
return -errno;
int err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
return err;
}
lfs_emubd_tole32(emu);
size_t res = fwrite(&emu->cfg, sizeof(emu->cfg), 1, f);
lfs_emubd_fromle32(emu);
if (res < 1) {
return -errno;
int err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
fclose(f);
return err;
}
int err = fclose(f);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
return err;
}
snprintf(emu->child, LFS_NAME_MAX, "stats");
snprintf(emu->child, LFS_NAME_MAX, ".stats");
f = fopen(emu->path, "w");
if (!f) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
return err;
}
lfs_emubd_tole32(emu);
res = fwrite(&emu->stats, sizeof(emu->stats), 1, f);
lfs_emubd_fromle32(emu);
if (res < 1) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
fclose(f);
return err;
}
err = fclose(f);
if (err) {
return -errno;
err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
return err;
}
snprintf(emu->child, LFS_NAME_MAX, ".history");
f = fopen(emu->path, "w");
if (!f) {
err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
return err;
}
lfs_emubd_tole32(emu);
res = fwrite(&emu->history, sizeof(emu->history), 1, f);
lfs_emubd_fromle32(emu);
if (res < 1) {
err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
fclose(f);
return err;
}
err = fclose(f);
if (err) {
err = -errno;
LFS_TRACE("lfs_emubd_sync -> %d", err);
return err;
}
LFS_TRACE("lfs_emubd_sync -> %d", 0);
return 0;
}

View File

@@ -1,19 +1,8 @@
/*
* Block device emulated on standard files
*
* Copyright (c) 2017 ARM Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* Copyright (c) 2017, Arm Limited. All rights reserved.
* SPDX-License-Identifier: BSD-3-Clause
*/
#ifndef LFS_EMUBD_H
#define LFS_EMUBD_H
@@ -21,22 +10,15 @@
#include "lfs.h"
#include "lfs_util.h"
#ifdef __cplusplus
extern "C"
{
#endif
// Config options
#ifndef LFS_EMUBD_READ_SIZE
#define LFS_EMUBD_READ_SIZE 1
#endif
#ifndef LFS_EMUBD_PROG_SIZE
#define LFS_EMUBD_PROG_SIZE 1
#endif
#ifndef LFS_EMUBD_ERASE_SIZE
#define LFS_EMUBD_ERASE_SIZE 512
#endif
#ifndef LFS_EMUBD_TOTAL_SIZE
#define LFS_EMUBD_TOTAL_SIZE 524288
#ifndef LFS_EMUBD_ERASE_VALUE
#define LFS_EMUBD_ERASE_VALUE 0x00
#endif
@@ -51,6 +33,10 @@ typedef struct lfs_emubd {
uint64_t erase_count;
} stats;
struct {
lfs_block_t blocks[4];
} history;
struct {
uint32_t read_size;
uint32_t prog_size;
@@ -86,4 +72,8 @@ int lfs_emubd_erase(const struct lfs_config *cfg, lfs_block_t block);
int lfs_emubd_sync(const struct lfs_config *cfg);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif

5505
lfs.c

File diff suppressed because it is too large Load Diff

488
lfs.h
View File

@@ -1,19 +1,8 @@
/*
* The little filesystem
*
* Copyright (c) 2017 ARM Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* Copyright (c) 2017, Arm Limited. All rights reserved.
* SPDX-License-Identifier: BSD-3-Clause
*/
#ifndef LFS_H
#define LFS_H
@@ -21,6 +10,28 @@
#include <stdint.h>
#include <stdbool.h>
#ifdef __cplusplus
extern "C"
{
#endif
/// Version info ///
// Software library version
// Major (top-nibble), incremented on backwards incompatible changes
// Minor (bottom-nibble), incremented on feature additions
#define LFS_VERSION 0x00020001
#define LFS_VERSION_MAJOR (0xffff & (LFS_VERSION >> 16))
#define LFS_VERSION_MINOR (0xffff & (LFS_VERSION >> 0))
// Version of On-disk data structures
// Major (top-nibble), incremented on backwards incompatible changes
// Minor (bottom-nibble), incremented on feature additions
#define LFS_DISK_VERSION 0x00020000
#define LFS_DISK_VERSION_MAJOR (0xffff & (LFS_DISK_VERSION >> 16))
#define LFS_DISK_VERSION_MINOR (0xffff & (LFS_DISK_VERSION >> 0))
/// Definitions ///
@@ -33,50 +44,99 @@ typedef int32_t lfs_soff_t;
typedef uint32_t lfs_block_t;
// Max name size in bytes
// Maximum name size in bytes, may be redefined to reduce the size of the
// info struct. Limited to <= 1022. Stored in superblock and must be
// respected by other littlefs drivers.
#ifndef LFS_NAME_MAX
#define LFS_NAME_MAX 255
#endif
// Maximum size of a file in bytes, may be redefined to limit to support other
// drivers. Limited on disk to <= 4294967296. However, above 2147483647 the
// functions lfs_file_seek, lfs_file_size, and lfs_file_tell will return
// incorrect values due to using signed integers. Stored in superblock and
// must be respected by other littlefs drivers.
#ifndef LFS_FILE_MAX
#define LFS_FILE_MAX 2147483647
#endif
// Maximum size of custom attributes in bytes, may be redefined, but there is
// no real benefit to using a smaller LFS_ATTR_MAX. Limited to <= 1022.
#ifndef LFS_ATTR_MAX
#define LFS_ATTR_MAX 1022
#endif
// Possible error codes, these are negative to allow
// valid positive return values
enum lfs_error {
LFS_ERR_OK = 0, // No error
LFS_ERR_IO = -5, // Error during device operation
LFS_ERR_CORRUPT = -52, // Corrupted
LFS_ERR_NOENT = -2, // No directory entry
LFS_ERR_EXIST = -17, // Entry already exists
LFS_ERR_NOTDIR = -20, // Entry is not a dir
LFS_ERR_ISDIR = -21, // Entry is a dir
LFS_ERR_NOTEMPTY = -39, // Dir is not empty
LFS_ERR_INVAL = -22, // Invalid parameter
LFS_ERR_NOSPC = -28, // No space left on device
LFS_ERR_NOMEM = -12, // No more memory available
LFS_ERR_OK = 0, // No error
LFS_ERR_IO = -5, // Error during device operation
LFS_ERR_CORRUPT = -84, // Corrupted
LFS_ERR_NOENT = -2, // No directory entry
LFS_ERR_EXIST = -17, // Entry already exists
LFS_ERR_NOTDIR = -20, // Entry is not a dir
LFS_ERR_ISDIR = -21, // Entry is a dir
LFS_ERR_NOTEMPTY = -39, // Dir is not empty
LFS_ERR_BADF = -9, // Bad file number
LFS_ERR_FBIG = -27, // File too large
LFS_ERR_INVAL = -22, // Invalid parameter
LFS_ERR_NOSPC = -28, // No space left on device
LFS_ERR_NOMEM = -12, // No more memory available
LFS_ERR_NOATTR = -61, // No data/attr available
LFS_ERR_NAMETOOLONG = -36, // File name too long
};
// File types
enum lfs_type {
LFS_TYPE_REG = 0x11,
LFS_TYPE_DIR = 0x22,
LFS_TYPE_SUPERBLOCK = 0x2e,
// file types
LFS_TYPE_REG = 0x001,
LFS_TYPE_DIR = 0x002,
// internally used types
LFS_TYPE_SPLICE = 0x400,
LFS_TYPE_NAME = 0x000,
LFS_TYPE_STRUCT = 0x200,
LFS_TYPE_USERATTR = 0x300,
LFS_TYPE_FROM = 0x100,
LFS_TYPE_TAIL = 0x600,
LFS_TYPE_GLOBALS = 0x700,
LFS_TYPE_CRC = 0x500,
// internally used type specializations
LFS_TYPE_CREATE = 0x401,
LFS_TYPE_DELETE = 0x4ff,
LFS_TYPE_SUPERBLOCK = 0x0ff,
LFS_TYPE_DIRSTRUCT = 0x200,
LFS_TYPE_CTZSTRUCT = 0x202,
LFS_TYPE_INLINESTRUCT = 0x201,
LFS_TYPE_SOFTTAIL = 0x600,
LFS_TYPE_HARDTAIL = 0x601,
LFS_TYPE_MOVESTATE = 0x7ff,
// internal chip sources
LFS_FROM_NOOP = 0x000,
LFS_FROM_MOVE = 0x101,
LFS_FROM_USERATTRS = 0x102,
};
// File open flags
enum lfs_open_flags {
// open flags
LFS_O_RDONLY = 1, // Open a file as read only
LFS_O_WRONLY = 2, // Open a file as write only
LFS_O_RDWR = 3, // Open a file as read and write
LFS_O_CREAT = 0x0100, // Create a file if it does not exist
LFS_O_EXCL = 0x0200, // Fail if a file already exists
LFS_O_TRUNC = 0x0400, // Truncate the existing file to zero size
LFS_O_APPEND = 0x0800, // Move to end of file on every write
LFS_O_RDONLY = 1, // Open a file as read only
LFS_O_WRONLY = 2, // Open a file as write only
LFS_O_RDWR = 3, // Open a file as read and write
LFS_O_CREAT = 0x0100, // Create a file if it does not exist
LFS_O_EXCL = 0x0200, // Fail if a file already exists
LFS_O_TRUNC = 0x0400, // Truncate the existing file to zero size
LFS_O_APPEND = 0x0800, // Move to end of file on every write
// internally used flags
LFS_F_DIRTY = 0x10000, // File does not match storage
LFS_F_WRITING = 0x20000, // File has been written since last flush
LFS_F_READING = 0x40000, // File has been read since last flush
LFS_F_ERRED = 0x80000, // An error occured during write
LFS_F_DIRTY = 0x010000, // File does not match storage
LFS_F_WRITING = 0x020000, // File has been written since last flush
LFS_F_READING = 0x040000, // File has been read since last flush
LFS_F_ERRED = 0x080000, // An error occured during write
LFS_F_INLINE = 0x100000, // Currently inlined in directory entry
LFS_F_OPENED = 0x200000, // File has been opened
};
// File seek flags
@@ -114,152 +174,222 @@ struct lfs_config {
// are propogated to the user.
int (*sync)(const struct lfs_config *c);
// Minimum size of a block read. This determines the size of read buffers.
// This may be larger than the physical read size to improve performance
// by caching more of the block device.
// Minimum size of a block read. All read operations will be a
// multiple of this value.
lfs_size_t read_size;
// Minimum size of a block program. This determines the size of program
// buffers. This may be larger than the physical program size to improve
// performance by caching more of the block device.
// Must be a multiple of the read size.
// Minimum size of a block program. All program operations will be a
// multiple of this value.
lfs_size_t prog_size;
// Size of an erasable block. This does not impact ram consumption and
// may be larger than the physical erase size. However, this should be
// kept small as each file currently takes up an entire block.
// Must be a multiple of the program size.
// may be larger than the physical erase size. However, non-inlined files
// take up at minimum one block. Must be a multiple of the read
// and program sizes.
lfs_size_t block_size;
// Number of erasable blocks on the device.
lfs_size_t block_count;
// Number of blocks to lookahead during block allocation. A larger
// lookahead reduces the number of passes required to allocate a block.
// The lookahead buffer requires only 1 bit per block so it can be quite
// large with little ram impact. Should be a multiple of 32.
lfs_size_t lookahead;
// Number of erase cycles before littlefs evicts metadata logs and moves
// the metadata to another block. Suggested values are in the
// range 100-1000, with large values having better performance at the cost
// of less consistent wear distribution.
//
// Set to -1 to disable block-level wear-leveling.
int32_t block_cycles;
// Optional, statically allocated read buffer. Must be read sized.
// Size of block caches. Each cache buffers a portion of a block in RAM.
// The littlefs needs a read cache, a program cache, and one additional
// cache per file. Larger caches can improve performance by storing more
// data and reducing the number of disk accesses. Must be a multiple of
// the read and program sizes, and a factor of the block size.
lfs_size_t cache_size;
// Size of the lookahead buffer in bytes. A larger lookahead buffer
// increases the number of blocks found during an allocation pass. The
// lookahead buffer is stored as a compact bitmap, so each byte of RAM
// can track 8 blocks. Must be a multiple of 8.
lfs_size_t lookahead_size;
// Optional statically allocated read buffer. Must be cache_size.
// By default lfs_malloc is used to allocate this buffer.
void *read_buffer;
// Optional, statically allocated program buffer. Must be program sized.
// Optional statically allocated program buffer. Must be cache_size.
// By default lfs_malloc is used to allocate this buffer.
void *prog_buffer;
// Optional, statically allocated lookahead buffer. Must be 1 bit per
// lookahead block.
// Optional statically allocated lookahead buffer. Must be lookahead_size
// and aligned to a 32-bit boundary. By default lfs_malloc is used to
// allocate this buffer.
void *lookahead_buffer;
// Optional, statically allocated buffer for files. Must be program sized.
// If enabled, only one file may be opened at a time.
void *file_buffer;
};
// Optional upper limit on length of file names in bytes. No downside for
// larger names except the size of the info struct which is controlled by
// the LFS_NAME_MAX define. Defaults to LFS_NAME_MAX when zero. Stored in
// superblock and must be respected by other littlefs drivers.
lfs_size_t name_max;
// Optional upper limit on files in bytes. No downside for larger files
// but must be <= LFS_FILE_MAX. Defaults to LFS_FILE_MAX when zero. Stored
// in superblock and must be respected by other littlefs drivers.
lfs_size_t file_max;
// Optional upper limit on custom attributes in bytes. No downside for
// larger attributes size but must be <= LFS_ATTR_MAX. Defaults to
// LFS_ATTR_MAX when zero.
lfs_size_t attr_max;
};
// File info structure
struct lfs_info {
// Type of the file, either LFS_TYPE_REG or LFS_TYPE_DIR
uint8_t type;
// Size of the file, only valid for REG files
// Size of the file, only valid for REG files. Limited to 32-bits.
lfs_size_t size;
// Name of the file stored as a null-terminated string
// Name of the file stored as a null-terminated string. Limited to
// LFS_NAME_MAX+1, which can be changed by redefining LFS_NAME_MAX to
// reduce RAM. LFS_NAME_MAX is stored in superblock and must be
// respected by other littlefs drivers.
char name[LFS_NAME_MAX+1];
};
// Custom attribute structure, used to describe custom attributes
// committed atomically during file writes.
struct lfs_attr {
// 8-bit type of attribute, provided by user and used to
// identify the attribute
uint8_t type;
/// littlefs data structures ///
typedef struct lfs_entry {
lfs_off_t off;
// Pointer to buffer containing the attribute
void *buffer;
struct lfs_disk_entry {
uint8_t type;
uint8_t elen;
uint8_t alen;
uint8_t nlen;
union {
struct {
lfs_block_t head;
lfs_size_t size;
} file;
lfs_block_t dir[2];
} u;
} d;
} lfs_entry_t;
// Size of attribute in bytes, limited to LFS_ATTR_MAX
lfs_size_t size;
};
// Optional configuration provided during lfs_file_opencfg
struct lfs_file_config {
// Optional statically allocated file buffer. Must be cache_size.
// By default lfs_malloc is used to allocate this buffer.
void *buffer;
// Optional list of custom attributes related to the file. If the file
// is opened with read access, these attributes will be read from disk
// during the open call. If the file is opened with write access, the
// attributes will be written to disk every file sync or close. This
// write occurs atomically with update to the file's contents.
//
// Custom attributes are uniquely identified by an 8-bit type and limited
// to LFS_ATTR_MAX bytes. When read, if the stored attribute is smaller
// than the buffer, it will be padded with zeros. If the stored attribute
// is larger, then it will be silently truncated. If the attribute is not
// found, it will be created implicitly.
struct lfs_attr *attrs;
// Number of custom attributes in the list
lfs_size_t attr_count;
};
/// internal littlefs data structures ///
typedef struct lfs_cache {
lfs_block_t block;
lfs_off_t off;
lfs_size_t size;
uint8_t *buffer;
} lfs_cache_t;
typedef struct lfs_mdir {
lfs_block_t pair[2];
uint32_t rev;
lfs_off_t off;
uint32_t etag;
uint16_t count;
bool erased;
bool split;
lfs_block_t tail[2];
} lfs_mdir_t;
// littlefs directory type
typedef struct lfs_dir {
struct lfs_dir *next;
uint16_t id;
uint8_t type;
lfs_mdir_t m;
lfs_off_t pos;
lfs_block_t head[2];
} lfs_dir_t;
// littlefs file type
typedef struct lfs_file {
struct lfs_file *next;
lfs_block_t pair[2];
lfs_off_t poff;
uint16_t id;
uint8_t type;
lfs_mdir_t m;
lfs_block_t head;
lfs_size_t size;
struct lfs_ctz {
lfs_block_t head;
lfs_size_t size;
} ctz;
uint32_t flags;
lfs_off_t pos;
lfs_block_t block;
lfs_off_t off;
lfs_cache_t cache;
const struct lfs_file_config *cfg;
} lfs_file_t;
typedef struct lfs_dir {
struct lfs_dir *next;
lfs_block_t pair[2];
lfs_off_t off;
lfs_block_t head[2];
lfs_off_t pos;
struct lfs_disk_dir {
uint32_t rev;
lfs_size_t size;
lfs_block_t tail[2];
} d;
} lfs_dir_t;
typedef struct lfs_superblock {
lfs_off_t off;
struct lfs_disk_superblock {
uint8_t type;
uint8_t elen;
uint8_t alen;
uint8_t nlen;
lfs_block_t root[2];
uint32_t block_size;
uint32_t block_count;
uint32_t version;
char magic[8];
} d;
uint32_t version;
lfs_size_t block_size;
lfs_size_t block_count;
lfs_size_t name_max;
lfs_size_t file_max;
lfs_size_t attr_max;
} lfs_superblock_t;
typedef struct lfs_free {
lfs_block_t begin;
lfs_block_t end;
lfs_block_t off;
uint32_t *buffer;
} lfs_free_t;
// The littlefs type
// The littlefs filesystem type
typedef struct lfs {
const struct lfs_config *cfg;
lfs_block_t root[2];
lfs_file_t *files;
lfs_dir_t *dirs;
lfs_cache_t rcache;
lfs_cache_t pcache;
lfs_free_t free;
bool deorphaned;
lfs_block_t root[2];
struct lfs_mlist {
struct lfs_mlist *next;
uint16_t id;
uint8_t type;
lfs_mdir_t m;
} *mlist;
uint32_t seed;
struct lfs_gstate {
uint32_t tag;
lfs_block_t pair[2];
} gstate, gpending, gdelta;
struct lfs_free {
lfs_block_t off;
lfs_block_t size;
lfs_block_t i;
lfs_block_t ack;
uint32_t *buffer;
} free;
const struct lfs_config *cfg;
lfs_size_t name_max;
lfs_size_t file_max;
lfs_size_t attr_max;
#ifdef LFS_MIGRATE
struct lfs1 *lfs1;
#endif
} lfs_t;
@@ -268,7 +398,8 @@ typedef struct lfs {
// Format a block device with the littlefs
//
// Requires a littlefs object and config struct. This clobbers the littlefs
// object, and does not leave the filesystem mounted.
// object, and does not leave the filesystem mounted. The config struct must
// be zeroed for defaults and backwards compatibility.
//
// Returns a negative error code on failure.
int lfs_format(lfs_t *lfs, const struct lfs_config *config);
@@ -277,7 +408,8 @@ int lfs_format(lfs_t *lfs, const struct lfs_config *config);
//
// Requires a littlefs object and config struct. Multiple filesystems
// may be mounted simultaneously with multiple littlefs objects. Both
// lfs and config must be allocated while mounted.
// lfs and config must be allocated while mounted. The config struct must
// be zeroed for defaults and backwards compatibility.
//
// Returns a negative error code on failure.
int lfs_mount(lfs_t *lfs, const struct lfs_config *config);
@@ -301,10 +433,6 @@ int lfs_remove(lfs_t *lfs, const char *path);
// If the destination exists, it must match the source in type.
// If the destination is a directory, the directory must be empty.
//
// Note: If power loss occurs, it is possible that the file or directory
// will exist in both the oldpath and newpath simultaneously after the
// next mount.
//
// Returns a negative error code on failure.
int lfs_rename(lfs_t *lfs, const char *oldpath, const char *newpath);
@@ -314,19 +442,64 @@ int lfs_rename(lfs_t *lfs, const char *oldpath, const char *newpath);
// Returns a negative error code on failure.
int lfs_stat(lfs_t *lfs, const char *path, struct lfs_info *info);
// Get a custom attribute
//
// Custom attributes are uniquely identified by an 8-bit type and limited
// to LFS_ATTR_MAX bytes. When read, if the stored attribute is smaller than
// the buffer, it will be padded with zeros. If the stored attribute is larger,
// then it will be silently truncated. If no attribute is found, the error
// LFS_ERR_NOATTR is returned and the buffer is filled with zeros.
//
// Returns the size of the attribute, or a negative error code on failure.
// Note, the returned size is the size of the attribute on disk, irrespective
// of the size of the buffer. This can be used to dynamically allocate a buffer
// or check for existance.
lfs_ssize_t lfs_getattr(lfs_t *lfs, const char *path,
uint8_t type, void *buffer, lfs_size_t size);
// Set custom attributes
//
// Custom attributes are uniquely identified by an 8-bit type and limited
// to LFS_ATTR_MAX bytes. If an attribute is not found, it will be
// implicitly created.
//
// Returns a negative error code on failure.
int lfs_setattr(lfs_t *lfs, const char *path,
uint8_t type, const void *buffer, lfs_size_t size);
// Removes a custom attribute
//
// If an attribute is not found, nothing happens.
//
// Returns a negative error code on failure.
int lfs_removeattr(lfs_t *lfs, const char *path, uint8_t type);
/// File operations ///
// Open a file
//
// The mode that the file is opened in is determined
// by the flags, which are values from the enum lfs_open_flags
// that are bitwise-ored together.
// The mode that the file is opened in is determined by the flags, which
// are values from the enum lfs_open_flags that are bitwise-ored together.
//
// Returns a negative error code on failure.
int lfs_file_open(lfs_t *lfs, lfs_file_t *file,
const char *path, int flags);
// Open a file with extra configuration
//
// The mode that the file is opened in is determined by the flags, which
// are values from the enum lfs_open_flags that are bitwise-ored together.
//
// The config struct provides additional config options per file as described
// above. The config struct must be allocated while the file is open, and the
// config struct must be zeroed for defaults and backwards compatibility.
//
// Returns a negative error code on failure.
int lfs_file_opencfg(lfs_t *lfs, lfs_file_t *file,
const char *path, int flags,
const struct lfs_file_config *config);
// Close a file
//
// Any pending writes are written out to storage as though
@@ -360,7 +533,7 @@ lfs_ssize_t lfs_file_write(lfs_t *lfs, lfs_file_t *file,
// Change the position of the file
//
// The change in position is determined by the offset and whence flag.
// Returns the old position of the file, or a negative error code on failure.
// Returns the new position of the file, or a negative error code on failure.
lfs_soff_t lfs_file_seek(lfs_t *lfs, lfs_file_t *file,
lfs_soff_t off, int whence);
@@ -377,7 +550,7 @@ lfs_soff_t lfs_file_tell(lfs_t *lfs, lfs_file_t *file);
// Change the position of the file to the beginning of the file
//
// Equivalent to lfs_file_seek(lfs, file, 0, LFS_SEEK_CUR)
// Equivalent to lfs_file_seek(lfs, file, 0, LFS_SEEK_SET)
// Returns a negative error code on failure.
int lfs_file_rewind(lfs_t *lfs, lfs_file_t *file);
@@ -410,7 +583,8 @@ int lfs_dir_close(lfs_t *lfs, lfs_dir_t *dir);
// Read an entry in the directory
//
// Fills out the info structure, based on the specified file or directory.
// Returns a negative error code on failure.
// Returns a positive value on success, 0 at the end of directory,
// or a negative error code on failure.
int lfs_dir_read(lfs_t *lfs, lfs_dir_t *dir, struct lfs_info *info);
// Change the position of the directory
@@ -435,7 +609,15 @@ lfs_soff_t lfs_dir_tell(lfs_t *lfs, lfs_dir_t *dir);
int lfs_dir_rewind(lfs_t *lfs, lfs_dir_t *dir);
/// Miscellaneous littlefs specific operations ///
/// Filesystem-level filesystem operations
// Finds the current size of the filesystem
//
// Note: Result is best effort. If files share COW structures, the returned
// size may be larger than the filesystem actually is.
//
// Returns the number of allocated blocks, or a negative error code on failure.
lfs_ssize_t lfs_fs_size(lfs_t *lfs);
// Traverse through all blocks in use by the filesystem
//
@@ -444,16 +626,26 @@ int lfs_dir_rewind(lfs_t *lfs, lfs_dir_t *dir);
// blocks are in use or how much of the storage is available.
//
// Returns a negative error code on failure.
int lfs_traverse(lfs_t *lfs, int (*cb)(void*, lfs_block_t), void *data);
int lfs_fs_traverse(lfs_t *lfs, int (*cb)(void*, lfs_block_t), void *data);
// Prunes any recoverable errors that may have occured in the filesystem
#ifdef LFS_MIGRATE
// Attempts to migrate a previous version of littlefs
//
// Not needed to be called by user unless an operation is interrupted
// but the filesystem is still mounted. This is already called on first
// allocation.
// Behaves similarly to the lfs_format function. Attempts to mount
// the previous version of littlefs and update the filesystem so it can be
// mounted with the current version of littlefs.
//
// Requires a littlefs object and config struct. This clobbers the littlefs
// object, and does not leave the filesystem mounted. The config struct must
// be zeroed for defaults and backwards compatibility.
//
// Returns a negative error code on failure.
int lfs_deorphan(lfs_t *lfs);
int lfs_migrate(lfs_t *lfs, const struct lfs_config *cfg);
#endif
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif

View File

@@ -1,24 +1,17 @@
/*
* lfs util functions
*
* Copyright (c) 2017 ARM Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* Copyright (c) 2017, Arm Limited. All rights reserved.
* SPDX-License-Identifier: BSD-3-Clause
*/
#include "lfs_util.h"
// Only compile if user does not provide custom config
#ifndef LFS_CONFIG
void lfs_crc(uint32_t *restrict crc, const void *buffer, size_t size) {
// Software CRC implementation with small lookup table
uint32_t lfs_crc(uint32_t crc, const void *buffer, size_t size) {
static const uint32_t rtable[16] = {
0x00000000, 0x1db71064, 0x3b6e20c8, 0x26d930ac,
0x76dc4190, 0x6b6b51f4, 0x4db26158, 0x5005713c,
@@ -29,8 +22,12 @@ void lfs_crc(uint32_t *restrict crc, const void *buffer, size_t size) {
const uint8_t *data = buffer;
for (size_t i = 0; i < size; i++) {
*crc = (*crc >> 4) ^ rtable[(*crc ^ (data[i] >> 0)) & 0xf];
*crc = (*crc >> 4) ^ rtable[(*crc ^ (data[i] >> 4)) & 0xf];
crc = (crc >> 4) ^ rtable[(crc ^ (data[i] >> 0)) & 0xf];
crc = (crc >> 4) ^ rtable[(crc ^ (data[i] >> 4)) & 0xf];
}
return crc;
}
#endif

View File

@@ -1,30 +1,95 @@
/*
* lfs utility functions
*
* Copyright (c) 2017 ARM Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* Copyright (c) 2017, Arm Limited. All rights reserved.
* SPDX-License-Identifier: BSD-3-Clause
*/
#ifndef LFS_UTIL_H
#define LFS_UTIL_H
#include <stdlib.h>
// Users can override lfs_util.h with their own configuration by defining
// LFS_CONFIG as a header file to include (-DLFS_CONFIG=lfs_config.h).
//
// If LFS_CONFIG is used, none of the default utils will be emitted and must be
// provided by the config file. To start, I would suggest copying lfs_util.h
// and modifying as needed.
#ifdef LFS_CONFIG
#define LFS_STRINGIZE(x) LFS_STRINGIZE2(x)
#define LFS_STRINGIZE2(x) #x
#include LFS_STRINGIZE(LFS_CONFIG)
#else
// System includes
#include <stdint.h>
#include <stdbool.h>
#include <string.h>
#include <inttypes.h>
#ifndef LFS_NO_MALLOC
#include <stdlib.h>
#endif
#ifndef LFS_NO_ASSERT
#include <assert.h>
#endif
#if !defined(LFS_NO_DEBUG) || \
!defined(LFS_NO_WARN) || \
!defined(LFS_NO_ERROR) || \
defined(LFS_YES_TRACE)
#include <stdio.h>
#endif
#ifdef __cplusplus
extern "C"
{
#endif
// Builtin functions, these may be replaced by more
// efficient implementations in the system
// Macros, may be replaced by system specific wrappers. Arguments to these
// macros must not have side-effects as the macros can be removed for a smaller
// code footprint
// Logging functions
#ifdef LFS_YES_TRACE
#define LFS_TRACE(fmt, ...) \
printf("lfs_trace:%d: " fmt "\n", __LINE__, __VA_ARGS__)
#else
#define LFS_TRACE(fmt, ...)
#endif
#ifndef LFS_NO_DEBUG
#define LFS_DEBUG(fmt, ...) \
printf("lfs_debug:%d: " fmt "\n", __LINE__, __VA_ARGS__)
#else
#define LFS_DEBUG(fmt, ...)
#endif
#ifndef LFS_NO_WARN
#define LFS_WARN(fmt, ...) \
printf("lfs_warn:%d: " fmt "\n", __LINE__, __VA_ARGS__)
#else
#define LFS_WARN(fmt, ...)
#endif
#ifndef LFS_NO_ERROR
#define LFS_ERROR(fmt, ...) \
printf("lfs_error:%d: " fmt "\n", __LINE__, __VA_ARGS__)
#else
#define LFS_ERROR(fmt, ...)
#endif
// Runtime assertions
#ifndef LFS_NO_ASSERT
#define LFS_ASSERT(test) assert(test)
#else
#define LFS_ASSERT(test)
#endif
// Builtin functions, these may be replaced by more efficient
// toolchain-specific implementations. LFS_NO_INTRINSICS falls back to a more
// expensive basic C implementation for debugging purposes
// Min/max functions for unsigned 32-bit numbers
static inline uint32_t lfs_max(uint32_t a, uint32_t b) {
return (a > b) ? a : b;
}
@@ -33,26 +98,69 @@ static inline uint32_t lfs_min(uint32_t a, uint32_t b) {
return (a < b) ? a : b;
}
static inline uint32_t lfs_ctz(uint32_t a) {
return __builtin_ctz(a);
// Align to nearest multiple of a size
static inline uint32_t lfs_aligndown(uint32_t a, uint32_t alignment) {
return a - (a % alignment);
}
static inline uint32_t lfs_alignup(uint32_t a, uint32_t alignment) {
return lfs_aligndown(a + alignment-1, alignment);
}
// Find the next smallest power of 2 less than or equal to a
static inline uint32_t lfs_npw2(uint32_t a) {
#if !defined(LFS_NO_INTRINSICS) && (defined(__GNUC__) || defined(__CC_ARM))
return 32 - __builtin_clz(a-1);
#else
uint32_t r = 0;
uint32_t s;
a -= 1;
s = (a > 0xffff) << 4; a >>= s; r |= s;
s = (a > 0xff ) << 3; a >>= s; r |= s;
s = (a > 0xf ) << 2; a >>= s; r |= s;
s = (a > 0x3 ) << 1; a >>= s; r |= s;
return (r | (a >> 1)) + 1;
#endif
}
// Count the number of trailing binary zeros in a
// lfs_ctz(0) may be undefined
static inline uint32_t lfs_ctz(uint32_t a) {
#if !defined(LFS_NO_INTRINSICS) && defined(__GNUC__)
return __builtin_ctz(a);
#else
return lfs_npw2((a & -a) + 1) - 1;
#endif
}
// Count the number of binary ones in a
static inline uint32_t lfs_popc(uint32_t a) {
#if !defined(LFS_NO_INTRINSICS) && (defined(__GNUC__) || defined(__CC_ARM))
return __builtin_popcount(a);
#else
a = a - ((a >> 1) & 0x55555555);
a = (a & 0x33333333) + ((a >> 2) & 0x33333333);
return (((a + (a >> 4)) & 0xf0f0f0f) * 0x1010101) >> 24;
#endif
}
// Find the sequence comparison of a and b, this is the distance
// between a and b ignoring overflow
static inline int lfs_scmp(uint32_t a, uint32_t b) {
return (int)(unsigned)(a - b);
}
// Convert between 32-bit little-endian and native order
static inline uint32_t lfs_fromle32(uint32_t a) {
#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#if !defined(LFS_NO_INTRINSICS) && ( \
(defined( BYTE_ORDER ) && defined( ORDER_LITTLE_ENDIAN ) && BYTE_ORDER == ORDER_LITTLE_ENDIAN ) || \
(defined(__BYTE_ORDER ) && defined(__ORDER_LITTLE_ENDIAN ) && __BYTE_ORDER == __ORDER_LITTLE_ENDIAN ) || \
(defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__))
return a;
#elif defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
#elif !defined(LFS_NO_INTRINSICS) && ( \
(defined( BYTE_ORDER ) && defined( ORDER_BIG_ENDIAN ) && BYTE_ORDER == ORDER_BIG_ENDIAN ) || \
(defined(__BYTE_ORDER ) && defined(__ORDER_BIG_ENDIAN ) && __BYTE_ORDER == __ORDER_BIG_ENDIAN ) || \
(defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__))
return __builtin_bswap32(a);
#else
return (((uint8_t*)&a)[0] << 0) |
@@ -66,15 +174,57 @@ static inline uint32_t lfs_tole32(uint32_t a) {
return lfs_fromle32(a);
}
// CRC-32 with polynomial = 0x04c11db7
void lfs_crc(uint32_t *crc, const void *buffer, size_t size);
// Convert between 32-bit big-endian and native order
static inline uint32_t lfs_frombe32(uint32_t a) {
#if !defined(LFS_NO_INTRINSICS) && ( \
(defined( BYTE_ORDER ) && defined( ORDER_LITTLE_ENDIAN ) && BYTE_ORDER == ORDER_LITTLE_ENDIAN ) || \
(defined(__BYTE_ORDER ) && defined(__ORDER_LITTLE_ENDIAN ) && __BYTE_ORDER == __ORDER_LITTLE_ENDIAN ) || \
(defined(__BYTE_ORDER__) && defined(__ORDER_LITTLE_ENDIAN__) && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__))
return __builtin_bswap32(a);
#elif !defined(LFS_NO_INTRINSICS) && ( \
(defined( BYTE_ORDER ) && defined( ORDER_BIG_ENDIAN ) && BYTE_ORDER == ORDER_BIG_ENDIAN ) || \
(defined(__BYTE_ORDER ) && defined(__ORDER_BIG_ENDIAN ) && __BYTE_ORDER == __ORDER_BIG_ENDIAN ) || \
(defined(__BYTE_ORDER__) && defined(__ORDER_BIG_ENDIAN__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__))
return a;
#else
return (((uint8_t*)&a)[0] << 24) |
(((uint8_t*)&a)[1] << 16) |
(((uint8_t*)&a)[2] << 8) |
(((uint8_t*)&a)[3] << 0);
#endif
}
static inline uint32_t lfs_tobe32(uint32_t a) {
return lfs_frombe32(a);
}
// Calculate CRC-32 with polynomial = 0x04c11db7
uint32_t lfs_crc(uint32_t crc, const void *buffer, size_t size);
// Allocate memory, only used if buffers are not provided to littlefs
// Note, memory must be 64-bit aligned
static inline void *lfs_malloc(size_t size) {
#ifndef LFS_NO_MALLOC
return malloc(size);
#else
(void)size;
return NULL;
#endif
}
// Deallocate memory, only used if buffers are not provided to littlefs
static inline void lfs_free(void *p) {
#ifndef LFS_NO_MALLOC
free(p);
#else
(void)p;
#endif
}
// Logging functions, these may be replaced by system-specific
// logging functions
#define LFS_DEBUG(fmt, ...) printf("lfs debug: " fmt "\n", __VA_ARGS__)
#define LFS_WARN(fmt, ...) printf("lfs warn: " fmt "\n", __VA_ARGS__)
#define LFS_ERROR(fmt, ...) printf("lfs error: " fmt "\n", __VA_ARGS__)
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif
#endif

44
scripts/corrupt.py Executable file
View File

@@ -0,0 +1,44 @@
#!/usr/bin/env python2
import struct
import sys
import os
import argparse
def corrupt(block):
with open(block, 'r+b') as file:
# skip rev
file.read(4)
# go to last commit
tag = 0xffffffff
while True:
try:
ntag, = struct.unpack('>I', file.read(4))
except struct.error:
break
tag ^= ntag
size = (tag & 0x3ff) if (tag & 0x3ff) != 0x3ff else 0
file.seek(size, os.SEEK_CUR)
# lob off last 3 bytes
file.seek(-(size + 3), os.SEEK_CUR)
file.truncate()
def main(args):
if args.n or not args.blocks:
with open('blocks/.history', 'rb') as file:
for i in range(int(args.n or 1)):
last, = struct.unpack('<I', file.read(4))
args.blocks.append('blocks/%x' % last)
for block in args.blocks:
print 'corrupting %s' % block
corrupt(block)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument('-n')
parser.add_argument('blocks', nargs='*')
main(parser.parse_args())

112
scripts/debug.py Executable file
View File

@@ -0,0 +1,112 @@
#!/usr/bin/env python2
import struct
import binascii
TYPES = {
(0x700, 0x400): 'splice',
(0x7ff, 0x401): 'create',
(0x7ff, 0x4ff): 'delete',
(0x700, 0x000): 'name',
(0x7ff, 0x001): 'name reg',
(0x7ff, 0x002): 'name dir',
(0x7ff, 0x0ff): 'name superblock',
(0x700, 0x200): 'struct',
(0x7ff, 0x200): 'struct dir',
(0x7ff, 0x202): 'struct ctz',
(0x7ff, 0x201): 'struct inline',
(0x700, 0x300): 'userattr',
(0x700, 0x600): 'tail',
(0x7ff, 0x600): 'tail soft',
(0x7ff, 0x601): 'tail hard',
(0x700, 0x700): 'gstate',
(0x7ff, 0x7ff): 'gstate move',
(0x700, 0x500): 'crc',
}
def typeof(type):
for prefix in range(12):
mask = 0x7ff & ~((1 << prefix)-1)
if (mask, type & mask) in TYPES:
return TYPES[mask, type & mask] + (
' %0*x' % (prefix/4, type & ((1 << prefix)-1))
if prefix else '')
else:
return '%02x' % type
def main(*blocks):
# find most recent block
file = None
rev = None
crc = None
versions = []
for block in blocks:
try:
nfile = open(block, 'rb')
ndata = nfile.read(4)
ncrc = binascii.crc32(ndata)
nrev, = struct.unpack('<I', ndata)
assert rev != nrev
if not file or ((rev - nrev) & 0x80000000):
file = nfile
rev = nrev
crc = ncrc
versions.append((nrev, '%s (rev %d)' % (block, nrev)))
except (IOError, struct.error):
pass
if not file:
print 'Bad metadata pair {%s}' % ', '.join(blocks)
return 1
print "--- %s ---" % ', '.join(v for _,v in sorted(versions, reverse=True))
# go through each tag, print useful information
print "%-4s %-8s %-14s %3s %4s %s" % (
'off', 'tag', 'type', 'id', 'len', 'dump')
tag = 0xffffffff
off = 4
while True:
try:
data = file.read(4)
crc = binascii.crc32(data, crc)
ntag, = struct.unpack('>I', data)
except struct.error:
break
tag ^= ntag
off += 4
type = (tag & 0x7ff00000) >> 20
id = (tag & 0x000ffc00) >> 10
size = (tag & 0x000003ff) >> 0
iscrc = (type & 0x700) == 0x500
data = file.read(size if size != 0x3ff else 0)
if iscrc:
crc = binascii.crc32(data[:4], crc)
else:
crc = binascii.crc32(data, crc)
print '%04x: %08x %-15s %3s %4s %-23s %-8s' % (
off, tag,
typeof(type) + (' bad!' if iscrc and ~crc else ''),
hex(id)[2:] if id != 0x3ff else '.',
size if size != 0x3ff else 'x',
' '.join('%02x' % ord(c) for c in data[:8]),
''.join(c if c >= ' ' and c <= '~' else '.' for c in data[:8]))
off += size if size != 0x3ff else 0
if iscrc:
crc = 0
tag ^= (type & 1) << 31
return 0
if __name__ == "__main__":
import sys
sys.exit(main(*sys.argv[1:]))

61
scripts/prefix.py Executable file
View File

@@ -0,0 +1,61 @@
#!/usr/bin/env python2
# This script replaces prefixes of files, and symbols in that file.
# Useful for creating different versions of the codebase that don't
# conflict at compile time.
#
# example:
# $ ./scripts/prefix.py lfs2
import os
import os.path
import re
import glob
import itertools
import tempfile
import shutil
import subprocess
DEFAULT_PREFIX = "lfs"
def subn(from_prefix, to_prefix, name):
name, count1 = re.subn('\\b'+from_prefix, to_prefix, name)
name, count2 = re.subn('\\b'+from_prefix.upper(), to_prefix.upper(), name)
name, count3 = re.subn('\\B-D'+from_prefix.upper(),
'-D'+to_prefix.upper(), name)
return name, count1+count2+count3
def main(from_prefix, to_prefix=None, files=None):
if not to_prefix:
from_prefix, to_prefix = DEFAULT_PREFIX, from_prefix
if not files:
files = subprocess.check_output([
'git', 'ls-tree', '-r', '--name-only', 'HEAD']).split()
for oldname in files:
# Rename any matching file names
newname, namecount = subn(from_prefix, to_prefix, oldname)
if namecount:
subprocess.check_call(['git', 'mv', oldname, newname])
# Rename any prefixes in file
count = 0
with open(newname+'~', 'w') as tempf:
with open(newname) as newf:
for line in newf:
line, n = subn(from_prefix, to_prefix, line)
count += n
tempf.write(line)
shutil.copystat(newname, newname+'~')
os.rename(newname+'~', newname)
subprocess.check_call(['git', 'add', newname])
# Summary
print '%s: %d replacements' % (
'%s -> %s' % (oldname, newname) if namecount else oldname,
count)
if __name__ == "__main__":
import sys
sys.exit(main(*sys.argv[1:]))

28
scripts/results.py Executable file
View File

@@ -0,0 +1,28 @@
#!/usr/bin/env python2
import struct
import sys
import time
import os
import re
def main():
with open('blocks/.config') as file:
read_size, prog_size, block_size, block_count = (
struct.unpack('<LLLL', file.read()))
real_size = sum(
os.path.getsize(os.path.join('blocks', f))
for f in os.listdir('blocks') if re.match('\d+', f))
with open('blocks/.stats') as file:
read_count, prog_count, erase_count = (
struct.unpack('<QQQ', file.read()))
runtime = time.time() - os.stat('blocks').st_ctime
print 'results: %dB %dB %dB %.3fs' % (
read_count, prog_count, erase_count, runtime)
if __name__ == "__main__":
main(*sys.argv[1:])

96
scripts/template.fmt Normal file
View File

@@ -0,0 +1,96 @@
/// AUTOGENERATED TEST ///
#include "lfs.h"
#include "emubd/lfs_emubd.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
// test stuff
static void test_assert(const char *file, unsigned line,
const char *s, uintmax_t v, uintmax_t e) {{
if (v != e) {{
fprintf(stderr, "\033[97m%s:%u: \033[91m"
"assert failed with %jd, expected %jd\033[0m\n"
" %s\n\n", file, line, v, e, s);
exit(-2);
}}
}}
#define test_assert(v, e) \
test_assert(__FILE__, __LINE__, #v " => " #e, v, e)
// implicit variable for asserts
uintmax_t test;
// utility functions for traversals
static int __attribute__((used)) test_count(void *p, lfs_block_t b) {{
(void)b;
unsigned *u = (unsigned*)p;
*u += 1;
return 0;
}}
// lfs declarations
lfs_t lfs;
lfs_emubd_t bd;
// other declarations for convenience
lfs_file_t file;
lfs_dir_t dir;
struct lfs_info info;
uint8_t buffer[1024];
char path[1024];
// test configuration options
#ifndef LFS_READ_SIZE
#define LFS_READ_SIZE 16
#endif
#ifndef LFS_PROG_SIZE
#define LFS_PROG_SIZE LFS_READ_SIZE
#endif
#ifndef LFS_BLOCK_SIZE
#define LFS_BLOCK_SIZE 512
#endif
#ifndef LFS_BLOCK_COUNT
#define LFS_BLOCK_COUNT 1024
#endif
#ifndef LFS_BLOCK_CYCLES
#define LFS_BLOCK_CYCLES 1024
#endif
#ifndef LFS_CACHE_SIZE
#define LFS_CACHE_SIZE (64 % LFS_PROG_SIZE == 0 ? 64 : LFS_PROG_SIZE)
#endif
#ifndef LFS_LOOKAHEAD_SIZE
#define LFS_LOOKAHEAD_SIZE 16
#endif
const struct lfs_config cfg = {{
.context = &bd,
.read = &lfs_emubd_read,
.prog = &lfs_emubd_prog,
.erase = &lfs_emubd_erase,
.sync = &lfs_emubd_sync,
.read_size = LFS_READ_SIZE,
.prog_size = LFS_PROG_SIZE,
.block_size = LFS_BLOCK_SIZE,
.block_count = LFS_BLOCK_COUNT,
.block_cycles = LFS_BLOCK_CYCLES,
.cache_size = LFS_CACHE_SIZE,
.lookahead_size = LFS_LOOKAHEAD_SIZE,
}};
// Entry point
int main(void) {{
lfs_emubd_create(&cfg, "blocks");
{tests}
lfs_emubd_destroy(&cfg);
}}

81
scripts/test.py Executable file
View File

@@ -0,0 +1,81 @@
#!/usr/bin/env python2
import re
import sys
import subprocess
import os
def generate(test):
with open("scripts/template.fmt") as file:
template = file.read()
haslines = 'TEST_LINE' in os.environ and 'TEST_FILE' in os.environ
lines = []
for offset, line in enumerate(
re.split('(?<=(?:.;| [{}]))\n', test.read())):
match = re.match('((?: *\n)*)( *)(.*)=>(.*);',
line, re.DOTALL | re.MULTILINE)
if match:
preface, tab, test, expect = match.groups()
lines.extend(['']*preface.count('\n'))
lines.append(tab+'test_assert({test}, {expect});'.format(
test=test.strip(), expect=expect.strip()))
else:
lines.append(line)
# Create test file
with open('test.c', 'w') as file:
if 'TEST_LINE' in os.environ and 'TEST_FILE' in os.environ:
lines.insert(0, '#line %d "%s"' % (
int(os.environ['TEST_LINE']) + 1,
os.environ['TEST_FILE']))
lines.append('#line %d "test.c"' % (
template[:template.find('{tests}')].count('\n')
+ len(lines) + 2))
file.write(template.format(tests='\n'.join(lines)))
# Remove build artifacts to force rebuild
try:
os.remove('test.o')
os.remove('lfs')
except OSError:
pass
def compile():
subprocess.check_call([
os.environ.get('MAKE', 'make'),
'--no-print-directory', '-s'])
def execute():
if 'EXEC' in os.environ:
subprocess.check_call([os.environ['EXEC'], "./lfs"])
else:
subprocess.check_call(["./lfs"])
def main(test=None):
try:
if test and not test.startswith('-'):
with open(test) as file:
generate(file)
else:
generate(sys.stdin)
compile()
if test == '-s':
sys.exit(1)
execute()
except subprocess.CalledProcessError:
# Python stack trace is counterproductive, just exit
sys.exit(2)
except KeyboardInterrupt:
# Python stack trace is counterproductive, just exit
sys.exit(3)
if __name__ == "__main__":
main(*sys.argv[1:])

View File

@@ -1,30 +0,0 @@
#!/usr/bin/env python
import struct
import sys
import time
import os
import re
def main():
with open('blocks/config') as file:
s = struct.unpack('<LLLL', file.read())
print 'read_size: %d' % s[0]
print 'prog_size: %d' % s[1]
print 'block_size: %d' % s[2]
print 'block_size: %d' % s[3]
print 'real_size: %d' % sum(
os.path.getsize(os.path.join('blocks', f))
for f in os.listdir('blocks') if re.match('\d+', f))
with open('blocks/stats') as file:
s = struct.unpack('<QQQ', file.read())
print 'read_count: %d' % s[0]
print 'prog_count: %d' % s[1]
print 'erase_count: %d' % s[2]
print 'runtime: %.3f' % (time.time() - os.stat('blocks').st_ctime)
if __name__ == "__main__":
main(*sys.argv[1:])

View File

@@ -1,105 +0,0 @@
/// AUTOGENERATED TEST ///
#include "lfs.h"
#include "emubd/lfs_emubd.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
// test stuff
void test_log(const char *s, uintmax_t v) {{
printf("%s: %jd\n", s, v);
}}
void test_assert(const char *file, unsigned line,
const char *s, uintmax_t v, uintmax_t e) {{
static const char *last[6] = {{0, 0}};
if (v != e || !(last[0] == s || last[1] == s ||
last[2] == s || last[3] == s ||
last[4] == s || last[5] == s)) {{
test_log(s, v);
last[0] = last[1];
last[1] = last[2];
last[2] = last[3];
last[3] = last[4];
last[4] = last[5];
last[5] = s;
}}
if (v != e) {{
fprintf(stderr, "\033[31m%s:%u: assert %s failed with %jd, "
"expected %jd\033[0m\n", file, line, s, v, e);
exit(-2);
}}
}}
#define test_assert(s, v, e) test_assert(__FILE__, __LINE__, s, v, e)
// utility functions for traversals
int test_count(void *p, lfs_block_t b) {{
unsigned *u = (unsigned*)p;
*u += 1;
return 0;
}}
// lfs declarations
lfs_t lfs;
lfs_emubd_t bd;
lfs_file_t file[4];
lfs_dir_t dir[4];
struct lfs_info info;
uint8_t buffer[1024];
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
lfs_size_t size;
lfs_size_t wsize;
lfs_size_t rsize;
uintmax_t res;
#ifndef LFS_READ_SIZE
#define LFS_READ_SIZE 16
#endif
#ifndef LFS_PROG_SIZE
#define LFS_PROG_SIZE 16
#endif
#ifndef LFS_BLOCK_SIZE
#define LFS_BLOCK_SIZE 512
#endif
#ifndef LFS_BLOCK_COUNT
#define LFS_BLOCK_COUNT 1024
#endif
#ifndef LFS_LOOKAHEAD
#define LFS_LOOKAHEAD 128
#endif
const struct lfs_config cfg = {{
.context = &bd,
.read = &lfs_emubd_read,
.prog = &lfs_emubd_prog,
.erase = &lfs_emubd_erase,
.sync = &lfs_emubd_sync,
.read_size = LFS_READ_SIZE,
.prog_size = LFS_PROG_SIZE,
.block_size = LFS_BLOCK_SIZE,
.block_count = LFS_BLOCK_COUNT,
.lookahead = LFS_LOOKAHEAD,
}};
// Entry point
int main() {{
lfs_emubd_create(&cfg, "blocks");
{tests}
lfs_emubd_destroy(&cfg);
}}

View File

@@ -1,49 +0,0 @@
#!/usr/bin/env python
import re
import sys
import subprocess
import os
def generate(test):
with open("tests/template.fmt") as file:
template = file.read()
lines = []
for line in re.split('(?<=[;{}])\n', test.read()):
match = re.match('(?: *\n)*( *)(.*)=>(.*);', line, re.DOTALL | re.MULTILINE)
if match:
tab, test, expect = match.groups()
lines.append(tab+'res = {test};'.format(test=test.strip()))
lines.append(tab+'test_assert("{name}", res, {expect});'.format(
name = re.match('\w*', test.strip()).group(),
expect = expect.strip()))
else:
lines.append(line)
with open('test.c', 'w') as file:
file.write(template.format(tests='\n'.join(lines)))
def compile():
os.environ['CFLAGS'] = os.environ.get('CFLAGS', '') + ' -Werror'
subprocess.check_call(['make', '--no-print-directory', '-s'], env=os.environ)
def execute():
subprocess.check_call(["./lfs"])
def main(test=None):
if test and not test.startswith('-'):
with open(test) as file:
generate(file)
else:
generate(sys.stdin)
compile()
if test == '-s':
sys.exit(1)
execute()
if __name__ == "__main__":
main(*sys.argv[1:])

View File

@@ -1,16 +1,18 @@
#!/bin/bash
set -eu
set -euE
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Allocator tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
SIZE=15000
lfs_mkdir() {
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "$1") => 0;
lfs_unmount(&lfs) => 0;
@@ -18,7 +20,7 @@ TEST
}
lfs_remove() {
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "$1/eggs") => 0;
lfs_remove(&lfs, "$1/bacon") => 0;
@@ -29,22 +31,23 @@ TEST
}
lfs_alloc_singleproc() {
tests/test.py << TEST
scripts/test.py << TEST
const char *names[] = {"bacon", "eggs", "pancakes"};
lfs_file_t files[sizeof(names)/sizeof(names[0])];
lfs_mount(&lfs, &cfg) => 0;
for (int n = 0; n < sizeof(names)/sizeof(names[0]); n++) {
sprintf((char*)buffer, "$1/%s", names[n]);
lfs_file_open(&lfs, &file[n], (char*)buffer,
for (unsigned n = 0; n < sizeof(names)/sizeof(names[0]); n++) {
sprintf(path, "$1/%s", names[n]);
lfs_file_open(&lfs, &files[n], path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_APPEND) => 0;
}
for (int n = 0; n < sizeof(names)/sizeof(names[0]); n++) {
size = strlen(names[n]);
for (unsigned n = 0; n < sizeof(names)/sizeof(names[0]); n++) {
lfs_size_t size = strlen(names[n]);
for (int i = 0; i < $SIZE; i++) {
lfs_file_write(&lfs, &file[n], names[n], size) => size;
lfs_file_write(&lfs, &files[n], names[n], size) => size;
}
}
for (int n = 0; n < sizeof(names)/sizeof(names[0]); n++) {
lfs_file_close(&lfs, &file[n]) => 0;
for (unsigned n = 0; n < sizeof(names)/sizeof(names[0]); n++) {
lfs_file_close(&lfs, &files[n]) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
@@ -53,16 +56,16 @@ TEST
lfs_alloc_multiproc() {
for name in bacon eggs pancakes
do
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "$1/$name",
lfs_file_open(&lfs, &file, "$1/$name",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_APPEND) => 0;
size = strlen("$name");
lfs_size_t size = strlen("$name");
memcpy(buffer, "$name", size);
for (int i = 0; i < $SIZE; i++) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
done
@@ -71,15 +74,15 @@ done
lfs_verify() {
for name in bacon eggs pancakes
do
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "$1/$name", LFS_O_RDONLY) => 0;
size = strlen("$name");
lfs_file_open(&lfs, &file, "$1/$name", LFS_O_RDONLY) => 0;
lfs_size_t size = strlen("$name");
for (int i = 0; i < $SIZE; i++) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "$name", size) => 0;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
done
@@ -115,19 +118,19 @@ lfs_remove multiprocreuse
lfs_remove singleprocreuse
echo "--- Exhaustion test ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
size = strlen("exhaustion");
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
lfs_size_t size = strlen("exhaustion");
memcpy(buffer, "exhaustion", size);
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_sync(&lfs, &file[0]) => 0;
lfs_file_write(&lfs, &file, buffer, size) => size;
lfs_file_sync(&lfs, &file) => 0;
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
lfs_ssize_t res;
while (true) {
res = lfs_file_write(&lfs, &file[0], buffer, size);
res = lfs_file_write(&lfs, &file, buffer, size);
if (res < 0) {
break;
}
@@ -136,45 +139,45 @@ tests/test.py << TEST
}
res => LFS_ERR_NOSPC;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_RDONLY);
size = strlen("exhaustion");
lfs_file_size(&lfs, &file[0]) => size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_RDONLY);
lfs_size_t size = strlen("exhaustion");
lfs_file_size(&lfs, &file) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "exhaustion", size) => 0;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Exhaustion wraparound test ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_file_open(&lfs, &file[0], "padding", LFS_O_WRONLY | LFS_O_CREAT);
size = strlen("buffering");
lfs_file_open(&lfs, &file, "padding", LFS_O_WRONLY | LFS_O_CREAT);
lfs_size_t size = strlen("buffering");
memcpy(buffer, "buffering", size);
for (int i = 0; i < $SIZE; i++) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_remove(&lfs, "padding") => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
size = strlen("exhaustion");
memcpy(buffer, "exhaustion", size);
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_sync(&lfs, &file[0]) => 0;
lfs_file_write(&lfs, &file, buffer, size) => size;
lfs_file_sync(&lfs, &file) => 0;
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
lfs_ssize_t res;
while (true) {
res = lfs_file_write(&lfs, &file[0], buffer, size);
res = lfs_file_write(&lfs, &file, buffer, size);
if (res < 0) {
break;
}
@@ -183,89 +186,299 @@ tests/test.py << TEST
}
res => LFS_ERR_NOSPC;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_RDONLY);
size = strlen("exhaustion");
lfs_file_size(&lfs, &file[0]) => size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_RDONLY);
lfs_size_t size = strlen("exhaustion");
lfs_file_size(&lfs, &file) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "exhaustion", size) => 0;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Dir exhaustion test ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
size = strlen("blahblahblahblah");
// find out max file size
lfs_mkdir(&lfs, "exhaustiondir") => 0;
lfs_size_t size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < (cfg.block_count-6)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
int count = 0;
int err;
while (true) {
err = lfs_file_write(&lfs, &file, buffer, size);
if (err < 0) {
break;
}
count += 1;
}
lfs_file_close(&lfs, &file[0]) => 0;
err => LFS_ERR_NOSPC;
lfs_file_close(&lfs, &file) => 0;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_remove(&lfs, "exhaustiondir") => 0;
// see if dir fits with max file size
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
for (int i = 0; i < count; i++) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
lfs_mkdir(&lfs, "exhaustiondir") => 0;
lfs_remove(&lfs, "exhaustiondir") => 0;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_WRONLY | LFS_O_APPEND);
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < (cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
// see if dir fits with > max file size
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
for (int i = 0; i < count+1; i++) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_mkdir(&lfs, "exhaustiondir") => LFS_ERR_NOSPC;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Chained dir exhaustion test ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < (cfg.block_count-24)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
// find out max file size
lfs_mkdir(&lfs, "exhaustiondir") => 0;
for (int i = 0; i < 10; i++) {
sprintf(path, "dirwithanexhaustivelylongnameforpadding%d", i);
lfs_mkdir(&lfs, path) => 0;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_size_t size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
int count = 0;
int err;
while (true) {
err = lfs_file_write(&lfs, &file, buffer, size);
if (err < 0) {
break;
}
for (int i = 0; i < 9; i++) {
sprintf((char*)buffer, "dirwithanexhaustivelylongnameforpadding%d", i);
lfs_mkdir(&lfs, (char*)buffer) => 0;
count += 1;
}
err => LFS_ERR_NOSPC;
lfs_file_close(&lfs, &file) => 0;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_remove(&lfs, "exhaustiondir") => 0;
for (int i = 0; i < 10; i++) {
sprintf(path, "dirwithanexhaustivelylongnameforpadding%d", i);
lfs_remove(&lfs, path) => 0;
}
// see that chained dir fails
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
for (int i = 0; i < count+1; i++) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_sync(&lfs, &file) => 0;
for (int i = 0; i < 10; i++) {
sprintf(path, "dirwithanexhaustivelylongnameforpadding%d", i);
lfs_mkdir(&lfs, path) => 0;
}
lfs_mkdir(&lfs, "exhaustiondir") => LFS_ERR_NOSPC;
lfs_remove(&lfs, "exhaustion") => 0;
lfs_file_open(&lfs, &file[0], "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
// shorten file to try a second chained dir
while (true) {
err = lfs_mkdir(&lfs, "exhaustiondir");
if (err != LFS_ERR_NOSPC) {
break;
}
lfs_ssize_t filesize = lfs_file_size(&lfs, &file);
filesize > 0 => true;
lfs_file_truncate(&lfs, &file, filesize - size) => 0;
lfs_file_sync(&lfs, &file) => 0;
}
err => 0;
lfs_mkdir(&lfs, "exhaustiondir2") => LFS_ERR_NOSPC;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Split dir test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
// create one block hole for half a directory
lfs_file_open(&lfs, &file, "bump", LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (lfs_size_t i = 0; i < cfg.block_size; i += 2) {
memcpy(&buffer[i], "hi", 2);
}
lfs_file_write(&lfs, &file, buffer, cfg.block_size) => cfg.block_size;
lfs_file_close(&lfs, &file) => 0;
lfs_file_open(&lfs, &file, "exhaustion", LFS_O_WRONLY | LFS_O_CREAT);
lfs_size_t size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < (cfg.block_count-4)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
// remount to force reset of lookahead
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, &cfg) => 0;
// open hole
lfs_remove(&lfs, "bump") => 0;
lfs_mkdir(&lfs, "splitdir") => 0;
lfs_file_open(&lfs, &file, "splitdir/bump",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (lfs_size_t i = 0; i < cfg.block_size; i += 2) {
memcpy(&buffer[i], "hi", 2);
}
lfs_file_write(&lfs, &file, buffer, 2*cfg.block_size) => LFS_ERR_NOSPC;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Outdated lookahead test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
// fill completely with two files
lfs_file_open(&lfs, &file, "exhaustion1",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_size_t size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < ((cfg.block_count-2)/2)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
lfs_file_open(&lfs, &file, "exhaustion2",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < (cfg.block_count-26)*(cfg.block_size-8);
i < ((cfg.block_count-2+1)/2)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_mkdir(&lfs, "exhaustiondir") => 0;
lfs_mkdir(&lfs, "exhaustiondir2") => LFS_ERR_NOSPC;
// remount to force reset of lookahead
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, &cfg) => 0;
// rewrite one file
lfs_file_open(&lfs, &file, "exhaustion1",
LFS_O_WRONLY | LFS_O_TRUNC) => 0;
lfs_file_sync(&lfs, &file) => 0;
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < ((cfg.block_count-2)/2)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
// rewrite second file, this requires lookahead does not
// use old population
lfs_file_open(&lfs, &file, "exhaustion2",
LFS_O_WRONLY | LFS_O_TRUNC) => 0;
lfs_file_sync(&lfs, &file) => 0;
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < ((cfg.block_count-2+1)/2)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
TEST
echo "--- Outdated lookahead and split dir test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
echo "--- Results ---"
tests/stats.py
lfs_mount(&lfs, &cfg) => 0;
// fill completely with two files
lfs_file_open(&lfs, &file, "exhaustion1",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_size_t size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < ((cfg.block_count-2)/2)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
lfs_file_open(&lfs, &file, "exhaustion2",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < ((cfg.block_count-2+1)/2)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
// remount to force reset of lookahead
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, &cfg) => 0;
// rewrite one file with a hole of one block
lfs_file_open(&lfs, &file, "exhaustion1",
LFS_O_WRONLY | LFS_O_TRUNC) => 0;
lfs_file_sync(&lfs, &file) => 0;
size = strlen("blahblahblahblah");
memcpy(buffer, "blahblahblahblah", size);
for (lfs_size_t i = 0;
i < ((cfg.block_count-2)/2 - 1)*(cfg.block_size-8);
i += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file) => 0;
// try to allocate a directory, should fail!
lfs_mkdir(&lfs, "split") => LFS_ERR_NOSPC;
// file should not fail
lfs_file_open(&lfs, &file, "notasplit",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_write(&lfs, &file, "hi", 2) => 2;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/results.py

294
tests/test_attrs.sh Executable file
View File

@@ -0,0 +1,294 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Attr tests ==="
rm -rf blocks
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "hello") => 0;
lfs_file_open(&lfs, &file, "hello/hello",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_write(&lfs, &file, "hello", strlen("hello"))
=> strlen("hello");
lfs_file_close(&lfs, &file);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Set/get attribute ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(buffer, 0, sizeof(buffer));
lfs_setattr(&lfs, "hello", 'A', "aaaa", 4) => 0;
lfs_setattr(&lfs, "hello", 'B', "bbbbbb", 6) => 0;
lfs_setattr(&lfs, "hello", 'C', "ccccc", 5) => 0;
lfs_getattr(&lfs, "hello", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "hello", 'B', buffer+4, 6) => 6;
lfs_getattr(&lfs, "hello", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "bbbbbb", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "hello", 'B', "", 0) => 0;
lfs_getattr(&lfs, "hello", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "hello", 'B', buffer+4, 6) => 0;
lfs_getattr(&lfs, "hello", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "\0\0\0\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_removeattr(&lfs, "hello", 'B') => 0;
lfs_getattr(&lfs, "hello", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "hello", 'B', buffer+4, 6) => LFS_ERR_NOATTR;
lfs_getattr(&lfs, "hello", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "\0\0\0\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "hello", 'B', "dddddd", 6) => 0;
lfs_getattr(&lfs, "hello", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "hello", 'B', buffer+4, 6) => 6;
lfs_getattr(&lfs, "hello", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "dddddd", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "hello", 'B', "eee", 3) => 0;
lfs_getattr(&lfs, "hello", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "hello", 'B', buffer+4, 6) => 3;
lfs_getattr(&lfs, "hello", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "eee\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "hello", 'A', buffer, LFS_ATTR_MAX+1) => LFS_ERR_NOSPC;
lfs_setattr(&lfs, "hello", 'B', "fffffffff", 9) => 0;
lfs_getattr(&lfs, "hello", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "hello", 'B', buffer+4, 6) => 9;
lfs_getattr(&lfs, "hello", 'C', buffer+10, 5) => 5;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(buffer, 0, sizeof(buffer));
lfs_getattr(&lfs, "hello", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "hello", 'B', buffer+4, 9) => 9;
lfs_getattr(&lfs, "hello", 'C', buffer+13, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "fffffffff", 9) => 0;
memcmp(buffer+13, "ccccc", 5) => 0;
lfs_file_open(&lfs, &file, "hello/hello", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, buffer, sizeof(buffer)) => strlen("hello");
memcmp(buffer, "hello", strlen("hello")) => 0;
lfs_file_close(&lfs, &file);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Set/get root attribute ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(buffer, 0, sizeof(buffer));
lfs_setattr(&lfs, "/", 'A', "aaaa", 4) => 0;
lfs_setattr(&lfs, "/", 'B', "bbbbbb", 6) => 0;
lfs_setattr(&lfs, "/", 'C', "ccccc", 5) => 0;
lfs_getattr(&lfs, "/", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "/", 'B', buffer+4, 6) => 6;
lfs_getattr(&lfs, "/", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "bbbbbb", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "/", 'B', "", 0) => 0;
lfs_getattr(&lfs, "/", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "/", 'B', buffer+4, 6) => 0;
lfs_getattr(&lfs, "/", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "\0\0\0\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_removeattr(&lfs, "/", 'B') => 0;
lfs_getattr(&lfs, "/", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "/", 'B', buffer+4, 6) => LFS_ERR_NOATTR;
lfs_getattr(&lfs, "/", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "\0\0\0\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "/", 'B', "dddddd", 6) => 0;
lfs_getattr(&lfs, "/", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "/", 'B', buffer+4, 6) => 6;
lfs_getattr(&lfs, "/", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "dddddd", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "/", 'B', "eee", 3) => 0;
lfs_getattr(&lfs, "/", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "/", 'B', buffer+4, 6) => 3;
lfs_getattr(&lfs, "/", 'C', buffer+10, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "eee\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
lfs_setattr(&lfs, "/", 'A', buffer, LFS_ATTR_MAX+1) => LFS_ERR_NOSPC;
lfs_setattr(&lfs, "/", 'B', "fffffffff", 9) => 0;
lfs_getattr(&lfs, "/", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "/", 'B', buffer+4, 6) => 9;
lfs_getattr(&lfs, "/", 'C', buffer+10, 5) => 5;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(buffer, 0, sizeof(buffer));
lfs_getattr(&lfs, "/", 'A', buffer, 4) => 4;
lfs_getattr(&lfs, "/", 'B', buffer+4, 9) => 9;
lfs_getattr(&lfs, "/", 'C', buffer+13, 5) => 5;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "fffffffff", 9) => 0;
memcmp(buffer+13, "ccccc", 5) => 0;
lfs_file_open(&lfs, &file, "hello/hello", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, buffer, sizeof(buffer)) => strlen("hello");
memcmp(buffer, "hello", strlen("hello")) => 0;
lfs_file_close(&lfs, &file);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Set/get file attribute ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(buffer, 0, sizeof(buffer));
struct lfs_attr attrs1[] = {
{'A', buffer, 4},
{'B', buffer+4, 6},
{'C', buffer+10, 5},
};
struct lfs_file_config cfg1 = {.attrs=attrs1, .attr_count=3};
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_WRONLY, &cfg1) => 0;
memcpy(buffer, "aaaa", 4);
memcpy(buffer+4, "bbbbbb", 6);
memcpy(buffer+10, "ccccc", 5);
lfs_file_close(&lfs, &file) => 0;
memset(buffer, 0, 15);
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_RDONLY, &cfg1) => 0;
lfs_file_close(&lfs, &file) => 0;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "bbbbbb", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
attrs1[1].size = 0;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_WRONLY, &cfg1) => 0;
lfs_file_close(&lfs, &file) => 0;
memset(buffer, 0, 15);
attrs1[1].size = 6;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_RDONLY, &cfg1) => 0;
lfs_file_close(&lfs, &file) => 0;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "\0\0\0\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
attrs1[1].size = 6;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_WRONLY, &cfg1) => 0;
memcpy(buffer+4, "dddddd", 6);
lfs_file_close(&lfs, &file) => 0;
memset(buffer, 0, 15);
attrs1[1].size = 6;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_RDONLY, &cfg1) => 0;
lfs_file_close(&lfs, &file) => 0;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "dddddd", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
attrs1[1].size = 3;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_WRONLY, &cfg1) => 0;
memcpy(buffer+4, "eee", 3);
lfs_file_close(&lfs, &file) => 0;
memset(buffer, 0, 15);
attrs1[1].size = 6;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_RDONLY, &cfg1) => 0;
lfs_file_close(&lfs, &file) => 0;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "eee\0\0\0", 6) => 0;
memcmp(buffer+10, "ccccc", 5) => 0;
attrs1[0].size = LFS_ATTR_MAX+1;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_WRONLY, &cfg1)
=> LFS_ERR_NOSPC;
struct lfs_attr attrs2[] = {
{'A', buffer, 4},
{'B', buffer+4, 9},
{'C', buffer+13, 5},
};
struct lfs_file_config cfg2 = {.attrs=attrs2, .attr_count=3};
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_RDWR, &cfg2) => 0;
memcpy(buffer+4, "fffffffff", 9);
lfs_file_close(&lfs, &file) => 0;
attrs1[0].size = 4;
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_RDONLY, &cfg1) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(buffer, 0, sizeof(buffer));
struct lfs_attr attrs2[] = {
{'A', buffer, 4},
{'B', buffer+4, 9},
{'C', buffer+13, 5},
};
struct lfs_file_config cfg2 = {.attrs=attrs2, .attr_count=3};
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_RDONLY, &cfg2) => 0;
lfs_file_close(&lfs, &file) => 0;
memcmp(buffer, "aaaa", 4) => 0;
memcmp(buffer+4, "fffffffff", 9) => 0;
memcmp(buffer+13, "ccccc", 5) => 0;
lfs_file_open(&lfs, &file, "hello/hello", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, buffer, sizeof(buffer)) => strlen("hello");
memcmp(buffer, "hello", strlen("hello")) => 0;
lfs_file_close(&lfs, &file);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Deferred file attributes ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(buffer, 0, sizeof(buffer));
struct lfs_attr attrs1[] = {
{'B', "gggg", 4},
{'C', "", 0},
{'D', "hhhh", 4},
};
struct lfs_file_config cfg1 = {.attrs=attrs1, .attr_count=3};
lfs_file_opencfg(&lfs, &file, "hello/hello", LFS_O_WRONLY, &cfg1) => 0;
lfs_getattr(&lfs, "hello/hello", 'B', buffer, 9) => 9;
lfs_getattr(&lfs, "hello/hello", 'C', buffer+9, 9) => 5;
lfs_getattr(&lfs, "hello/hello", 'D', buffer+18, 9) => LFS_ERR_NOATTR;
memcmp(buffer, "fffffffff", 9) => 0;
memcmp(buffer+9, "ccccc\0\0\0\0", 9) => 0;
memcmp(buffer+18, "\0\0\0\0\0\0\0\0\0", 9) => 0;
lfs_file_sync(&lfs, &file) => 0;
lfs_getattr(&lfs, "hello/hello", 'B', buffer, 9) => 4;
lfs_getattr(&lfs, "hello/hello", 'C', buffer+9, 9) => 0;
lfs_getattr(&lfs, "hello/hello", 'D', buffer+18, 9) => 4;
memcmp(buffer, "gggg\0\0\0\0\0", 9) => 0;
memcmp(buffer+9, "\0\0\0\0\0\0\0\0\0", 9) => 0;
memcmp(buffer+18, "hhhh\0\0\0\0\0", 9) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/results.py

View File

@@ -1,5 +1,7 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Corrupt tests ==="
@@ -7,7 +9,7 @@ NAMEMULT=64
FILEMULT=1
lfs_mktree() {
tests/test.py ${1:-} << TEST
scripts/test.py ${1:-} << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -23,22 +25,22 @@ tests/test.py ${1:-} << TEST
buffer[j+$NAMEMULT+1] = '0'+i;
}
buffer[2*$NAMEMULT+1] = '\0';
lfs_file_open(&lfs, &file[0], (char*)buffer,
lfs_file_open(&lfs, &file, (char*)buffer,
LFS_O_WRONLY | LFS_O_CREAT) => 0;
size = $NAMEMULT;
lfs_size_t size = $NAMEMULT;
for (int j = 0; j < i*$FILEMULT; j++) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
}
lfs_chktree() {
tests/test.py ${1:-} << TEST
scripts/test.py ${1:-} << TEST
lfs_mount(&lfs, &cfg) => 0;
for (int i = 1; i < 10; i++) {
for (int j = 0; j < $NAMEMULT; j++) {
@@ -53,15 +55,16 @@ tests/test.py ${1:-} << TEST
buffer[j+$NAMEMULT+1] = '0'+i;
}
buffer[2*$NAMEMULT+1] = '\0';
lfs_file_open(&lfs, &file[0], (char*)buffer, LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file, (char*)buffer, LFS_O_RDONLY) => 0;
size = $NAMEMULT;
lfs_size_t size = $NAMEMULT;
for (int j = 0; j < i*$FILEMULT; j++) {
lfs_file_read(&lfs, &file[0], rbuffer, size) => size;
uint8_t rbuffer[1024];
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(buffer, rbuffer, size) => 0;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
@@ -71,24 +74,25 @@ echo "--- Sanity check ---"
rm -rf blocks
lfs_mktree
lfs_chktree
BLOCKS="$(ls blocks | grep -vw '[01]')"
echo "--- Block corruption ---"
for i in {0..33}
for b in $BLOCKS
do
rm -rf blocks
mkdir blocks
ln -s /dev/zero blocks/$(printf '%x' $i)
ln -s /dev/zero blocks/$b
lfs_mktree
lfs_chktree
done
echo "--- Block persistance ---"
for i in {0..33}
for b in $BLOCKS
do
rm -rf blocks
mkdir blocks
lfs_mktree
chmod a-w blocks/$(printf '%x' $i)
chmod a-w blocks/$b || true
lfs_mktree
lfs_chktree
done
@@ -96,7 +100,7 @@ done
echo "--- Big region corruption ---"
rm -rf blocks
mkdir blocks
for i in {2..255}
for i in {2..512}
do
ln -s /dev/zero blocks/$(printf '%x' $i)
done
@@ -106,12 +110,11 @@ lfs_chktree
echo "--- Alternating corruption ---"
rm -rf blocks
mkdir blocks
for i in {2..511..2}
for i in {2..1024..2}
do
ln -s /dev/zero blocks/$(printf '%x' $i)
done
lfs_mktree
lfs_chktree
echo "--- Results ---"
tests/stats.py
scripts/results.py

View File

@@ -1,187 +1,191 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Directory tests ==="
LARGESIZE=128
echo "=== Directory tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
echo "--- Root directory ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Directory creation ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "potato") => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- File creation ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "burito", LFS_O_CREAT | LFS_O_WRONLY) => 0;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_open(&lfs, &file, "burito", LFS_O_CREAT | LFS_O_WRONLY) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Directory iteration ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "potato") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "burito") => 0;
info.type => LFS_TYPE_REG;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "potato") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Directory failures ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "potato") => LFS_ERR_EXIST;
lfs_dir_open(&lfs, &dir[0], "tomato") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir[0], "burito") => LFS_ERR_NOTDIR;
lfs_file_open(&lfs, &file[0], "tomato", LFS_O_RDONLY) => LFS_ERR_NOENT;
lfs_file_open(&lfs, &file[0], "potato", LFS_O_RDONLY) => LFS_ERR_ISDIR;
lfs_dir_open(&lfs, &dir, "tomato") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "burito") => LFS_ERR_NOTDIR;
lfs_file_open(&lfs, &file, "tomato", LFS_O_RDONLY) => LFS_ERR_NOENT;
lfs_file_open(&lfs, &file, "potato", LFS_O_RDONLY) => LFS_ERR_ISDIR;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Nested directories ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "potato/baked") => 0;
lfs_mkdir(&lfs, "potato/sweet") => 0;
lfs_mkdir(&lfs, "potato/fried") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "potato") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "potato") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "baked") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "fried") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Multi-block directory ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "cactus") => 0;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf((char*)buffer, "cactus/test%d", i);
lfs_mkdir(&lfs, (char*)buffer) => 0;
sprintf(path, "cactus/test%03d", i);
lfs_mkdir(&lfs, path) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "cactus") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "cactus") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf((char*)buffer, "test%d", i);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
sprintf(path, "test%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
info.type => LFS_TYPE_DIR;
}
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Directory remove ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "potato") => LFS_ERR_NOTEMPTY;
lfs_remove(&lfs, "potato/sweet") => 0;
lfs_remove(&lfs, "potato/baked") => 0;
lfs_remove(&lfs, "potato/fried") => 0;
lfs_dir_open(&lfs, &dir[0], "potato") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "potato") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_remove(&lfs, "potato") => 0;
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "burito") => 0;
info.type => LFS_TYPE_REG;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "cactus") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "burito") => 0;
info.type => LFS_TYPE_REG;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "cactus") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Directory rename ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "coldpotato") => 0;
lfs_mkdir(&lfs, "coldpotato/baked") => 0;
@@ -189,67 +193,67 @@ tests/test.py << TEST
lfs_mkdir(&lfs, "coldpotato/fried") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "coldpotato", "hotpotato") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "hotpotato") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "hotpotato") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "baked") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "fried") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "warmpotato") => 0;
lfs_mkdir(&lfs, "warmpotato/mushy") => 0;
lfs_rename(&lfs, "hotpotato", "warmpotato") => LFS_ERR_INVAL;
lfs_rename(&lfs, "hotpotato", "warmpotato") => LFS_ERR_NOTEMPTY;
lfs_remove(&lfs, "warmpotato/mushy") => 0;
lfs_rename(&lfs, "hotpotato", "warmpotato") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "warmpotato") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "warmpotato") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "baked") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "fried") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "coldpotato") => 0;
lfs_rename(&lfs, "warmpotato/baked", "coldpotato/baked") => 0;
@@ -259,101 +263,227 @@ tests/test.py << TEST
lfs_remove(&lfs, "warmpotato") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "coldpotato") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "coldpotato") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "baked") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "fried") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "sweet") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Recursive remove ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "coldpotato") => LFS_ERR_NOTEMPTY;
lfs_dir_open(&lfs, &dir[0], "coldpotato") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "coldpotato") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
while (true) {
int err = lfs_dir_read(&lfs, &dir[0], &info);
int err = lfs_dir_read(&lfs, &dir, &info);
err >= 0 => 1;
if (err == 0) {
break;
}
strcpy((char*)buffer, "coldpotato/");
strcat((char*)buffer, info.name);
lfs_remove(&lfs, (char*)buffer) => 0;
strcpy(path, "coldpotato/");
strcat(path, info.name);
lfs_remove(&lfs, path) => 0;
}
lfs_remove(&lfs, "coldpotato") => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "burito") => 0;
info.type => LFS_TYPE_REG;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "cactus") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Multi-block rename ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
for (int i = 0; i < $LARGESIZE; i++) {
char oldpath[1024];
char newpath[1024];
sprintf(oldpath, "cactus/test%03d", i);
sprintf(newpath, "cactus/tedd%03d", i);
lfs_rename(&lfs, oldpath, newpath) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir, "cactus") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf(path, "tedd%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
info.type => LFS_TYPE_DIR;
}
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Multi-block remove ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "cactus") => LFS_ERR_NOTEMPTY;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf((char*)buffer, "cactus/test%d", i);
lfs_remove(&lfs, (char*)buffer) => 0;
sprintf(path, "cactus/tedd%03d", i);
lfs_remove(&lfs, path) => 0;
}
lfs_remove(&lfs, "cactus") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "burito") => 0;
info.type => LFS_TYPE_REG;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py
echo "--- Multi-block directory with files ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "prickly-pear") => 0;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf(path, "prickly-pear/test%03d", i);
lfs_file_open(&lfs, &file, path, LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_size_t size = 6;
memcpy(buffer, "Hello", size);
lfs_file_write(&lfs, &file, buffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir, "prickly-pear") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf(path, "test%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
info.type => LFS_TYPE_REG;
info.size => 6;
}
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Multi-block rename with files ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
for (int i = 0; i < $LARGESIZE; i++) {
char oldpath[1024];
char newpath[1024];
sprintf(oldpath, "prickly-pear/test%03d", i);
sprintf(newpath, "prickly-pear/tedd%03d", i);
lfs_rename(&lfs, oldpath, newpath) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir, "prickly-pear") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf(path, "tedd%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
info.type => LFS_TYPE_REG;
info.size => 6;
}
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Multi-block remove with files ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "prickly-pear") => LFS_ERR_NOTEMPTY;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf(path, "prickly-pear/tedd%03d", i);
lfs_remove(&lfs, path) => 0;
}
lfs_remove(&lfs, "prickly-pear") => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "burito") => 0;
info.type => LFS_TYPE_REG;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/results.py

251
tests/test_entries.sh Executable file
View File

@@ -0,0 +1,251 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Entry tests ==="
# Note: These tests are intended for 512 byte inline size at different
# inline sizes they should still pass, but won't be testing anything
rm -rf blocks
function read_file {
cat << TEST
size = $2;
lfs_file_open(&lfs, &file, "$1", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(rbuffer, wbuffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
TEST
}
function write_file {
cat << TEST
size = $2;
lfs_file_open(&lfs, &file, "$1",
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC) => 0;
memset(wbuffer, 'c', size);
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
TEST
}
echo "--- Entry grow test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
lfs_size_t size;
lfs_mount(&lfs, &cfg) => 0;
$(write_file "hi0" 20)
$(write_file "hi1" 20)
$(write_file "hi2" 20)
$(write_file "hi3" 20)
$(read_file "hi1" 20)
$(write_file "hi1" 200)
$(read_file "hi0" 20)
$(read_file "hi1" 200)
$(read_file "hi2" 20)
$(read_file "hi3" 20)
lfs_unmount(&lfs) => 0;
TEST
echo "--- Entry shrink test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
lfs_size_t size;
lfs_mount(&lfs, &cfg) => 0;
$(write_file "hi0" 20)
$(write_file "hi1" 200)
$(write_file "hi2" 20)
$(write_file "hi3" 20)
$(read_file "hi1" 200)
$(write_file "hi1" 20)
$(read_file "hi0" 20)
$(read_file "hi1" 20)
$(read_file "hi2" 20)
$(read_file "hi3" 20)
lfs_unmount(&lfs) => 0;
TEST
echo "--- Entry spill test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
lfs_size_t size;
lfs_mount(&lfs, &cfg) => 0;
$(write_file "hi0" 200)
$(write_file "hi1" 200)
$(write_file "hi2" 200)
$(write_file "hi3" 200)
$(read_file "hi0" 200)
$(read_file "hi1" 200)
$(read_file "hi2" 200)
$(read_file "hi3" 200)
lfs_unmount(&lfs) => 0;
TEST
echo "--- Entry push spill test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
lfs_size_t size;
lfs_mount(&lfs, &cfg) => 0;
$(write_file "hi0" 200)
$(write_file "hi1" 20)
$(write_file "hi2" 200)
$(write_file "hi3" 200)
$(read_file "hi1" 20)
$(write_file "hi1" 200)
$(read_file "hi0" 200)
$(read_file "hi1" 200)
$(read_file "hi2" 200)
$(read_file "hi3" 200)
lfs_unmount(&lfs) => 0;
TEST
echo "--- Entry push spill two test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
lfs_size_t size;
lfs_mount(&lfs, &cfg) => 0;
$(write_file "hi0" 200)
$(write_file "hi1" 20)
$(write_file "hi2" 200)
$(write_file "hi3" 200)
$(write_file "hi4" 200)
$(read_file "hi1" 20)
$(write_file "hi1" 200)
$(read_file "hi0" 200)
$(read_file "hi1" 200)
$(read_file "hi2" 200)
$(read_file "hi3" 200)
$(read_file "hi4" 200)
lfs_unmount(&lfs) => 0;
TEST
echo "--- Entry drop test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
lfs_size_t size;
lfs_mount(&lfs, &cfg) => 0;
$(write_file "hi0" 200)
$(write_file "hi1" 200)
$(write_file "hi2" 200)
$(write_file "hi3" 200)
lfs_remove(&lfs, "hi1") => 0;
lfs_stat(&lfs, "hi1", &info) => LFS_ERR_NOENT;
$(read_file "hi0" 200)
$(read_file "hi2" 200)
$(read_file "hi3" 200)
lfs_remove(&lfs, "hi2") => 0;
lfs_stat(&lfs, "hi2", &info) => LFS_ERR_NOENT;
$(read_file "hi0" 200)
$(read_file "hi3" 200)
lfs_remove(&lfs, "hi3") => 0;
lfs_stat(&lfs, "hi3", &info) => LFS_ERR_NOENT;
$(read_file "hi0" 200)
lfs_remove(&lfs, "hi0") => 0;
lfs_stat(&lfs, "hi0", &info) => LFS_ERR_NOENT;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Create too big ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
memset(path, 'm', 200);
path[200] = '\0';
lfs_size_t size = 400;
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC) => 0;
uint8_t wbuffer[1024];
memset(wbuffer, 'c', size);
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
size = 400;
lfs_file_open(&lfs, &file, path, LFS_O_RDONLY) => 0;
uint8_t rbuffer[1024];
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(rbuffer, wbuffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Resize too big ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
memset(path, 'm', 200);
path[200] = '\0';
lfs_size_t size = 40;
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC) => 0;
uint8_t wbuffer[1024];
memset(wbuffer, 'c', size);
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
size = 40;
lfs_file_open(&lfs, &file, path, LFS_O_RDONLY) => 0;
uint8_t rbuffer[1024];
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(rbuffer, wbuffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
size = 400;
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC) => 0;
memset(wbuffer, 'c', size);
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
size = 400;
lfs_file_open(&lfs, &file, path, LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(rbuffer, wbuffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/results.py

View File

@@ -1,55 +1,60 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== File tests ==="
SMALLSIZE=32
MEDIUMSIZE=8192
LARGESIZE=262144
echo "=== File tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
echo "--- Simple file test ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "hello", LFS_O_WRONLY | LFS_O_CREAT) => 0;
size = strlen("Hello World!\n");
lfs_file_open(&lfs, &file, "hello", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_size_t size = strlen("Hello World!\n");
uint8_t wbuffer[1024];
memcpy(wbuffer, "Hello World!\n", size);
lfs_file_write(&lfs, &file[0], wbuffer, size) => size;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
lfs_file_open(&lfs, &file[0], "hello", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file, "hello", LFS_O_RDONLY) => 0;
size = strlen("Hello World!\n");
lfs_file_read(&lfs, &file[0], rbuffer, size) => size;
uint8_t rbuffer[1024];
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(rbuffer, wbuffer, size) => 0;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
w_test() {
tests/test.py << TEST
scripts/test.py ${4:-} << TEST
lfs_size_t size = $1;
lfs_size_t chunk = 31;
srand(0);
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "$2",
lfs_file_open(&lfs, &file, "$2",
${3:-LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC}) => 0;
for (lfs_size_t i = 0; i < size; i += chunk) {
chunk = (chunk < size - i) ? chunk : size - i;
for (lfs_size_t b = 0; b < chunk; b++) {
buffer[b] = rand() & 0xff;
}
lfs_file_write(&lfs, &file[0], buffer, chunk) => chunk;
lfs_file_write(&lfs, &file, buffer, chunk) => chunk;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
}
r_test() {
tests/test.py << TEST
scripts/test.py << TEST
lfs_size_t size = $1;
lfs_size_t chunk = 29;
srand(0);
@@ -57,15 +62,15 @@ tests/test.py << TEST
lfs_stat(&lfs, "$2", &info) => 0;
info.type => LFS_TYPE_REG;
info.size => size;
lfs_file_open(&lfs, &file[0], "$2", ${3:-LFS_O_RDONLY}) => 0;
lfs_file_open(&lfs, &file, "$2", ${3:-LFS_O_RDONLY}) => 0;
for (lfs_size_t i = 0; i < size; i += chunk) {
chunk = (chunk < size - i) ? chunk : size - i;
lfs_file_read(&lfs, &file[0], buffer, chunk) => chunk;
lfs_file_read(&lfs, &file, buffer, chunk) => chunk;
for (lfs_size_t b = 0; b < chunk && i+b < size; b++) {
buffer[b] => rand() & 0xff;
}
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
}
@@ -105,35 +110,112 @@ r_test $LARGESIZE largeavacado
r_test 0 noavacado
echo "--- Dir check ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hello") => 0;
info.type => LFS_TYPE_REG;
info.size => strlen("Hello World!\n");
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "smallavacado") => 0;
info.type => LFS_TYPE_REG;
info.size => $SMALLSIZE;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "mediumavacado") => 0;
info.type => LFS_TYPE_REG;
info.size => $MEDIUMSIZE;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "largeavacado") => 0;
info.type => LFS_TYPE_REG;
info.size => $LARGESIZE;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "mediumavacado") => 0;
info.type => LFS_TYPE_REG;
info.size => $MEDIUMSIZE;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "noavacado") => 0;
info.type => LFS_TYPE_REG;
info.size => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "smallavacado") => 0;
info.type => LFS_TYPE_REG;
info.size => $SMALLSIZE;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py
echo "--- Many files test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
scripts/test.py << TEST
// Create 300 files of 7 bytes
lfs_mount(&lfs, &cfg) => 0;
for (unsigned i = 0; i < 300; i++) {
sprintf(path, "file_%03d", i);
lfs_file_open(&lfs, &file, path,
LFS_O_RDWR | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfs_size_t size = 7;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
snprintf((char*)wbuffer, size, "Hi %03d", i);
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_rewind(&lfs, &file) => 0;
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(wbuffer, rbuffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
echo "--- Many files with flush test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
scripts/test.py << TEST
// Create 300 files of 7 bytes
lfs_mount(&lfs, &cfg) => 0;
for (unsigned i = 0; i < 300; i++) {
sprintf(path, "file_%03d", i);
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfs_size_t size = 7;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
snprintf((char*)wbuffer, size, "Hi %03d", i);
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
lfs_file_open(&lfs, &file, path, LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(wbuffer, rbuffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
echo "--- Many files with power cycle test ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
scripts/test.py << TEST
// Create 300 files of 7 bytes
lfs_mount(&lfs, &cfg) => 0;
for (unsigned i = 0; i < 300; i++) {
sprintf(path, "file_%03d", i);
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_EXCL) => 0;
lfs_size_t size = 7;
uint8_t wbuffer[1024];
uint8_t rbuffer[1024];
snprintf((char*)wbuffer, size, "Hi %03d", i);
lfs_file_write(&lfs, &file, wbuffer, size) => size;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, path, LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, rbuffer, size) => size;
memcmp(wbuffer, rbuffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
scripts/results.py

View File

@@ -1,49 +1,51 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Formatting tests ==="
rm -rf blocks
echo "--- Basic formatting ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
echo "--- Basic mounting ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Invalid superblocks ---"
ln -f -s /dev/zero blocks/0
ln -f -s /dev/zero blocks/1
tests/test.py << TEST
lfs_format(&lfs, &cfg) => LFS_ERR_CORRUPT;
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => LFS_ERR_NOSPC;
TEST
rm blocks/0 blocks/1
echo "--- Basic mounting ---"
tests/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
tests/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Invalid mount ---"
tests/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
rm blocks/0 blocks/1
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => LFS_ERR_CORRUPT;
TEST
echo "--- Valid corrupt mount ---"
tests/test.py << TEST
echo "--- Expanding superblock ---"
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
rm blocks/0
tests/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
for (int i = 0; i < 100; i++) {
lfs_mkdir(&lfs, "dummy") => 0;
lfs_remove(&lfs, "dummy") => 0;
}
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "dummy") => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py
scripts/results.py

190
tests/test_interspersed.sh Executable file
View File

@@ -0,0 +1,190 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Interspersed tests ==="
rm -rf blocks
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
echo "--- Interspersed file test ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_t files[4];
lfs_file_open(&lfs, &files[0], "a", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &files[1], "b", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &files[2], "c", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &files[3], "d", LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_write(&lfs, &files[0], (const void*)"a", 1) => 1;
lfs_file_write(&lfs, &files[1], (const void*)"b", 1) => 1;
lfs_file_write(&lfs, &files[2], (const void*)"c", 1) => 1;
lfs_file_write(&lfs, &files[3], (const void*)"d", 1) => 1;
}
lfs_file_close(&lfs, &files[0]);
lfs_file_close(&lfs, &files[1]);
lfs_file_close(&lfs, &files[2]);
lfs_file_close(&lfs, &files[3]);
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "a") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "b") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "c") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "d") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_file_open(&lfs, &files[0], "a", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &files[1], "b", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &files[2], "c", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &files[3], "d", LFS_O_RDONLY) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_read(&lfs, &files[0], buffer, 1) => 1;
buffer[0] => 'a';
lfs_file_read(&lfs, &files[1], buffer, 1) => 1;
buffer[0] => 'b';
lfs_file_read(&lfs, &files[2], buffer, 1) => 1;
buffer[0] => 'c';
lfs_file_read(&lfs, &files[3], buffer, 1) => 1;
buffer[0] => 'd';
}
lfs_file_close(&lfs, &files[0]);
lfs_file_close(&lfs, &files[1]);
lfs_file_close(&lfs, &files[2]);
lfs_file_close(&lfs, &files[3]);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Interspersed remove file test ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_t files[4];
lfs_file_open(&lfs, &files[0], "e", LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &files[0], (const void*)"e", 1) => 1;
}
lfs_remove(&lfs, "a") => 0;
lfs_remove(&lfs, "b") => 0;
lfs_remove(&lfs, "c") => 0;
lfs_remove(&lfs, "d") => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &files[0], (const void*)"e", 1) => 1;
}
lfs_file_close(&lfs, &files[0]);
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "e") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_file_open(&lfs, &files[0], "e", LFS_O_RDONLY) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_read(&lfs, &files[0], buffer, 1) => 1;
buffer[0] => 'e';
}
lfs_file_close(&lfs, &files[0]);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Remove inconveniently test ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_t files[4];
lfs_file_open(&lfs, &files[0], "e", LFS_O_WRONLY | LFS_O_TRUNC) => 0;
lfs_file_open(&lfs, &files[1], "f", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &files[2], "g", LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &files[0], (const void*)"e", 1) => 1;
lfs_file_write(&lfs, &files[1], (const void*)"f", 1) => 1;
lfs_file_write(&lfs, &files[2], (const void*)"g", 1) => 1;
}
lfs_remove(&lfs, "f") => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &files[0], (const void*)"e", 1) => 1;
lfs_file_write(&lfs, &files[1], (const void*)"f", 1) => 1;
lfs_file_write(&lfs, &files[2], (const void*)"g", 1) => 1;
}
lfs_file_close(&lfs, &files[0]);
lfs_file_close(&lfs, &files[1]);
lfs_file_close(&lfs, &files[2]);
lfs_dir_open(&lfs, &dir, "/") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "e") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "g") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_file_open(&lfs, &files[0], "e", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &files[1], "g", LFS_O_RDONLY) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_read(&lfs, &files[0], buffer, 1) => 1;
buffer[0] => 'e';
lfs_file_read(&lfs, &files[1], buffer, 1) => 1;
buffer[0] => 'g';
}
lfs_file_close(&lfs, &files[0]);
lfs_file_close(&lfs, &files[1]);
lfs_unmount(&lfs) => 0;
TEST
scripts/results.py

View File

@@ -1,9 +1,11 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Move tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
@@ -17,220 +19,315 @@ tests/test.py << TEST
lfs_mkdir(&lfs, "a/hi/bonjour") => 0;
lfs_mkdir(&lfs, "a/hi/ohayo") => 0;
lfs_file_open(&lfs, &file[0], "a/hello", LFS_O_CREAT | LFS_O_WRONLY) => 0;
lfs_file_write(&lfs, &file[0], "hola\n", 5) => 5;
lfs_file_write(&lfs, &file[0], "bonjour\n", 8) => 8;
lfs_file_write(&lfs, &file[0], "ohayo\n", 6) => 6;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_open(&lfs, &file, "a/hello", LFS_O_CREAT | LFS_O_WRONLY) => 0;
lfs_file_write(&lfs, &file, "hola\n", 5) => 5;
lfs_file_write(&lfs, &file, "bonjour\n", 8) => 8;
lfs_file_write(&lfs, &file, "ohayo\n", 6) => 6;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move file ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "a/hello", "b/hello") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "a") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "a") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hi") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_open(&lfs, &dir[0], "b") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "b") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move file corrupt source ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "b/hello", "c/hello") => 0;
lfs_unmount(&lfs) => 0;
TEST
rm -v blocks/7
tests/test.py << TEST
scripts/corrupt.py -n 1
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "b") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "b") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_open(&lfs, &dir[0], "c") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "c") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move file corrupt source and dest ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "c/hello", "d/hello") => 0;
lfs_unmount(&lfs) => 0;
TEST
rm -v blocks/8
rm -v blocks/a
tests/test.py << TEST
scripts/corrupt.py -n 2
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "c") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "c") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_open(&lfs, &dir[0], "d") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "d") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move file after corrupt ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "c/hello", "d/hello") => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir, "c") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "d") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move dir ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "a/hi", "b/hi") => 0;
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "a") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "a") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_open(&lfs, &dir[0], "b") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "b") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hi") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move dir corrupt source ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "b/hi", "c/hi") => 0;
lfs_unmount(&lfs) => 0;
TEST
rm -v blocks/7
tests/test.py << TEST
scripts/corrupt.py -n 1
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "b") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "b") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_open(&lfs, &dir[0], "c") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "c") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hi") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move dir corrupt source and dest ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "c/hi", "d/hi") => 0;
lfs_unmount(&lfs) => 0;
TEST
rm -v blocks/9
rm -v blocks/a
tests/test.py << TEST
scripts/corrupt.py -n 2
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "c") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "c") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hi") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_open(&lfs, &dir[0], "d") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "d") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move dir after corrupt ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_rename(&lfs, "c/hi", "d/hi") => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir, "c") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "d") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hello") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hi") => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move check ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "a/hi") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir[0], "b/hi") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir[0], "d/hi") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "a/hi") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "b/hi") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "c/hi") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir[0], "c/hi") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "d/hi") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "hola") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "bonjour") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hola") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "ohayo") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir[0], "a/hello") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir[0], "b/hello") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir[0], "d/hello") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "a/hello") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "b/hello") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "c/hello") => LFS_ERR_NOENT;
lfs_file_open(&lfs, &file[0], "c/hello", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file[0], buffer, 5) => 5;
lfs_file_open(&lfs, &file, "d/hello", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, buffer, 5) => 5;
memcmp(buffer, "hola\n", 5) => 0;
lfs_file_read(&lfs, &file[0], buffer, 8) => 8;
lfs_file_read(&lfs, &file, buffer, 8) => 8;
memcmp(buffer, "bonjour\n", 8) => 0;
lfs_file_read(&lfs, &file[0], buffer, 6) => 6;
lfs_file_read(&lfs, &file, buffer, 6) => 6;
memcmp(buffer, "ohayo\n", 6) => 0;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Move state stealing ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_remove(&lfs, "b") => 0;
lfs_remove(&lfs, "c") => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir, "a/hi") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "b") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "c") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "d/hi") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "bonjour") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "hola") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "ohayo") => 0;
lfs_dir_read(&lfs, &dir, &info) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_dir_open(&lfs, &dir, "a/hello") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "b") => LFS_ERR_NOENT;
lfs_dir_open(&lfs, &dir, "c") => LFS_ERR_NOENT;
lfs_file_open(&lfs, &file, "d/hello", LFS_O_RDONLY) => 0;
lfs_file_read(&lfs, &file, buffer, 5) => 5;
memcmp(buffer, "hola\n", 5) => 0;
lfs_file_read(&lfs, &file, buffer, 8) => 8;
memcmp(buffer, "bonjour\n", 8) => 0;
lfs_file_read(&lfs, &file, buffer, 6) => 6;
memcmp(buffer, "ohayo\n", 6) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py
scripts/results.py

View File

@@ -1,41 +1,46 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Orphan tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
echo "--- Orphan test ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "parent") => 0;
lfs_mkdir(&lfs, "parent/orphan") => 0;
lfs_mkdir(&lfs, "parent/child") => 0;
lfs_remove(&lfs, "parent/orphan") => 0;
TEST
# remove most recent file, this should be the update to the previous
# corrupt most recent commit, this should be the update to the previous
# linked-list entry and should orphan the child
rm -v blocks/8
tests/test.py << TEST
scripts/corrupt.py
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "parent/orphan", &info) => LFS_ERR_NOENT;
unsigned before = 0;
lfs_traverse(&lfs, test_count, &before) => 0;
test_log("before", before);
lfs_deorphan(&lfs) => 0;
lfs_stat(&lfs, "parent/orphan", &info) => LFS_ERR_NOENT;
unsigned after = 0;
lfs_traverse(&lfs, test_count, &after) => 0;
test_log("after", after);
lfs_ssize_t before = lfs_fs_size(&lfs);
before => 8;
lfs_unmount(&lfs) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "parent/orphan", &info) => LFS_ERR_NOENT;
lfs_ssize_t orphaned = lfs_fs_size(&lfs);
orphaned => 8;
lfs_mkdir(&lfs, "parent/otherchild") => 0;
lfs_stat(&lfs, "parent/orphan", &info) => LFS_ERR_NOENT;
lfs_ssize_t deorphaned = lfs_fs_size(&lfs);
deorphaned => 8;
int diff = before - after;
diff => 2;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py
scripts/results.py

View File

@@ -1,186 +0,0 @@
#!/bin/bash
set -eu
echo "=== Parallel tests ==="
rm -rf blocks
tests/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
echo "--- Parallel file test ---"
tests/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "a", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &file[1], "b", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &file[2], "c", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &file[3], "d", LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_write(&lfs, &file[0], (const void*)"a", 1) => 1;
lfs_file_write(&lfs, &file[1], (const void*)"b", 1) => 1;
lfs_file_write(&lfs, &file[2], (const void*)"c", 1) => 1;
lfs_file_write(&lfs, &file[3], (const void*)"d", 1) => 1;
}
lfs_file_close(&lfs, &file[0]);
lfs_file_close(&lfs, &file[1]);
lfs_file_close(&lfs, &file[2]);
lfs_file_close(&lfs, &file[3]);
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "a") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "b") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "c") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "d") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_file_open(&lfs, &file[0], "a", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file[1], "b", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file[2], "c", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file[3], "d", LFS_O_RDONLY) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_read(&lfs, &file[0], buffer, 1) => 1;
buffer[0] => 'a';
lfs_file_read(&lfs, &file[1], buffer, 1) => 1;
buffer[0] => 'b';
lfs_file_read(&lfs, &file[2], buffer, 1) => 1;
buffer[0] => 'c';
lfs_file_read(&lfs, &file[3], buffer, 1) => 1;
buffer[0] => 'd';
}
lfs_file_close(&lfs, &file[0]);
lfs_file_close(&lfs, &file[1]);
lfs_file_close(&lfs, &file[2]);
lfs_file_close(&lfs, &file[3]);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Parallel remove file test ---"
tests/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "e", LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &file[0], (const void*)"e", 1) => 1;
}
lfs_remove(&lfs, "a") => 0;
lfs_remove(&lfs, "b") => 0;
lfs_remove(&lfs, "c") => 0;
lfs_remove(&lfs, "d") => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &file[0], (const void*)"e", 1) => 1;
}
lfs_file_close(&lfs, &file[0]);
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "e") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_file_open(&lfs, &file[0], "e", LFS_O_RDONLY) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_read(&lfs, &file[0], buffer, 1) => 1;
buffer[0] => 'e';
}
lfs_file_close(&lfs, &file[0]);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Remove inconveniently test ---"
tests/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "e", LFS_O_WRONLY | LFS_O_TRUNC) => 0;
lfs_file_open(&lfs, &file[1], "f", LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_open(&lfs, &file[2], "g", LFS_O_WRONLY | LFS_O_CREAT) => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &file[0], (const void*)"e", 1) => 1;
lfs_file_write(&lfs, &file[1], (const void*)"f", 1) => 1;
lfs_file_write(&lfs, &file[2], (const void*)"g", 1) => 1;
}
lfs_remove(&lfs, "f") => 0;
for (int i = 0; i < 5; i++) {
lfs_file_write(&lfs, &file[0], (const void*)"e", 1) => 1;
lfs_file_write(&lfs, &file[1], (const void*)"f", 1) => 1;
lfs_file_write(&lfs, &file[2], (const void*)"g", 1) => 1;
}
lfs_file_close(&lfs, &file[0]);
lfs_file_close(&lfs, &file[1]);
lfs_file_close(&lfs, &file[2]);
lfs_dir_open(&lfs, &dir[0], "/") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, ".") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "..") => 0;
info.type => LFS_TYPE_DIR;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "e") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, "g") => 0;
info.type => LFS_TYPE_REG;
info.size => 10;
lfs_dir_read(&lfs, &dir[0], &info) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_file_open(&lfs, &file[0], "e", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file[1], "g", LFS_O_RDONLY) => 0;
for (int i = 0; i < 10; i++) {
lfs_file_read(&lfs, &file[0], buffer, 1) => 1;
buffer[0] => 'e';
lfs_file_read(&lfs, &file[1], buffer, 1) => 1;
buffer[0] => 'g';
}
lfs_file_close(&lfs, &file[0]);
lfs_file_close(&lfs, &file[1]);
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py

View File

@@ -1,13 +1,15 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Path tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "tea") => 0;
lfs_mkdir(&lfs, "coffee") => 0;
@@ -25,7 +27,7 @@ tests/test.py << TEST
TEST
echo "--- Root path tests ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "tea/hottea", &info) => 0;
strcmp(info.name, "hottea") => 0;
@@ -39,7 +41,7 @@ tests/test.py << TEST
TEST
echo "--- Redundant slash path tests ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "/tea/hottea", &info) => 0;
strcmp(info.name, "hottea") => 0;
@@ -55,7 +57,7 @@ tests/test.py << TEST
TEST
echo "--- Dot path tests ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "./tea/hottea", &info) => 0;
strcmp(info.name, "hottea") => 0;
@@ -73,7 +75,7 @@ tests/test.py << TEST
TEST
echo "--- Dot dot path tests ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "coffee/../tea/hottea", &info) => 0;
strcmp(info.name, "hottea") => 0;
@@ -90,8 +92,24 @@ tests/test.py << TEST
lfs_unmount(&lfs) => 0;
TEST
echo "--- Trailing dot path tests ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "tea/hottea/", &info) => 0;
strcmp(info.name, "hottea") => 0;
lfs_stat(&lfs, "tea/hottea/.", &info) => 0;
strcmp(info.name, "hottea") => 0;
lfs_stat(&lfs, "tea/hottea/./.", &info) => 0;
strcmp(info.name, "hottea") => 0;
lfs_stat(&lfs, "tea/hottea/..", &info) => 0;
strcmp(info.name, "tea") => 0;
lfs_stat(&lfs, "tea/hottea/../.", &info) => 0;
strcmp(info.name, "tea") => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Root dot dot path tests ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "coffee/../../../../../../tea/hottea", &info) => 0;
strcmp(info.name, "hottea") => 0;
@@ -103,21 +121,82 @@ tests/test.py << TEST
TEST
echo "--- Root tests ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_stat(&lfs, "/", &info) => 0;
info.type => LFS_TYPE_DIR;
strcmp(info.name, "/") => 0;
lfs_mkdir(&lfs, "/") => LFS_ERR_EXIST;
lfs_file_open(&lfs, &file, "/", LFS_O_WRONLY | LFS_O_CREAT)
=> LFS_ERR_ISDIR;
// more corner cases
lfs_remove(&lfs, "") => LFS_ERR_INVAL;
lfs_remove(&lfs, ".") => LFS_ERR_INVAL;
lfs_remove(&lfs, "..") => LFS_ERR_INVAL;
lfs_remove(&lfs, "/") => LFS_ERR_INVAL;
lfs_remove(&lfs, "//") => LFS_ERR_INVAL;
lfs_remove(&lfs, "./") => LFS_ERR_INVAL;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Sketchy path tests ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "dirt/ground") => LFS_ERR_NOENT;
lfs_mkdir(&lfs, "dirt/ground/earth") => LFS_ERR_NOENT;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py
echo "--- Superblock conflict test ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "littlefs") => 0;
lfs_remove(&lfs, "littlefs") => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Max path test ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(path, 'w', LFS_NAME_MAX+1);
path[LFS_NAME_MAX+2] = '\0';
lfs_mkdir(&lfs, path) => LFS_ERR_NAMETOOLONG;
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT) => LFS_ERR_NAMETOOLONG;
memcpy(path, "coffee/", strlen("coffee/"));
memset(path+strlen("coffee/"), 'w', LFS_NAME_MAX+1);
path[strlen("coffee/")+LFS_NAME_MAX+2] = '\0';
lfs_mkdir(&lfs, path) => LFS_ERR_NAMETOOLONG;
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT) => LFS_ERR_NAMETOOLONG;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Really big path test ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
memset(path, 'w', LFS_NAME_MAX);
path[LFS_NAME_MAX] = '\0';
lfs_mkdir(&lfs, path) => 0;
lfs_remove(&lfs, path) => 0;
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_remove(&lfs, path) => 0;
memcpy(path, "coffee/", strlen("coffee/"));
memset(path+strlen("coffee/"), 'w', LFS_NAME_MAX);
path[strlen("coffee/")+LFS_NAME_MAX] = '\0';
lfs_mkdir(&lfs, path) => 0;
lfs_remove(&lfs, path) => 0;
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_remove(&lfs, path) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/results.py

View File

@@ -1,361 +1,431 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Seek tests ==="
SMALLSIZE=4
MEDIUMSIZE=128
LARGESIZE=132
echo "=== Seek tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
lfs_mount(&lfs, &cfg) => 0;
lfs_mkdir(&lfs, "hello") => 0;
for (int i = 0; i < $LARGESIZE; i++) {
sprintf((char*)buffer, "hello/kitty%d", i);
lfs_file_open(&lfs, &file[0], (char*)buffer,
sprintf(path, "hello/kitty%03d", i);
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_APPEND) => 0;
size = strlen("kittycatcat");
lfs_size_t size = strlen("kittycatcat");
memcpy(buffer, "kittycatcat", size);
for (int j = 0; j < $LARGESIZE; j++) {
lfs_file_write(&lfs, &file[0], buffer, size);
lfs_file_write(&lfs, &file, buffer, size);
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
echo "--- Simple dir seek ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "hello") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "hello") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_soff_t pos;
int i;
for (i = 0; i < $SMALLSIZE; i++) {
sprintf((char*)buffer, "kitty%d", i);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
pos = lfs_dir_tell(&lfs, &dir[0]);
sprintf(path, "kitty%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
pos = lfs_dir_tell(&lfs, &dir);
}
pos >= 0 => 1;
lfs_dir_seek(&lfs, &dir[0], pos) => 0;
sprintf((char*)buffer, "kitty%d", i);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
lfs_dir_seek(&lfs, &dir, pos) => 0;
sprintf(path, "kitty%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
lfs_dir_rewind(&lfs, &dir[0]) => 0;
sprintf((char*)buffer, "kitty%d", 0);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_rewind(&lfs, &dir) => 0;
sprintf(path, "kitty%03d", 0);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
lfs_dir_seek(&lfs, &dir[0], pos) => 0;
sprintf((char*)buffer, "kitty%d", i);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
lfs_dir_seek(&lfs, &dir, pos) => 0;
sprintf(path, "kitty%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Large dir seek ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_dir_open(&lfs, &dir[0], "hello") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_open(&lfs, &dir, "hello") => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_soff_t pos;
int i;
for (i = 0; i < $MEDIUMSIZE; i++) {
sprintf((char*)buffer, "kitty%d", i);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
pos = lfs_dir_tell(&lfs, &dir[0]);
sprintf(path, "kitty%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
pos = lfs_dir_tell(&lfs, &dir);
}
pos >= 0 => 1;
lfs_dir_seek(&lfs, &dir[0], pos) => 0;
sprintf((char*)buffer, "kitty%d", i);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
lfs_dir_seek(&lfs, &dir, pos) => 0;
sprintf(path, "kitty%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
lfs_dir_rewind(&lfs, &dir[0]) => 0;
sprintf((char*)buffer, "kitty%d", 0);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_rewind(&lfs, &dir) => 0;
sprintf(path, "kitty%03d", 0);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, ".") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, "..") => 0;
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
lfs_dir_seek(&lfs, &dir[0], pos) => 0;
sprintf((char*)buffer, "kitty%d", i);
lfs_dir_read(&lfs, &dir[0], &info) => 1;
strcmp(info.name, (char*)buffer) => 0;
lfs_dir_seek(&lfs, &dir, pos) => 0;
sprintf(path, "kitty%03d", i);
lfs_dir_read(&lfs, &dir, &info) => 1;
strcmp(info.name, path) => 0;
lfs_dir_close(&lfs, &dir[0]) => 0;
lfs_dir_close(&lfs, &dir) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Simple file seek ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "hello/kitty42", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file, "hello/kitty042", LFS_O_RDONLY) => 0;
lfs_soff_t pos;
size = strlen("kittycatcat");
lfs_size_t size = strlen("kittycatcat");
for (int i = 0; i < $SMALLSIZE; i++) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
pos = lfs_file_tell(&lfs, &file[0]);
pos = lfs_file_tell(&lfs, &file);
}
pos >= 0 => 1;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_rewind(&lfs, &file[0]) => 0;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_rewind(&lfs, &file) => 0;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], 0, LFS_SEEK_CUR) => size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_CUR) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], size, LFS_SEEK_CUR) => 3*size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, size, LFS_SEEK_CUR) => 3*size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], -size, LFS_SEEK_CUR) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, -size, LFS_SEEK_CUR) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_size_t size = lfs_file_size(&lfs, &file[0]);
lfs_file_seek(&lfs, &file[0], 0, LFS_SEEK_CUR) => size;
size = lfs_file_size(&lfs, &file);
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_CUR) => size;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Large file seek ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "hello/kitty42", LFS_O_RDONLY) => 0;
lfs_file_open(&lfs, &file, "hello/kitty042", LFS_O_RDONLY) => 0;
lfs_soff_t pos;
size = strlen("kittycatcat");
lfs_size_t size = strlen("kittycatcat");
for (int i = 0; i < $MEDIUMSIZE; i++) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
pos = lfs_file_tell(&lfs, &file[0]);
pos = lfs_file_tell(&lfs, &file);
}
pos >= 0 => 1;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_rewind(&lfs, &file[0]) => 0;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_rewind(&lfs, &file) => 0;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], 0, LFS_SEEK_CUR) => size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_CUR) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], size, LFS_SEEK_CUR) => 3*size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, size, LFS_SEEK_CUR) => 3*size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], -size, LFS_SEEK_CUR) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, -size, LFS_SEEK_CUR) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_size_t size = lfs_file_size(&lfs, &file[0]);
lfs_file_seek(&lfs, &file[0], 0, LFS_SEEK_CUR) => size;
size = lfs_file_size(&lfs, &file);
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_CUR) => size;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Simple file seek and write ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "hello/kitty42", LFS_O_RDWR) => 0;
lfs_file_open(&lfs, &file, "hello/kitty042", LFS_O_RDWR) => 0;
lfs_soff_t pos;
size = strlen("kittycatcat");
lfs_size_t size = strlen("kittycatcat");
for (int i = 0; i < $SMALLSIZE; i++) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
pos = lfs_file_tell(&lfs, &file[0]);
pos = lfs_file_tell(&lfs, &file);
}
pos >= 0 => 1;
memcpy(buffer, "doggodogdog", size);
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_write(&lfs, &file, buffer, size) => size;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "doggodogdog", size) => 0;
lfs_file_rewind(&lfs, &file[0]) => 0;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_rewind(&lfs, &file) => 0;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "doggodogdog", size) => 0;
lfs_file_seek(&lfs, &file[0], -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_size_t size = lfs_file_size(&lfs, &file[0]);
lfs_file_seek(&lfs, &file[0], 0, LFS_SEEK_CUR) => size;
size = lfs_file_size(&lfs, &file);
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_CUR) => size;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Large file seek and write ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "hello/kitty42", LFS_O_RDWR) => 0;
lfs_file_open(&lfs, &file, "hello/kitty042", LFS_O_RDWR) => 0;
lfs_soff_t pos;
size = strlen("kittycatcat");
lfs_size_t size = strlen("kittycatcat");
for (int i = 0; i < $MEDIUMSIZE; i++) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
if (i != $SMALLSIZE) {
memcmp(buffer, "kittycatcat", size) => 0;
}
pos = lfs_file_tell(&lfs, &file[0]);
pos = lfs_file_tell(&lfs, &file);
}
pos >= 0 => 1;
memcpy(buffer, "doggodogdog", size);
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_write(&lfs, &file, buffer, size) => size;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "doggodogdog", size) => 0;
lfs_file_rewind(&lfs, &file[0]) => 0;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_rewind(&lfs, &file) => 0;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_seek(&lfs, &file[0], pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, pos, LFS_SEEK_SET) => pos;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "doggodogdog", size) => 0;
lfs_file_seek(&lfs, &file[0], -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, -size, LFS_SEEK_END) >= 0 => 1;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_size_t size = lfs_file_size(&lfs, &file[0]);
lfs_file_seek(&lfs, &file[0], 0, LFS_SEEK_CUR) => size;
size = lfs_file_size(&lfs, &file);
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_CUR) => size;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Boundary seek and write ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "hello/kitty42", LFS_O_RDWR) => 0;
lfs_file_open(&lfs, &file, "hello/kitty042", LFS_O_RDWR) => 0;
size = strlen("hedgehoghog");
lfs_size_t size = strlen("hedgehoghog");
const lfs_soff_t offsets[] = {512, 1020, 513, 1021, 511, 1019};
for (int i = 0; i < sizeof(offsets) / sizeof(offsets[0]); i++) {
for (unsigned i = 0; i < sizeof(offsets) / sizeof(offsets[0]); i++) {
lfs_soff_t off = offsets[i];
memcpy(buffer, "hedgehoghog", size);
lfs_file_seek(&lfs, &file[0], off, LFS_SEEK_SET) => off;
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file[0], off, LFS_SEEK_SET) => off;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, off, LFS_SEEK_SET) => off;
lfs_file_write(&lfs, &file, buffer, size) => size;
lfs_file_seek(&lfs, &file, off, LFS_SEEK_SET) => off;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "hedgehoghog", size) => 0;
lfs_file_seek(&lfs, &file[0], 0, LFS_SEEK_SET) => 0;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_SET) => 0;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "kittycatcat", size) => 0;
lfs_file_sync(&lfs, &file[0]) => 0;
lfs_file_sync(&lfs, &file) => 0;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Out-of-bounds seek ---"
tests/test.py << TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file[0], "hello/kitty42", LFS_O_RDWR) => 0;
lfs_file_open(&lfs, &file, "hello/kitty042", LFS_O_RDWR) => 0;
size = strlen("kittycatcat");
lfs_file_size(&lfs, &file[0]) => $LARGESIZE*size;
lfs_file_seek(&lfs, &file[0], ($LARGESIZE+$SMALLSIZE)*size,
lfs_size_t size = strlen("kittycatcat");
lfs_file_size(&lfs, &file) => $LARGESIZE*size;
lfs_file_seek(&lfs, &file, ($LARGESIZE+$SMALLSIZE)*size,
LFS_SEEK_SET) => ($LARGESIZE+$SMALLSIZE)*size;
lfs_file_read(&lfs, &file[0], buffer, size) => 0;
lfs_file_read(&lfs, &file, buffer, size) => 0;
memcpy(buffer, "porcupineee", size);
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_file_write(&lfs, &file, buffer, size) => size;
lfs_file_seek(&lfs, &file[0], ($LARGESIZE+$SMALLSIZE)*size,
lfs_file_seek(&lfs, &file, ($LARGESIZE+$SMALLSIZE)*size,
LFS_SEEK_SET) => ($LARGESIZE+$SMALLSIZE)*size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "porcupineee", size) => 0;
lfs_file_seek(&lfs, &file[0], $LARGESIZE*size,
lfs_file_seek(&lfs, &file, $LARGESIZE*size,
LFS_SEEK_SET) => $LARGESIZE*size;
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "\0\0\0\0\0\0\0\0\0\0\0", size) => 0;
lfs_file_seek(&lfs, &file[0], -(($LARGESIZE+$SMALLSIZE)*size),
lfs_file_seek(&lfs, &file, -(($LARGESIZE+$SMALLSIZE)*size),
LFS_SEEK_CUR) => LFS_ERR_INVAL;
lfs_file_tell(&lfs, &file[0]) => ($LARGESIZE+1)*size;
lfs_file_tell(&lfs, &file) => ($LARGESIZE+1)*size;
lfs_file_seek(&lfs, &file[0], -(($LARGESIZE+2*$SMALLSIZE)*size),
lfs_file_seek(&lfs, &file, -(($LARGESIZE+2*$SMALLSIZE)*size),
LFS_SEEK_END) => LFS_ERR_INVAL;
lfs_file_tell(&lfs, &file[0]) => ($LARGESIZE+1)*size;
lfs_file_tell(&lfs, &file) => ($LARGESIZE+1)*size;
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Results ---"
tests/stats.py
echo "--- Inline write and seek ---"
for SIZE in $SMALLSIZE $MEDIUMSIZE $LARGESIZE
do
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "hello/tinykitty$SIZE",
LFS_O_RDWR | LFS_O_CREAT) => 0;
int j = 0;
int k = 0;
memcpy(buffer, "abcdefghijklmnopqrstuvwxyz", 26);
for (unsigned i = 0; i < $SIZE; i++) {
lfs_file_write(&lfs, &file, &buffer[j++ % 26], 1) => 1;
lfs_file_tell(&lfs, &file) => i+1;
lfs_file_size(&lfs, &file) => i+1;
}
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_SET) => 0;
lfs_file_tell(&lfs, &file) => 0;
lfs_file_size(&lfs, &file) => $SIZE;
for (unsigned i = 0; i < $SIZE; i++) {
uint8_t c;
lfs_file_read(&lfs, &file, &c, 1) => 1;
c => buffer[k++ % 26];
}
lfs_file_sync(&lfs, &file) => 0;
lfs_file_tell(&lfs, &file) => $SIZE;
lfs_file_size(&lfs, &file) => $SIZE;
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_SET) => 0;
for (unsigned i = 0; i < $SIZE; i++) {
lfs_file_write(&lfs, &file, &buffer[j++ % 26], 1) => 1;
lfs_file_tell(&lfs, &file) => i+1;
lfs_file_size(&lfs, &file) => $SIZE;
lfs_file_sync(&lfs, &file) => 0;
lfs_file_tell(&lfs, &file) => i+1;
lfs_file_size(&lfs, &file) => $SIZE;
if (i < $SIZE-2) {
uint8_t c[3];
lfs_file_seek(&lfs, &file, -1, LFS_SEEK_CUR) => i;
lfs_file_read(&lfs, &file, &c, 3) => 3;
lfs_file_tell(&lfs, &file) => i+3;
lfs_file_size(&lfs, &file) => $SIZE;
lfs_file_seek(&lfs, &file, i+1, LFS_SEEK_SET) => i+1;
lfs_file_tell(&lfs, &file) => i+1;
lfs_file_size(&lfs, &file) => $SIZE;
}
}
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_SET) => 0;
lfs_file_tell(&lfs, &file) => 0;
lfs_file_size(&lfs, &file) => $SIZE;
for (unsigned i = 0; i < $SIZE; i++) {
uint8_t c;
lfs_file_read(&lfs, &file, &c, 1) => 1;
c => buffer[k++ % 26];
}
lfs_file_sync(&lfs, &file) => 0;
lfs_file_tell(&lfs, &file) => $SIZE;
lfs_file_size(&lfs, &file) => $SIZE;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
done
scripts/results.py

View File

@@ -1,104 +1,309 @@
#!/bin/bash
set -eu
export TEST_FILE=$0
trap 'export TEST_LINE=$LINENO' DEBUG
echo "=== Truncate tests ==="
SMALLSIZE=32
MEDIUMSIZE=2048
LARGESIZE=8192
echo "=== Truncate tests ==="
rm -rf blocks
tests/test.py << TEST
scripts/test.py << TEST
lfs_format(&lfs, &cfg) => 0;
TEST
echo "--- Simple truncate ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldynoop",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
strcpy((char*)buffer, "hair");
lfs_size_t size = strlen((char*)buffer);
for (lfs_off_t j = 0; j < $LARGESIZE; j += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_size(&lfs, &file) => $LARGESIZE;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldynoop", LFS_O_RDWR) => 0;
lfs_file_size(&lfs, &file) => $LARGESIZE;
lfs_file_truncate(&lfs, &file, $MEDIUMSIZE) => 0;
lfs_file_size(&lfs, &file) => $MEDIUMSIZE;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldynoop", LFS_O_RDONLY) => 0;
lfs_file_size(&lfs, &file) => $MEDIUMSIZE;
lfs_size_t size = strlen("hair");
for (lfs_off_t j = 0; j < $MEDIUMSIZE; j += size) {
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "hair", size) => 0;
}
lfs_file_read(&lfs, &file, buffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Truncate and read ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldyread",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
strcpy((char*)buffer, "hair");
lfs_size_t size = strlen((char*)buffer);
for (lfs_off_t j = 0; j < $LARGESIZE; j += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_size(&lfs, &file) => $LARGESIZE;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldyread", LFS_O_RDWR) => 0;
lfs_file_size(&lfs, &file) => $LARGESIZE;
lfs_file_truncate(&lfs, &file, $MEDIUMSIZE) => 0;
lfs_file_size(&lfs, &file) => $MEDIUMSIZE;
lfs_size_t size = strlen("hair");
for (lfs_off_t j = 0; j < $MEDIUMSIZE; j += size) {
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "hair", size) => 0;
}
lfs_file_read(&lfs, &file, buffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldyread", LFS_O_RDONLY) => 0;
lfs_file_size(&lfs, &file) => $MEDIUMSIZE;
lfs_size_t size = strlen("hair");
for (lfs_off_t j = 0; j < $MEDIUMSIZE; j += size) {
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "hair", size) => 0;
}
lfs_file_read(&lfs, &file, buffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Write, truncate, and read ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "sequence",
LFS_O_RDWR | LFS_O_CREAT | LFS_O_TRUNC) => 0;
lfs_size_t size = lfs.cfg->cache_size;
lfs_size_t qsize = size / 4;
uint8_t *wb = buffer;
uint8_t *rb = buffer + size;
for (lfs_off_t j = 0; j < size; ++j) {
wb[j] = j;
}
/* Spread sequence over size */
lfs_file_write(&lfs, &file, wb, size) => size;
lfs_file_size(&lfs, &file) => size;
lfs_file_tell(&lfs, &file) => size;
lfs_file_seek(&lfs, &file, 0, LFS_SEEK_SET) => 0;
lfs_file_tell(&lfs, &file) => 0;
/* Chop off the last quarter */
lfs_size_t trunc = size - qsize;
lfs_file_truncate(&lfs, &file, trunc) => 0;
lfs_file_tell(&lfs, &file) => 0;
lfs_file_size(&lfs, &file) => trunc;
/* Read should produce first 3/4 */
lfs_file_read(&lfs, &file, rb, size) => trunc;
memcmp(rb, wb, trunc) => 0;
/* Move to 1/4 */
lfs_file_size(&lfs, &file) => trunc;
lfs_file_seek(&lfs, &file, qsize, LFS_SEEK_SET) => qsize;
lfs_file_tell(&lfs, &file) => qsize;
/* Chop to 1/2 */
trunc -= qsize;
lfs_file_truncate(&lfs, &file, trunc) => 0;
lfs_file_tell(&lfs, &file) => qsize;
lfs_file_size(&lfs, &file) => trunc;
/* Read should produce second quarter */
lfs_file_read(&lfs, &file, rb, size) => trunc - qsize;
memcmp(rb, wb + qsize, trunc - qsize) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
echo "--- Truncate and write ---"
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldywrite",
LFS_O_WRONLY | LFS_O_CREAT) => 0;
strcpy((char*)buffer, "hair");
lfs_size_t size = strlen((char*)buffer);
for (lfs_off_t j = 0; j < $LARGESIZE; j += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_size(&lfs, &file) => $LARGESIZE;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldywrite", LFS_O_RDWR) => 0;
lfs_file_size(&lfs, &file) => $LARGESIZE;
lfs_file_truncate(&lfs, &file, $MEDIUMSIZE) => 0;
lfs_file_size(&lfs, &file) => $MEDIUMSIZE;
strcpy((char*)buffer, "bald");
lfs_size_t size = strlen((char*)buffer);
for (lfs_off_t j = 0; j < $MEDIUMSIZE; j += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_size(&lfs, &file) => $MEDIUMSIZE;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
scripts/test.py << TEST
lfs_mount(&lfs, &cfg) => 0;
lfs_file_open(&lfs, &file, "baldywrite", LFS_O_RDONLY) => 0;
lfs_file_size(&lfs, &file) => $MEDIUMSIZE;
lfs_size_t size = strlen("bald");
for (lfs_off_t j = 0; j < $MEDIUMSIZE; j += size) {
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "bald", size) => 0;
}
lfs_file_read(&lfs, &file, buffer, size) => 0;
lfs_file_close(&lfs, &file) => 0;
lfs_unmount(&lfs) => 0;
TEST
# More aggressive general truncation tests
truncate_test() {
STARTSIZES="$1"
HOTSIZES="$2"
COLDSIZES="$3"
tests/test.py << TEST
STARTSEEKS="$2"
HOTSIZES="$3"
COLDSIZES="$4"
scripts/test.py << TEST
static const lfs_off_t startsizes[] = {$STARTSIZES};
static const lfs_off_t startseeks[] = {$STARTSEEKS};
static const lfs_off_t hotsizes[] = {$HOTSIZES};
lfs_mount(&lfs, &cfg) => 0;
for (int i = 0; i < sizeof(startsizes)/sizeof(startsizes[0]); i++) {
sprintf((char*)buffer, "hairyhead%d", i);
lfs_file_open(&lfs, &file[0], (const char*)buffer,
for (unsigned i = 0; i < sizeof(startsizes)/sizeof(startsizes[0]); i++) {
sprintf(path, "hairyhead%d", i);
lfs_file_open(&lfs, &file, path,
LFS_O_WRONLY | LFS_O_CREAT | LFS_O_TRUNC) => 0;
strcpy((char*)buffer, "hair");
size = strlen((char*)buffer);
for (int j = 0; j < startsizes[i]; j += size) {
lfs_file_write(&lfs, &file[0], buffer, size) => size;
lfs_size_t size = strlen((char*)buffer);
for (lfs_off_t j = 0; j < startsizes[i]; j += size) {
lfs_file_write(&lfs, &file, buffer, size) => size;
}
lfs_file_size(&lfs, &file[0]) => startsizes[i];
lfs_file_size(&lfs, &file) => startsizes[i];
lfs_file_truncate(&lfs, &file[0], hotsizes[i]) => 0;
lfs_file_size(&lfs, &file[0]) => hotsizes[i];
if (startseeks[i] != startsizes[i]) {
lfs_file_seek(&lfs, &file,
startseeks[i], LFS_SEEK_SET) => startseeks[i];
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_truncate(&lfs, &file, hotsizes[i]) => 0;
lfs_file_size(&lfs, &file) => hotsizes[i];
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
static const lfs_off_t startsizes[] = {$STARTSIZES};
static const lfs_off_t hotsizes[] = {$HOTSIZES};
static const lfs_off_t coldsizes[] = {$COLDSIZES};
lfs_mount(&lfs, &cfg) => 0;
for (int i = 0; i < sizeof(startsizes)/sizeof(startsizes[0]); i++) {
sprintf((char*)buffer, "hairyhead%d", i);
lfs_file_open(&lfs, &file[0], (const char*)buffer, LFS_O_RDWR) => 0;
lfs_file_size(&lfs, &file[0]) => hotsizes[i];
for (unsigned i = 0; i < sizeof(startsizes)/sizeof(startsizes[0]); i++) {
sprintf(path, "hairyhead%d", i);
lfs_file_open(&lfs, &file, path, LFS_O_RDWR) => 0;
lfs_file_size(&lfs, &file) => hotsizes[i];
size = strlen("hair");
int j = 0;
lfs_size_t size = strlen("hair");
lfs_off_t j = 0;
for (; j < startsizes[i] && j < hotsizes[i]; j += size) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "hair", size) => 0;
}
for (; j < hotsizes[i]; j += size) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "\0\0\0\0", size) => 0;
}
lfs_file_truncate(&lfs, &file[0], coldsizes[i]) => 0;
lfs_file_size(&lfs, &file[0]) => coldsizes[i];
lfs_file_truncate(&lfs, &file, coldsizes[i]) => 0;
lfs_file_size(&lfs, &file) => coldsizes[i];
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
TEST
tests/test.py << TEST
scripts/test.py << TEST
static const lfs_off_t startsizes[] = {$STARTSIZES};
static const lfs_off_t hotsizes[] = {$HOTSIZES};
static const lfs_off_t coldsizes[] = {$COLDSIZES};
lfs_mount(&lfs, &cfg) => 0;
for (int i = 0; i < sizeof(startsizes)/sizeof(startsizes[0]); i++) {
sprintf((char*)buffer, "hairyhead%d", i);
lfs_file_open(&lfs, &file[0], (const char*)buffer, LFS_O_RDONLY) => 0;
lfs_file_size(&lfs, &file[0]) => coldsizes[i];
for (unsigned i = 0; i < sizeof(startsizes)/sizeof(startsizes[0]); i++) {
sprintf(path, "hairyhead%d", i);
lfs_file_open(&lfs, &file, path, LFS_O_RDONLY) => 0;
lfs_file_size(&lfs, &file) => coldsizes[i];
size = strlen("hair");
int j = 0;
lfs_size_t size = strlen("hair");
lfs_off_t j = 0;
for (; j < startsizes[i] && j < hotsizes[i] && j < coldsizes[i];
j += size) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "hair", size) => 0;
}
for (; j < coldsizes[i]; j += size) {
lfs_file_read(&lfs, &file[0], buffer, size) => size;
lfs_file_read(&lfs, &file, buffer, size) => size;
memcmp(buffer, "\0\0\0\0", size) => 0;
}
lfs_file_close(&lfs, &file[0]) => 0;
lfs_file_close(&lfs, &file) => 0;
}
lfs_unmount(&lfs) => 0;
@@ -107,27 +312,44 @@ TEST
echo "--- Cold shrinking truncate ---"
truncate_test \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE"
echo "--- Cold expanding truncate ---"
truncate_test \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE"
echo "--- Warm shrinking truncate ---"
truncate_test \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
" 0, 0, 0, 0, 0"
echo "--- Warm expanding truncate ---"
truncate_test \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE"
echo "--- Results ---"
tests/stats.py
echo "--- Mid-file shrinking truncate ---"
truncate_test \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
" $LARGESIZE, $LARGESIZE, $LARGESIZE, $LARGESIZE, $LARGESIZE" \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
" 0, 0, 0, 0, 0"
echo "--- Mid-file expanding truncate ---"
truncate_test \
" 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE, 2*$LARGESIZE" \
" 0, 0, $SMALLSIZE, $MEDIUMSIZE, $LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE" \
"2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE, 2*$LARGESIZE"
scripts/results.py