github-mirrors/thirdparty-littlefs
1
0
Fork 0
mirror of https://github.com/eledio-devices/thirdparty-littlefs.git synced 2025-10-31 00:32:38 +01:00
Code Issues Packages Projects Releases Wiki Activity

Added support for RAM-independent reading of inline files

Browse Source 
One of the new features in LittleFS is "inline files", which is the
inlining of small files in the parent directory. Inline files have a big
limitation in that they no longer have a dedicated scratch area to write
out data before commit-time. This is fine as long as inline files are
small enough to fit in RAM.

However, this dependency on RAM creates an uncomfortable situation for
portability, with larger devices able to create larger files than
smaller devices. This problem is especially important on embedded
systems, where RAM is at a premium.

Recently, I realized this RAM requirement is necessary for _writing_
inline files, but not for _reading_ inline files. By allowing fetches of
specific slices of inline files it's possible to read inline files
without the RAM to back it.

However however, this creates a conflict with COW semantics. Normally,
when a file is open twice, it is referenced by a COW data structure that
can be updated independently. Inlines files that fit in RAM also allows
independent updates, but the moment an inline file can't fit in
RAM, any updates to that directory block could corrupt open files
referencing the inline file. The fact that this behaviour is only
inconsistent for inline files created on a different device with more
RAM creates a potential nightmare for user experience.

Fortunately, there is a workaround for this. When we are commiting to a
directory, any open files needs to live in a COW structure or in RAM.
While we could move large inline files to COW structures at open time,
this would break the separation of read/write operations and could lead
to write errors at read time (ie ENOSPC). But since this is only an
issue for commits, we can defer the move to a COW structure to any
commits to that directory. This means when committing to a directory we
need to find any _open_ large inline files and evict them from the
directory, leaving the file with a new COW structure even if it was
opened read only.

While complicated, the end result is inline files that can use the
MAX RAM that is available, but can be read with MIN RAM, even with
multiple write operations happening to the underlying directory block.
This prevents users from needing to learn the idiosyncrasies of inline
files to use the filesystem portably.
This commit is contained in:
Christopher Haster
2019-01-13 11:08:42 -06:00
parent 51b2c7e4b6
commit e1f9d2bc09
2 changed files with 184 additions and 122 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
lfs.h
View File

@@ -51,30 +51,21 @@ typedef uint32_t lfs_block_t;
#define LFS_NAME_MAX 255
#endif
// Maximum inline file size in bytes, may be redefined to limit RAM usage,
// but littlefs will automatically limit the LFS_INLINE_MAX to the
// configured cache_size. Limited to <= 1022. Stored in superblock and must
// be respected by other littlefs drivers.
#ifndef LFS_INLINE_MAX
#define LFS_INLINE_MAX 1022
#endif
// Maximum size of custom attributes in bytes, may be redefined, but there is
// no real benefit to using a smaller LFS_ATTR_MAX. Limited to <= 1022. Stored
// in superblock and must be respected by other littlefs drivers.
#ifndef LFS_ATTR_MAX
#define LFS_ATTR_MAX 1022
#endif
// Maximum size of a file in bytes, may be redefined to limit to support other
// drivers. Limited on disk to <= 4294967296. However, above 2147483647 the
// functions lfs_file_seek, lfs_file_size, and lfs_file_tell will return
// incorrect values due to signed sizes. Stored in superblock and must be
// respected by other littlefs drivers.
// incorrect values due to using signed integers. Stored in superblock and
// must be respected by other littlefs drivers.
#ifndef LFS_FILE_MAX
#define LFS_FILE_MAX 2147483647
#endif
// Maximum size of custom attributes in bytes, may be redefined, but there is
// no real benefit to using a smaller LFS_ATTR_MAX. Limited to <= 1022.
#ifndef LFS_ATTR_MAX
#define LFS_ATTR_MAX 1022
#endif
// Possible error codes, these are negative to allow
// valid positive return values
enum lfs_error {
@@ -234,24 +225,15 @@ struct lfs_config {
// superblock and must be respected by other littlefs drivers.
lfs_size_t name_max;
// Optional upper limit on inlined files in bytes. Inline files must be
// backed by RAM, but if a file fits in RAM it can be inlined into its
// directory block without needing its own data block. Must be <=
// cache_size and LFS_INLINE_MAX. Defaults to min(LFS_INLINE_MAX,
// cache_size) when zero. Stored in superblock and must be respected by
// other littlefs drivers.
lfs_size_t inline_max;
// Optional upper limit on custom attributes in bytes. No downside for
// larger attributes size but must be <= LFS_ATTR_MAX. Defaults to
// LFS_ATTR_MAX when zero. Stored in superblock and must be respected by
// other littlefs drivers.
lfs_size_t attr_max;
// Optional upper limit on files in bytes. No downside for larger files
// but must be <= LFS_FILE_MAX. Defaults to LFS_FILE_MAX when zero. Stored
// in superblock and must be respected by other littlefs drivers.
lfs_size_t file_max;
// Optional upper limit on custom attributes in bytes. No downside for
// larger attributes size but must be <= LFS_ATTR_MAX. Defaults to
// LFS_ATTR_MAX when zero.
lfs_size_t attr_max;
};
// File info structure
@@ -362,11 +344,9 @@ typedef struct lfs_superblock {
uint32_t version;
lfs_size_t block_size;
lfs_size_t block_count;
lfs_size_t name_max;
lfs_size_t inline_max;
lfs_size_t attr_max;
lfs_size_t file_max;
lfs_size_t attr_max;
} lfs_superblock_t;
// The littlefs filesystem type
@@ -398,9 +378,8 @@ typedef struct lfs {
const struct lfs_config *cfg;
lfs_size_t name_max;
lfs_size_t inline_max;
lfs_size_t attr_max;
lfs_size_t file_max;
lfs_size_t attr_max;
} lfs_t;