Added better documentation

More documentation may still by worthwhile (design documentation?),
but for now this provides a reasonable baseline.
- readme
- license
- header documentation

LICENSE.md

@@ -0,0 +1,19 @@
+Copyright (c) 2016 Christopher Haster
+
+Permission is hereby granted, free of charge, to any person obtaining a
+copy of this software and associated documentation files (the "Software"),
+to deal in the Software without restriction, including without limitation
+the rights to use, copy, modify, merge, publish, distribute, sublicense,
+and/or sell copies of the Software, and to permit persons to whom the
+Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.

README.md

@@ -0,0 +1,124 @@
+## The little filesystem
+
+A little fail-safe filesystem designed for low ram/rom footprint.
+
+**Fail-safe** - The littlefs is designed to work consistently with random power
+failures. During filesystem operations the storage on disk is always kept
+in a valid state. The filesystem also has strong copy-on-write garuntees.
+When updating a file, the original file will remain unmodified until the
+file is closed, or sync is called.
+
+**Handles bad blocks** - While the littlefs does not implement static wear
+leveling, if the underlying block device reports write errors, the littlefs
+uses a form of dynamic wear leveling to manage blocks that go bad during
+the lifetime of the filesystem.
+
+**Constrained memory** - The littlefs is designed to work in bounded memory,
+recursion is avoided, and dynamic memory is kept to a minimum. The littlefs
+allocates two fixed-size buffers for general operations, and one fixed-size
+buffer per file. If there is only ever one file in use, these buffers can be
+provided statically.
+
+## Example
+
+Here's a simple example that updates a file named `boot_count` every time
+main runs. The program can be interrupted at any time without losing track
+of how many times it has been booted and without corrupting the filesystem:
+
+``` c
+#include "lfs.h"
+
+// variables used by the filesystem
+lfs_t lfs;
+lfs_file_t file;
+
+// configuration of the filesystem is provided by this struct
+const struct lfs_config cfg = {
+    // block device operations
+    .read  = user_provided_block_device_read,
+    .prog  = user_provided_block_device_prog,
+    .erase = user_provided_block_device_erase,
+    .sync  = user_provided_block_device_sync,
+
+    // block device configuration
+    .read_size = 16,
+    .prog_size = 16,
+    .block_size = 4096,
+    .block_count = 128,
+    .lookahead = 128,
+};
+
+// entry point
+int main(void) {
+    // mount the filesystem
+    int err = lfs_mount(&lfs, &cfg);
+
+    // reformat if we can't mount the filesystem
+    // this should only happen on the first boot
+    if (err) {
+        lfs_format(&lfs, &cfg);
+        lfs_mount(&lfs, &cfg);
+    }
+
+    // read current count
+    uint32_t boot_count = 0;
+    lfs_file_open(&lfs, &file, "boot_count", LFS_O_RDWR | LFS_O_CREAT);
+    lfs_file_read(&lfs, &file, &boot_count, sizeof(boot_count));
+
+    // update boot count
+    boot_count += 1;
+    printf("boot_count: %ld\n", boot_count);
+    lfs_file_rewind(&lfs, &file);
+    lfs_file_write(&lfs, &file, &boot_count, sizeof(boot_count));
+
+    // remember the storage is not updated until the file is closed successfully
+    lfs_file_close(&lfs, &file);
+
+    // release and resources we were using
+    lfs_unmount(&lfs);
+}
+```
+
+## Usage
+
+Detailed documentation (or at least as much detail as is currently available)
+can be cound in the comments in [lfs.h](lfs.h).
+
+As you may have noticed, the littlefs takes in a configuration structure that
+defines how the filesystem operates. The configuration struct provides the
+filesystem with the block device operations and dimensions, tweakable
+parameters that tradeoff memory usage for performance, and optional
+static buffers if the user wants to avoid dynamic memory.
+
+The state of the littlefs is stored in the `lfs_t` type which is left up
+to the user to allocate, allowing multiple filesystems to be in use
+simultaneously. With the `lfs_t` and configuration struct, a user can either
+format a block device or mount the filesystem.
+
+Once mounted, the littlefs provides a full set of posix-like file and
+directory functions, with the deviation that the allocation of filesystem
+structures must be provided by the user. An important addition is that
+no file updates will actually be written to disk until a sync or close
+is called.
+
+## Other notes
+
+All littlefs have the potential to return a negative error code. The errors
+can be either one of those found in the `enum lfs_error` in [lfs.h](lfs.h),
+or an error returned by the user's block device operations.
+
+It should also be noted that the littlefs does not do anything to insure
+that the data written to disk is machine portable. It should be fine as
+long as the machines involved share endianness and don't have really
+strange padding requirements. If the question does come up, the littlefs
+metadata should be stored on disk in little-endian format.
+
+## Testing
+
+The littlefs comes with a test suite designed to run on a pc using the
+[emulated block device](emubd/lfs_emubd.h) found in the emubd directory.
+The tests assume a linux environment and can be started with make:
+
+``` bash
+make test
+```

lfs.h

@@ -11,6 +11,8 @@
 #include <stdbool.h>
 
 
+/// Definitions ///
+
 // Type definitions
 typedef uint32_t lfs_size_t;
 typedef uint32_t lfs_off_t;
@@ -20,91 +22,107 @@ typedef int32_t  lfs_soff_t;
 
 typedef uint32_t lfs_block_t;
 
-
+// Max name size in bytes
-// Configurable littlefs constants
 #ifndef LFS_NAME_MAX
 #define LFS_NAME_MAX 255
 #endif
 
-// The littefs constants
+// Possible error codes, these are negative to allow
+// valid positive return values
 enum lfs_error {
-    LFS_ERR_OK      = 0,
+    LFS_ERR_OK      = 0,    // No error
-    LFS_ERR_IO      = -5,
+    LFS_ERR_IO      = -5,   // Error during device operation
-    LFS_ERR_CORRUPT = -77,
+    LFS_ERR_CORRUPT = -77,  // Corrupted
-    LFS_ERR_NOENT   = -2,
+    LFS_ERR_NOENT   = -2,   // No directory entry
-    LFS_ERR_EXISTS  = -17,
+    LFS_ERR_EXISTS  = -17,  // Entry already exists
-    LFS_ERR_NOTDIR  = -20,
+    LFS_ERR_NOTDIR  = -20,  // Entry is not a dir
-    LFS_ERR_ISDIR   = -21,
+    LFS_ERR_ISDIR   = -21,  // Entry is a dir
-    LFS_ERR_INVAL   = -22,
+    LFS_ERR_INVAL   = -22,  // Invalid parameter
-    LFS_ERR_NOSPC   = -28,
+    LFS_ERR_NOSPC   = -28,  // No space left on device
-    LFS_ERR_NOMEM   = -12,
+    LFS_ERR_NOMEM   = -12,  // No more memory available
 };
 
+// File types
 enum lfs_type {
     LFS_TYPE_REG        = 0x01,
     LFS_TYPE_DIR        = 0x02,
     LFS_TYPE_SUPERBLOCK = 0x12,
 };
 
+// File open flags
 enum lfs_open_flags {
     // open flags
-    LFS_O_RDONLY = 1,
+    LFS_O_RDONLY = 1,        // Open a file as read only
-    LFS_O_WRONLY = 2,
+    LFS_O_WRONLY = 2,        // Open a file as write only
-    LFS_O_RDWR   = 3,
+    LFS_O_RDWR   = 3,        // Open a file as read and write
-    LFS_O_CREAT  = 0x0100,
+    LFS_O_CREAT  = 0x0100,   // Create a file if it does not exist
-    LFS_O_EXCL   = 0x0200,
+    LFS_O_EXCL   = 0x0200,   // Fail if a file already exists
-    LFS_O_TRUNC  = 0x0400,
+    LFS_O_TRUNC  = 0x0400,   // Truncate the existing file to zero size
-    LFS_O_APPEND = 0x0800,
+    LFS_O_APPEND = 0x0800,   // Move to end of file on every write
 
     // internally used flags
-    LFS_F_DIRTY   = 0x10000,
+    LFS_F_DIRTY   = 0x10000, // File does not match storage
-    LFS_F_WRITING = 0x20000,
+    LFS_F_WRITING = 0x20000, // File has been written since last flush
-    LFS_F_READING = 0x40000,
+    LFS_F_READING = 0x40000, // File has been read since last flush
 };
 
+// File seek flags
 enum lfs_whence_flags {
-    LFS_SEEK_SET = 0,
+    LFS_SEEK_SET = 0,   // Seek relative to an absolute position
-    LFS_SEEK_CUR = 1,
+    LFS_SEEK_CUR = 1,   // Seek relative to the current file position
-    LFS_SEEK_END = 2,
+    LFS_SEEK_END = 2,   // Seek relative to the end of the file
 };
 
 
 // Configuration provided during initialization of the littlefs
 struct lfs_config {
-    // Opaque user provided context
+    // Opaque user provided context that can be used to pass
+    // information to the block device operations
     void *context;
 
-    // Read a region in a block
+    // Read a region in a block. Negative error codes are propogated
+    // to the user.
     int (*read)(const struct lfs_config *c, lfs_block_t block,
             lfs_off_t off, void *buffer, lfs_size_t size);
 
     // Program a region in a block. The block must have previously
-    // been erased.
+    // been erased. Negative error codes are propogated to the user.
+    // The prog function must return LFS_ERR_CORRUPT if the block should
+    // be considered bad.
     int (*prog)(const struct lfs_config *c, lfs_block_t block,
             lfs_off_t off, const void *buffer, lfs_size_t size);
 
     // Erase a block. A block must be erased before being programmed.
-    // The state of an erased block is undefined.
+    // The state of an erased block is undefined. Negative error codes
+    // are propogated to the user.
     int (*erase)(const struct lfs_config *c, lfs_block_t block);
 
-    // Sync the state of the underlying block device
+    // Sync the state of the underlying block device. Negative error codes
+    // are propogated to the user.
     int (*sync)(const struct lfs_config *c);
 
-    // Minimum size of a read. This may be larger than the physical
+    // Minimum size of a block read. This determines the size of read buffers.
-    // read size to cache reads from the block device.
+    // This may be larger than the physical read size to improve performance
+    // by caching more of the block device.
     lfs_size_t read_size;
 
-    // Minimum size of a program. This may be larger than the physical
+    // Minimum size of a block program. This determines the size of program
-    // program size to cache programs to the block device.
+    // buffers. This may be larger than the physical program size to improve
+    // performance by caching more of the block device.
     lfs_size_t prog_size;
 
-    // Size of an erasable block.
+    // Size of an erasable block. This does not impact ram consumption and
+    // may be larger than the physical erase size. However, this should be
+    // kept small as each file currently takes up an entire block .
     lfs_size_t block_size;
 
     // Number of erasable blocks on the device.
     lfs_size_t block_count;
 
-    // Number of blocks to lookahead during block allocation.
+    // Number of blocks to lookahead during block allocation. A larger
+    // lookahead reduces the number of passes required to allocate a block.
+    // The lookahead buffer requires only 1 bit per block so it can be quite
+    // large with little ram impact. Should be a multiple of 32.
     lfs_size_t lookahead;
 
     // Optional, statically allocated read buffer. Must be read sized.
@@ -113,18 +131,19 @@ struct lfs_config {
     // Optional, statically allocated program buffer. Must be program sized.
     void *prog_buffer;
 
-    // Optional, statically allocated lookahead buffer.
+    // Optional, statically allocated lookahead buffer. Must be 1 bit per
-    // Must be 1 bit per lookahead block.
+    // lookahead block.
     void *lookahead_buffer;
 
     // Optional, statically allocated buffer for files. Must be program sized.
-    // If enabled, only one file may be opened at a time
+    // If enabled, only one file may be opened at a time.
     void *file_buffer;
 };
 
+
 // File info structure
 struct lfs_info {
-    // Type of the file, either REG or DIR
+    // Type of the file, either LFS_TYPE_REG or LFS_TYPE_DIR
     uint8_t type;
 
     // Size of the file, only valid for REG files
@@ -135,7 +154,7 @@ struct lfs_info {
 };
 
 
-// littlefs data structures
+/// littlefs data structures ///
 typedef struct lfs_entry {
     lfs_off_t off;
 
@@ -208,7 +227,7 @@ typedef struct lfs_free {
     uint32_t *lookahead;
 } lfs_free_t;
 
-// littlefs type
+// The littlefs type
 typedef struct lfs {
     const struct lfs_config *cfg;
 
@@ -223,42 +242,191 @@ typedef struct lfs {
 } lfs_t;
 
 
-// filesystem functions
+/// Filesystem functions ///
+
+// Format a block device with the littlefs
+//
+// Requires a littlefs object and config struct. This clobbers the littlefs
+// object, and does not leave the filesystem mounted.
+//
+// Returns a negative error code on failure.
 int lfs_format(lfs_t *lfs, const struct lfs_config *config);
+
+// Mounts a littlefs
+//
+// Requires a littlefs object and config struct. Multiple filesystems
+// may be mounted simultaneously with multiple littlefs objects. Both
+// lfs and config must be allocated while mounted.
+//
+// Returns a negative error code on failure.
 int lfs_mount(lfs_t *lfs, const struct lfs_config *config);
+
+// Unmounts a littlefs
+//
+// Does nothing besides releasing any allocated resources.
+// Returns a negative error code on failure.
 int lfs_unmount(lfs_t *lfs);
 
-// general operations
+/// General operations ///
+
+// Removes a file or directory
+//
+// If removing a directory, the directory must be empty.
+// Returns a negative error code on failure.
 int lfs_remove(lfs_t *lfs, const char *path);
+
+// Rename or move a file or directory
+//
+// If the destination exists, it must match the source in type.
+// If the destination is a directory, the directory must be empty.
+//
+// Note: If power loss occurs, it is possible that the file or directory
+// will exist in both the oldpath and newpath simultaneously after the
+// next mount.
+//
+// Returns a negative error code on failure.
 int lfs_rename(lfs_t *lfs, const char *oldpath, const char *newpath);
+
+// Find info about a file or directory
+//
+// Fills out the info structure, based on the specified file or directory.
+// Returns a negative error code on failure.
 int lfs_stat(lfs_t *lfs, const char *path, struct lfs_info *info);
 
-// directory operations
-int lfs_mkdir(lfs_t *lfs, const char *path);
-int lfs_dir_open(lfs_t *lfs, lfs_dir_t *dir, const char *path);
-int lfs_dir_close(lfs_t *lfs, lfs_dir_t *dir);
-int lfs_dir_read(lfs_t *lfs, lfs_dir_t *dir, struct lfs_info *info);
-int lfs_dir_seek(lfs_t *lfs, lfs_dir_t *dir, lfs_off_t off);
-lfs_soff_t lfs_dir_tell(lfs_t *lfs, lfs_dir_t *dir);
-int lfs_dir_rewind(lfs_t *lfs, lfs_dir_t *dir);
 
-// file operations
+/// File operations ///
+
+// Open a file
+//
+// The mode that the file is opened in is determined
+// by the flags, which are values from the enum lfs_open_flags
+// that are bitwise-ored together.
+//
+// Returns a negative error code on failure.
 int lfs_file_open(lfs_t *lfs, lfs_file_t *file,
         const char *path, int flags);
+
+// Close a file
+//
+// Any pending writes are written out to storage as though
+// sync had been called and releases any allocated resources.
+//
+// Returns a negative error code on failure.
 int lfs_file_close(lfs_t *lfs, lfs_file_t *file);
+
+// Synchronize a file on storage
+//
+// Any pending writes are written out to storage.
+// Returns a negative error code on failure.
 int lfs_file_sync(lfs_t *lfs, lfs_file_t *file);
+
+// Read data from file
+//
+// Takes a buffer and size indicating where to store the read data.
+// Returns the number of bytes read, or a negative error code on failure.
 lfs_ssize_t lfs_file_read(lfs_t *lfs, lfs_file_t *file,
         void *buffer, lfs_size_t size);
+
+// Write data to file
+//
+// Takes a buffer and size indicating the data to write. The file will not
+// actually be updated on the storage until either sync or close is called.
+//
+// Returns the number of bytes written, or a negative error code on failure.
 lfs_ssize_t lfs_file_write(lfs_t *lfs, lfs_file_t *file,
         const void *buffer, lfs_size_t size);
+
+// Change the position of the file
+//
+// The change in position is determined by the offset and whence flag.
+// Returns the old position of the file, or a negative error code on failure.
 lfs_soff_t lfs_file_seek(lfs_t *lfs, lfs_file_t *file,
         lfs_soff_t off, int whence);
+
+// Return the position of the file
+//
+// Equivalent to lfs_file_seek(lfs, file, 0, LFS_SEEK_CUR)
+// Returns the position of the file, or a negative error code on failure.
 lfs_soff_t lfs_file_tell(lfs_t *lfs, lfs_file_t *file);
+
+// Change the position of the file to the beginning of the file
+//
+// Equivalent to lfs_file_seek(lfs, file, 0, LFS_SEEK_CUR)
+// Returns a negative error code on failure.
 int lfs_file_rewind(lfs_t *lfs, lfs_file_t *file);
+
+// Return the size of the file
+//
+// Similar to lfs_file_seek(lfs, file, 0, LFS_SEEK_END)
+// Returns the size of the file, or a negative error code on failure.
 lfs_soff_t lfs_file_size(lfs_t *lfs, lfs_file_t *file);
 
-// miscellaneous lfs specific operations
+
+/// Directory operations ///
+
+// Create a directory
+//
+// Returns a negative error code on failure.
+int lfs_mkdir(lfs_t *lfs, const char *path);
+
+// Open a directory
+//
+// Once open a directory can be used with read to iterate over files.
+// Returns a negative error code on failure.
+int lfs_dir_open(lfs_t *lfs, lfs_dir_t *dir, const char *path);
+
+// Close a directory
+//
+// Releases any allocated resources.
+// Returns a negative error code on failure.
+int lfs_dir_close(lfs_t *lfs, lfs_dir_t *dir);
+
+// Read an entry in the directory
+//
+// Fills out the info structure, based on the specified file or directory.
+// Returns a negative error code on failure.
+int lfs_dir_read(lfs_t *lfs, lfs_dir_t *dir, struct lfs_info *info);
+
+// Change the position of the directory
+//
+// The new off must be a value previous returned from tell and specifies
+// an absolute offset in the directory seek.
+//
+// Returns a negative error code on failure.
+int lfs_dir_seek(lfs_t *lfs, lfs_dir_t *dir, lfs_off_t off);
+
+// Return the position of the directory
+//
+// The returned offset is only meant to be consumed by seek and may not make
+// sense, but does indicate the current position in the directory iteration.
+//
+// Returns the position of the directory, or a negative error code on failure.
+lfs_soff_t lfs_dir_tell(lfs_t *lfs, lfs_dir_t *dir);
+
+// Change the position of the directory to the beginning of the directory
+//
+// Returns a negative error code on failure.
+int lfs_dir_rewind(lfs_t *lfs, lfs_dir_t *dir);
+
+
+/// Miscellaneous littlefs specific operations ///
+
+// Traverse through all blocks in use by the filesystem
+//
+// The provided callback will be called with each block address that is
+// currently in use by the filesystem. This can be used to determine which
+// blocks are in use or how much of the storage is available.
+//
+// Returns a negative error code on failure.
 int lfs_traverse(lfs_t *lfs, int (*cb)(void*, lfs_block_t), void *data);
+
+// Prunes any recoverable errors that may have occured in the filesystem
+//
+// Not needed to be called by user unless an operation is interrupted
+// but the filesystem is still mounted. This is already called on first
+// allocation.
+//
+// Returns a negative error code on failure.
 int lfs_deorphan(lfs_t *lfs);