Christopher Haster a43f9b3cd5 Modified lfs_dir_compact to avoid redundant erases during split ...

The commit machine in littlefs has three stages: commit, compact, and
then split. First we try to append our commit to the metadata log, if
that fails we try to compact the metadata log to remove duplicates and make
room for the commit, if that still fails we split the metadata into two
metadata-pairs and try again. Each stage is less efficient but also less
frequent.

However, in the case that we're filling up a directory with new files,
such as the bootstrap process in setting up a new system, we must pass
through all three stages rather quickly in order to get enough
metadata-pairs to hold all of our files. This means we'll compact,
split, and then need to compact again. This creates more erases than is
needed in the optimal case, which can be a big cost on disks with an
expensive erase operation.

In theory, we can actually avoid this redundant erase by reusing the
data we wrote out in the first attempt to compact. In practice, this
trick is very complicated to pull off.

1. We may need to cache a half-completed program while we write out the
   new metadata-pair. We need to write out the second pair first in
   order to get our new tail before we complete our first metadata-pair.
   This requires two pcaches, which we don't have

   The solution here is to just drop our cache and reconstruct what if
   would have been. This needs to be perfect down to the byte level
   because we don't have knowledge of where our cache lines are.

2. We may have written out entries that are then moved to the new
   metadata-pair.

   The solution here isn't pretty but it works, we just add a delete
   tag for any entry that was moved over.

In the end the solution ends up a bit hacky, with different layers poked
through the commit logic in order to manage writes at the byte level
from where we manage splits. But it works fairly well and saves erases.

2018-10-18 10:00:49 -05:00

100 KiB

Raw Blame History

View Raw