Added allocation randomization for dynamic wear-leveling

This implements the second step of full dynamic wear-leveling, block
allocation randomization. This is the key part the uniformly distributes
wear across the filesystem, even through reboots.

The entropy actually comes from the filesystem itself, by xoring
together all of the CRCs in the metadata-pairs on the filesystem. While
this sounds like a ridiculous operation, it's easy to do when we already
scan the metadata-pairs at mount time.

This gives us a random number we can use for block allocation.
Unfortunately it's not a great general purpose random generator as the
output only changes every filesystem write. Fortunately that's exactly
when we need our allocator.

---

Additionally, the randomization created a mess for the testing
framework. Fortunately, this method of randomization is deterministic.
A very useful property for reproducing bugs.
This commit is contained in:
Christopher Haster
2018-08-09 09:06:17 -05:00
parent e4a0d586d5
commit 126ef8b07f
10 changed files with 250 additions and 59 deletions

View File

@@ -3,37 +3,41 @@
import struct
import sys
import os
import argparse
def main(*paths):
# find most recent block
file = None
rev = None
for path in paths:
try:
nfile = open(path, 'r+b')
nrev, = struct.unpack('<I', nfile.read(4))
def corrupt(block):
with open(block, 'r+b') as file:
# skip rev
file.read(4)
assert rev != nrev
if not file or ((rev - nrev) & 0x80000000):
file = nfile
rev = nrev
except IOError:
pass
# go to last commit
tag = 0
while True:
try:
ntag, = struct.unpack('<I', file.read(4))
except struct.error:
break
# go to last commit
tag = 0
while True:
try:
ntag, = struct.unpack('<I', file.read(4))
except struct.error:
break
tag ^= ntag
file.seek(tag & 0xfff, os.SEEK_CUR)
tag ^= ntag
file.seek(tag & 0xfff, os.SEEK_CUR)
# lob off last 3 bytes
file.seek(-((tag & 0xfff) + 3), os.SEEK_CUR)
file.truncate()
# lob off last 3 bytes
file.seek(-((tag & 0xfff) + 3), os.SEEK_CUR)
file.truncate()
def main(args):
if args.n or not args.blocks:
with open('blocks/.history', 'rb') as file:
for i in range(int(args.n or 1)):
last, = struct.unpack('<I', file.read(4))
args.blocks.append('blocks/%x' % last)
for block in args.blocks:
print 'corrupting %s' % block
corrupt(block)
if __name__ == "__main__":
main(*sys.argv[1:])
parser = argparse.ArgumentParser()
parser.add_argument('-n')
parser.add_argument('blocks', nargs='*')
main(parser.parse_args())