Christopher Haster c7987a3162 Restructured .travis.yml to span more jobs ...

The core of littlefs's CI testing is the full test suite, `make test`, run
under a number of configurations:

- Processor architecture:
  - x86 (native)
  - Arm Thumb
  - MIPS
  - PowerPC
- Storage geometry:
  - rs=16   ps=16   cs=64   bs=512   (default)
  - rs=1    ps=1    cs=64   bs=4KiB  (NOR flash)
  - rs=512  ps=512  cs=512  bs=512   (eMMC)
  - rs=4KiB ps=4KiB cs=4KiB bs=32KiB (NAND flash)
- Other corner cases:
  - no intrinsics
  - no inline
  - byte-level read/writes
  - single block-cycles
  - odd block counts
  - odd block sizes

The number of different configurations we need to test quickly exceeds the
50 minute time limit Travis has on jobs. Fortunately, we can split these
tests out into multiple jobs. This seems to be the intended course of
action for large CI "builds" in Travis, as this gives Travis a finer
grain of control over limiting builds.

Unfortunately, this created a couple issues:

1. The Travis configuration isn't actually that flexible. It allows a
   single "matrix expansion" which can be generated from top-level lists
   of different configurations. But it doesn't let you generate a matrix
   from two seperate environment variable lists (for arch + geometry).

   Without multiple matrix expansions, we're stuck writing out each test
   permutation by hand.

   On the bright-side, this was a good chance to really learn how YAML
   anchors work. I'm torn because on one hand anchors add what feels
   like unnecessary complexity to a config language, on the other hand,
   they did help quite a bit in working around Travis's limitations.

2. Now that we have 47 jobs instead of 7, reporting a separate status
   for each job stops making sense.

   What I've opted for here is to use a special NAME variable to
   deduplicate jobs, and used a few state-less rules to hopefully have
   the reported status make sense most of the time.

   - Overwrite "pending" statuses so that the last job to start owns the
     most recent "pending" status
   - Don't overwrite "failure" statuses unless the job number matches
     our own (in the case of CI restarts)
   - Don't write "success" statuses unless the job number matches our
     own, this should delay a green check-mark until the last-to-start
     job finishes
   - Always overwrite non-failures with "failure" statuses

   This does mean a temporary "success" may appear if the last job
   terminates before earlier jobs. But this is the simpliest solution
   I can think of without storing some complex state somewhere.

   Note we can only report the size this way because it's cheap to
   calculate in every job.

2020-02-18 17:34:23 -06:00

16 KiB

Raw Blame History

View Raw