Added ./script/summary.py

A full summary of static measurements (code size, stack usage, etc) can now
be found with:

    make summary

This is done through the combination of a new ./scripts/summary.py
script and the ability of existing scripts to merge into existing csv
files, allowing multiple results to be merged either in a pipeline, or
in parallel with a single ./script/summary.py call.

The ./scripts/summary.py script can also be used to quickly compare
different builds or configurations. This is a proper implementation
of a similar but hacky shell script that has already been very useful
for making optimization decisions:

    $ ./scripts/structs.py new.csv -d old.csv --summary
    name (2 added, 0 removed)               code             stack            structs
    TOTAL                                  28648 (-2.7%)      2448               1012

Also some other small tweaks to scripts:

- Removed state saving diff rules. This isn't the most useful way to
  handle comparing changes.

- Added short flags for --summary (-Y) and --files (-F), since these
  are quite often used.
This commit is contained in:
Christopher Haster
2022-03-07 00:24:30 -06:00
parent eb8be9f351
commit 55b3c538d5
7 changed files with 580 additions and 110 deletions

View File

@@ -56,6 +56,9 @@ endif
ifdef EXEC
override TESTFLAGS += --exec="$(EXEC)"
endif
ifdef COVERAGE
override TESTFLAGS += --coverage
endif
ifdef BUILDDIR
override TESTFLAGS += --build-dir="$(BUILDDIR:/=)"
override CALLSFLAGS += --build-dir="$(BUILDDIR:/=)"
@@ -104,41 +107,34 @@ test%: tests/test$$(firstword $$(subst \#, ,%)).toml
code: $(OBJ)
./scripts/code.py $^ -S $(CODEFLAGS)
.PHONY: code-diff
code-diff: $(OBJ)
./scripts/code.py $^ -d $(TARGET).code.csv -o $(TARGET).code.csv $(CODEFLAGS)
.PHONY: data
data: $(OBJ)
./scripts/data.py $^ -S $(DATAFLAGS)
.PHONY: data-diff
data-diff: $(OBJ)
./scripts/data.py $^ -d $(TARGET).data.csv -o $(TARGET).data.csv $(DATAFLAGS)
.PHONY: stack
stack: $(CGI)
./scripts/stack.py $^ -S $(STACKFLAGS)
.PHONY: stack-diff
stack-diff: $(CGI)
./scripts/stack.py $^ -d $(TARGET).stack.csv -o $(TARGET).stack.csv $(STACKFLAGS)
.PHONY: structs
structs: $(OBJ)
./scripts/structs.py $^ -S $(STRUCTSFLAGS)
.PHONY: structs-diff
structs-diff: $(OBJ)
./scripts/structs.py $^ -d $(TARGET).structs.csv -o $(TARGET).structs.csv $(STRUCTSFLAGS)
.PHONY: coverage
coverage:
./scripts/coverage.py $(BUILDDIR)tests/*.toml.info -s $(COVERAGEFLAGS)
.PHONY: coverage-diff
coverage-diff:
./scripts/coverage.py $(BUILDDIR)tests/*.toml.info $(COVERAGEFLAGS)
.PHONY: summary
summary: $(OBJ) $(CGI)
$(strip \
./scripts/code.py $(OBJ) -q -o - $(CODEFLAGS) \
| ./scripts/data.py $(OBJ) -q -m - -o - $(DATAFLAGS) \
| ./scripts/stack.py $(CGI) -q -m - -o - $(STACKFLAGS) \
| ./scripts/structs.py $(OBJ) -q -m - -o - $(STRUCTFLAGS) \
$(if $(COVERAGE),\
| ./scripts/coverage.py $(BUILDDIR)tests/*.toml.info \
-q -m - -o - $(COVERAGEFLAGS)) \
| ./scripts/summary.py $(SUMMARYFLAGS))
# rules
-include $(DEP)