Testing a change
Compiling is not verification. Before a change is reported as done:
-
Build both dimensions.
make ndims=2
make ndims=3 -
Compare results with the parent revision whenever the change could alter them. In
tests/regression/, store a reference with the executable built from the parent commit, then rebuild with the change and compare:cd tests/regression
make set NDIMS=2 CASE=<case>.cfg # with the parent's dynearthsol2d
# rebuild the root executables with your change, then
make cmp NDIMS=2 CASE=<case>.cfgcompare.pyprints the relative difference of each field and fails above . A change that is meant to alter results should say so, with the case and the size of the difference, in the commit message and the pull request.make fresh-restart-cmpchecks that a restart reproduces an uninterrupted run.test-tiny.cfgis a one-second smoke test. -
Run the tests that cover the area you changed, for example the functional tests under
tests/functional/, the component checks undertests/residual-assembly-test/, or a benchmark underbenchmarks/. See Benchmarks for the list.
Pull requests run the same build matrix and functional tests in continuous integration, but CI cannot tell whether a change should have altered results; the pull-request template asks you to report the comparison.