Skip to main content
Version: Latest

Checkpointing and restart

DES3D writes checkpoint files at regular intervals so that a simulation can be resumed after interruption. With unchanged controls, the restart tests compare saved physical state with uninterrupted runs. Use one OpenMP thread for byte-exact PT continuation; multi-thread ADR reductions can change the last bits. The following state is persisted in the checkpoint:

  • dt and dt_PT (time step)
  • max_global_vel_mag
  • reference_frame_time
  • info_display_next_step
  • dhacc (surface marker correction)
  • strain_rate, viscosity, volume_old
  • plastic strain-rate (delta_plstrain), so a restarted run's diagnostic output matches a continuous one

Copy the original config and edit its existing [sim] section to select a checkpoint explicitly, giving the restart a distinct output name:

[sim]
modelname = result-r
is_restarting = yes
restarting_from_modelname = result
restarting_from_frame = 10

Replace result and 10 with your original model prefix and an output frame that has both save and checkpoint files, then run the executable with that edited config. Checkpoint frame numbers are output-frame indices, not necessarily physical step numbers. Retain the original physical controls and set the stopping step/time to permit continuation. DES3D requires both restart-selection parameters; rerunning an unchanged fresh config does not automatically find a checkpoint.

PT restores saved velocity, next-step dt and velocity scale, and does not repeat completed initial equilibration. Its optional running scale requires the stored history; old checkpoints without it cannot silently start a new history. Hydraulic checkpoints also require the raw signed pressure increment and the saved ordering that determines whether it has already been consumed. Dry checkpoints do not require that hydraulic record. See PT restart and limits.

Checkpoint/output format revision 4

The checkpoint and frame file format was bumped from revision 3 to 4. A revision (or dimensionality) mismatch between the file and the running binary is now a hard error with a diagnostic naming the expected and actual values, previously it was accepted silently and could be misread. A checkpoint with the earlier revision cannot be restarted by the newer solver. Preserve the original executable, configuration and data to continue it. This restart restriction does not imply that every older saved field is unreadable by Python: Dynearthsol.py reads frame metadata and available fields separately, subject to the schema that reader supports.

Conservative MMG remeshing adds new checkpoint records

Checkpoints now also carry nremesh, plstrain_remesh, and, in MMG builds, surface dh and refined_elem_size_n (plus, with remeshing_option = 13, side_prof_*). A checkpoint written before this change does not have these records, and restarting from it is a restart-mismatch error (21/22). Continue such checkpoints with their matching executable and preserve the original data; the new state is not inferred automatically. See Conservative remeshing for what changed in MMG remeshing itself.

The optional finite-density hydraulic model also saves the physical inventories, numerical split buffer, EOS temperature origin and pending thermal/pressure increments. Accepted shear heat is saved as energy, so a changed timestep must not reinterpret an old power as a new energy. The current shear-heating checkpoint stores its version, pending extensive energy and accepted power together; missing members or an unsupported version are rejected. A power-only intermediate checkpoint is not converted. These model-specific guards apply in addition to the file-format revision. See Poromechanics.

Frame files (HDF5 and plain binary) now also embed the full .info row (steps, time, dt, walltime, node/element/segment counts). If a run's .info file is lost, utils/recreate_info.py rebuilds it from the frame files alone; restart() also falls back to this embedded metadata automatically when .info is missing.