Branch off AWI-CM3 from fesom raw and binary restart fails

[FinnHeu]

Hi,

I am branching off an AWI-CM3-develop simulation from an existing spin-up simulation.

I thus add the raw restart directories/files from the spinup to the fesom block in my runscript:

fesom:
    [...]
    choose_general.run_number:
      1:
        restart_in_sources:
            fesom_raw_restart_info: "/work/ab0995/a270186/model_inputs/awicm3/pool/restarts/TUNE42PI_FES27_1629/fesom_raw_restart/${nproc}.info"
            fesom_raw_restart: "/work/ab0995/a270186/model_inputs/awicm3/pool/restarts/TUNE42PI_FES27_1629/fesom_raw_restart/np${nproc}/*"

However, despite the additions in the runscript, the raw restart files are not copied to /work and thus the run fails as it can not find the run.../work/fesom_raw_restart/.... The same thing occurred when linking in the binary restarts with fesom_bin_restart_info and fesom_bin_restart.

When I instead use netcdf restarts:

choose_general.run_number:
      1:
        restart_in_sources:
            par_oce_restart: "/work/bb1469/a270092/runtime/awicm3-develop/TUNE42PI_FES27/run_16200101-16291231/work/fesom.1620.oce.restart/*.nc"
            par_ice_restart: "/work/bb1469/a270092/runtime/awicm3-develop/TUNE42PI_FES27/run_16200101-16291231/work/fesom.1620.ice.restart/*.nc"

the restart files are successfully copied to run.../work/fesom.*.restart/*.nc.
I have to use the raw restarts, though. The netcdfs have the wrong timestamp.

Is my attempt with the raw restarts incorrect or is esm tools not yet set up to handle the raw/binary restarts at the branch off?

I am working on esm-tools branch origin/feat/awiesm3-v3.4-co2.
Cheers,
Finn

[mandresm]

Dear @FinnHeu,

Have you changed the number of nproc from the parent simulation? If that is the case, that would explain why files are not copied since the specified files would not exists as their paths contain nproc.

Even if you'd manage to copy those files by hardcoding the new nproc in the paths, fesom would not be able to restart with the raw restarts, as raw restarts are memory dumps and one file is required per MPI process. In that case, you'd need to use the old nproc number or restart from the nc restarts.

If nproc is the same in the new simulation, then I need to further investigate. In that case could you provide a path to the new experiment?