merge geomar dev into latest release #1352

seb-wahl · 2025-05-14T06:55:13Z

This merge request has the following changes. I started by merging v6.54.5 into geomar_dev:

adjusted all the yaml files after the removal of the environment_changes stuff.
allow more flexibility in echam when it comes to restarts: the output streams and restart streams can now differ. I defined the default output streams to match the restart streams. Removes a lot of confusing warnings for our setups
fixed a bug with a dr_hook_...* var not found anymore after the removal of the environment_changes stuff.
remove deprecated env var declarations that caused a lot of warnings
updated slurm.yaml to allow for the usage of mpirun (which we currently need on glogin), please check but I can't see any problems merging this
included our modifications to oasis.py. This needs to be reviewed carefully. Please see my comments in oasis.py.
includes foci-mops_lmu, a setup that includes an updated version of JSBACH 3.20p1 used and LMU Munich and now in Kiel within the RESCUE project. It allows to simulate carbon dioxide removal methods in JSBACH using a 12th tile. This means I had to make the static 11 tiles in jsbach.yaml a variable ${tiles}

Known bugs:

a -finished_config.yaml remains in the directory where I run esm_master and I get the following "error" message:

Build command finished on Wed May 14 08:37:43 2025.
/mnt/vast-nhr/home/sebastian.wahl01/u11179/esm/models/test/foci-mops_lmu/nemo-ORCA05_LIM2_FOCI_MOPS_OASISMCT4/CONFIG
/mnt/vast-nhr/home/sebastian.wahl01/u11179/esm/models/test/foci-mops_lmu
Problems moving ``./foci-mops_lmu-finished_config.yaml`` to ``./foci-mops_lmu/finished_config.yaml``
(esm-tools) [shk00060] u11179@glogin6 test $ ls
-finished_config.yaml  foci-mops_lmu

compiles and starts, but misses (as expected) information in existing JSBACH restart files. Cold start not yet implemented

links all currently known files correctly running not yet tested as info from Shradda is still missing

draft version of ECHAM76-LMU that start up correctly but still fails due to land sea mask mismatch

fixed restart setup in a hacky way (see foci.yaml) added a new runscript for FOCIOIFS

…geomar_dev exludes the conflicting oasis.py

This should be the only difference between geomar_dev and release that matters

…on glogin

allow different streams for restart and output for ECHAM and JSBACH

with FOCI-MOPS with the 12 tile edition

merge geomar dev into geomar branch

separate echam6 namlist templates for foci and foci_lmu

before merging this branch into geomar_dev differences where too large for an efficient direct merge

…ration

see runscripts/foci/README_important.txt for details add temraw output for SOLVe project in foci-moz

compiling, running and restart now work

added missing coupling dir for foci-moz

…-lmod setup on glogin

… to CMakeLists.txt) XIOS still failing fixed RESCUE postprocessing by adding the mandatory new option -l and -a to echam_postprocessing_RESCUE.sh that came in when MOZ was added

mandresm

@seb-wahl, as mentioned in our in-person meeting, I finished this review even if this is not meant to be merged in release so that we can use some of this comments as a basis for the PR og geomar branch into release.

mandresm · 2025-11-24T06:20:08Z

configs/components/icon/icon.yaml

+            active: ${lcarbon}
+
+   choose_jsbach_with_hd:
+      yes:


Would it make sense to change this to true/false? Just to remain consistent to all of our other switches that normally use booleans.

mandresm · 2025-11-24T06:26:01Z

configs/components/icon/icon.yaml

+               choose_icon.version:
+                  2.6.6-nwp:
+                     input_dir: /pool/data/ICON
+                     icon_pool: ${input_dir}/grids/private/mpim/icon_preprocessing/source
+                     grid_dir: ${input_dir}/grids
+                     aerosol_dir: ${icon_pool}/aerosol_29-06-2018/kinne/${resolution}_${icon.grid_label}
+                     jsbach_ic_dir: ${input_dir}/grids/private/jsbach/mpim/${grid_id}/land/r0002
+                     jsbach_forcing_dir: ${jsbach_ic_dir}


I would move most of this block into the icon section itself, instead of keeping it inside computer. The only parameter that seems computer-dependent is input_dir and that could probably be made independent by making it be:

input_dir: ${computer.pool_dir}/ICON

mandresm · 2025-11-24T06:29:54Z

configs/components/oifs/oifs.yaml

        destination: oifs-48r1
        with_xios: false
        major_version: 48r1
+


Suggested change

mandresm · 2025-11-24T07:00:06Z

configs/components/oifs/oifs.yaml

+            uwavein: uwavein
+            specwavein: specwavein
+            sfcwindin: sfcwindin
+            cdwavein: cdwavein


Suggested change

uwavein: uwavein

specwavein: specwavein

sfcwindin: sfcwindin

cdwavein: cdwavein

These files shouldn't be needed here as they are added when wam is turned on:

esm_tools/configs/components/oifs/oifs.yaml

Lines 915 to 927 in 51ef371

choose_wam:

1:

wam_number: "1"

add_input_files:

wam_grid_tables: wam_grid_tables

wam_subgrid_0: wam_subgrid_0

wam_subgrid_1: wam_subgrid_1

wam_subgrid_2: wam_subgrid_2

uwavein: uwavein

specwavein: specwavein

sfcwindin: sfcwindin

cdwavein: cdwavein

mandresm · 2025-11-24T07:05:37Z

configs/components/oifs/oifs43.env.yaml

-            OIFS_CFLAGS: '"-fp-model precise -O3 -g -traceback -qopt-report=0 -fpe0 -qopenmp -march=core-avx2 -mtune=core-avx2"' # -qoverride-limits -fast-transcendentals -m64 -fma -pc64"'
-            OIFS_CCDEFS: '"LINUX LITTLE INTEGER_IS_INT _ABI64 BLAS _OPENMP"'
+          choose_compiler_mpi: 
+            intel2022_openmpi:


check that this does not break anything for AWICM3 (in the esm_tests in GitHub)

mandresm · 2025-11-24T07:56:55Z

src/esm_parser/dict_to_yaml.py

+                # due to duplicate namelists (occur with ICON) we get an 
+                # IndexError: list index out of range
+                # workaround by seb-wahl
+                # TODO: find a better solution
+                provenance_comment = f"no provenance info"
+                if 0 <= indx < len(config): 
+                    provenance = getattr(config[indx], "provenance", [None])[-1]
+                    if provenance:
+                        provenance_comment = f"{provenance['yaml_file']},line:{provenance['line']},col:{provenance['col']}"


I need to debug this, there might be a way of getting rid of this problem. It's strange that this does not happen for ECHAM

mandresm · 2025-11-24T08:14:58Z

src/esm_runscripts/batch_system.py

+        # Search for ``post_run_commands``s in the components
+        for component in config.keys():
+            post_run_commands = config[component].get("post_run_commands")
+            if isinstance(post_run_commands, list):
+                for pr_command in post_run_commands:
+                    if isinstance(pr_command, str):
+                        extras.append(pr_command)
+                    else:
+                        user_error(
+                            'Invalid type for "post_run_commands"',
+                            (
+                                f'"{type(pr_command)}" type is not supported for '
+                                + f'elements of the "post_run_commands", defined in '
+                                + f'"{component}". Please, define '
+                                + '"post_run_commands" as a "list" of "strings" or a "list".'
+                            ),
+                        )
+            elif isinstance(post_run_commands, str):
+                extras.append(post_run_commands)
+            elif post_run_commands == None:
+                continue
+            else:
+                user_error(
+                    'Invalid type for "post_run_commands"',
+                    (
+                        f'"{type(post_run_commands)}" type is not supported for '
+                        + f'"post_run_commands" defined in "{component}". Please, define '
+                        + '"post_run_commands" as a "string" or a "list" of "strings".'
+                    ),
+                )
+        return extras


Suggested change

# Search for ``post_run_commands``s in the components

for component in config.keys():

post_run_commands = config[component].get("post_run_commands")

if isinstance(post_run_commands, list):

for pr_command in post_run_commands:

if isinstance(pr_command, str):

extras.append(pr_command)

else:

user_error(

'Invalid type for "post_run_commands"',

(

f'"{type(pr_command)}" type is not supported for '

+ f'elements of the "post_run_commands", defined in '

+ f'"{component}". Please, define '

+ '"post_run_commands" as a "list" of "strings" or a "list".'

),

)

elif isinstance(post_run_commands, str):

extras.append(post_run_commands)

elif post_run_commands == None:

continue

else:

user_error(

'Invalid type for "post_run_commands"',

(

f'"{type(post_run_commands)}" type is not supported for '

+ f'"post_run_commands" defined in "{component}". Please, define '

+ '"post_run_commands" as a "string" or a "list" of "strings".'

),

)

return extras

# Search for ``post_run_commands``s in the components

for component in config.keys():

post_run_commands = config[component].get("post_run_commands")

if isinstance(post_run_commands, str):

post_run_commands = [post_run_commands]

elif isinstance(post_run_commands, list):

pass

elif post_run_commands == None:

continue

else:

user_error(

'Invalid type for "post_run_commands"',

(

f'"{type(post_run_commands)}" type is not supported for '

+ f'"post_run_commands" defined in "{component}". Please, define '

+ '"post_run_commands" as a "string" or a "list" of "strings".'

),

)

for pr_command in post_run_commands:

if isinstance(pr_command, str):

extras.append(pr_command)

else:

user_error(

'Invalid type for "post_run_commands"',

(

f'"{type(pr_command)}" type is not supported for '

+ f'elements of the "post_run_commands", defined in '

+ f'"{component}". Please, define '

+ '"post_run_commands" as a "list" of "strings" or a "list".'

),

)

return extras

TODO for @mandresm: make it the same for pre_run_commands and generalize it in a single function that can be called both for pre and post run.

mandresm · 2025-11-24T08:18:53Z

src/esm_runscripts/batch_system.py

                ]["next_submit"]

+                # extra entries for each subjob
+                post_run_commands = batch_system.get_post_run_commands(config)


Suggested change

post_run_commands = batch_system.get_post_run_commands(config)

post_run_commands = batch_system.get_commands(config, "post_run_commands")

TODO for @mandresm: change it to something like this.

mandresm · 2025-11-24T08:45:28Z

src/esm_runscripts/coupler.py

+            self.process_ordering = full_config[name]["process_ordering"]
+            self.coupled_execs = []
+            for exe in self.process_ordering:
+                self.coupled_execs.append(full_config[exe]["executable"])
+            self.runtime = full_config["general"]["runtime"][5]
+            self.nb_of_couplings = 0
+            if "coupling_target_fields" in full_config[self.name]:
+                for restart_file in list(full_config[self.name]["coupling_target_fields"]):
+                    self.nb_of_couplings += len(
+                        list(full_config[self.name]["coupling_target_fields"][restart_file])
+                    )
+            if "coupling_input_fields" in full_config[self.name]:
+                for restart_file in list(full_config[self.name]["coupling_input_fields"]):
+                    self.nb_of_couplings += len(
+                        list(full_config[self.name]["coupling_input_fields"])
+                    )


TODO for @mandresm: if this belongs exclusively to oasis, move it to the oasis.py

mandresm · 2025-11-24T08:49:41Z

src/esm_runscripts/coupler.py

-        if coupler_name == "yac":
-            couplingfile = "coupling.xml"
-        else:
+        couplingfile = None


TODO @mandresm: check for attributes in self. If none, then just return (make it oasis independend)

!!! nproma = 32 in runscripts/foci/RESCUE/foci-mops-lmu-scenario.yaml !!!

seb-wahl and others added 30 commits December 10, 2024 12:17

first draft of FOCI-MOPS with ECHAM6 from LMU Munich

408eb64

compiles and starts, but misses (as expected) information in existing JSBACH restart files. Cold start not yet implemented

added a first draft of runscript for LMU edition

d30a261

links all currently known files correctly running not yet tested as info from Shradda is still missing

made number of tiles a variable

024c4d7

added missing info in xios.yaml

1945cd8

draft version of ECHAM76-LMU that start up correctly but still fails due to land sea mask mismatch

added namelists templates (or links)

ef7f705

added missing ;

102d1a2

fixed dynveg file placeholder

b75f091

fix echam lmu compilation when submodule has already been cloned

e6f624b

latest version of LMU runscript, still crashes

1f455e2

FOCI-MOPS LMU runs including updated output for jsbach and echam

a2dbbf4

fixed restart setup in a hacky way (see foci.yaml) added a new runscript for FOCIOIFS

Merge remote-tracking branch 'origin/feature/add-foci-mops-lmu' into …

03a6565

…geomar_dev exludes the conflicting oasis.py

manually add the modifications in oasis.py again

0b20a63

This should be the only difference between geomar_dev and release that matters

fixed merge error in foci.yaml

d3dd469

renamed nest restart labels to fix Sayantanis restart issues

957fcdc

added Joakims additions to slurm.py which are required to use mpirun …

88bd5cb

…on glogin

added hotfix for oasis_time_stamp support needed by focioifs

2556665

added postprocessing for RESCUE (temporary, should go somewhere else)

a5cd423

allow different streams for restart and output for ECHAM and JSBACH

fix paths in echam_monitoring

7ae29ad

fixed all bugs that occurred before the first longer run

8c11b6b

with FOCI-MOPS with the 12 tile edition

added working FOCILMU piCtl setup

802e5a8

updated pp echam RESCUE, now netCDF

0431584

increased compression to zip_3

ebf5633

Merge pull request #1350 from esm-tools/geomar_dev

2eb0efa

merge geomar dev into geomar branch

separate echam6 namelist templates for foci and foci_lmu

3c66177

updated settings for NEMO4 standalone done a loooong time ago

79edfd9

Merge pull request #1351 from esm-tools/geomar_dev

9f2de6f

separate echam6 namlist templates for foci and foci_lmu

manually resolve conflicts between release and geomar_dev

4d7c6b3

before merging this branch into geomar_dev differences where too large for an efficient direct merge

manual merge of selected yaml from release to avoid conflicts

9b454b7

Merge branch 'release_geomar_dev_intermediate' into geomar_dev

0d93f0f

fixed indentation bug in jsbach.yaml, remove deprecated env var decla…

ea78915

…ration

seb-wahl force-pushed the geomar_dev branch from 83f3c2a to 9088777 Compare September 26, 2025 13:18

seb-wahl added 21 commits September 26, 2025 16:50

IMPORTANT: CONSERV coupling vs. GAUSWGT in foci-moz and foc-default_mct5

40cff5a

see runscripts/foci/README_important.txt for details add temraw output for SOLVe project in foci-moz

turn error into warning for icon-fesom for now

945c370

updated default settings, namelist.echam for moz updated

60a1dc7

hopefully fixed the ICON restart issue

9bf7d1d

WIP: updated restart settings

a1ba382

WIP: updated restart settings 2

9a5da38

fixed whitespace in icon.yaml

cf1898a

bugfixes, cleanup

171d216

compiling, running and restart now work

change timestep to 900s, switch to Intel

6f55cb8

Merge remote-tracking branch 'origin/release' into feature/icon-fesom

a8dbcf4

added file movements for FESOM, added documentation

7d7acfe

fixed typo

193e0a0

updated ICON-FESOM instructions

2899ac6

added compilation support for moz with MCT2.8

3909675

added missing coupling dir for foci-moz

fixed typos for moz mct2 autotools compile

9ef85fe

revert compile settings for echam with mct4 or 5

777c693

Merge branch 'hotfix/echam_compile' into geomar_dev

bbc064b

added support for intel 2021 classic without using the deprecated nhr…

0901df5

…-lmod setup on glogin

added preliminary support for intel2025, echam compiles (with changes…

51ef371

… to CMakeLists.txt) XIOS still failing fixed RESCUE postprocessing by adding the mandatory new option -l and -a to echam_postprocessing_RESCUE.sh that came in when MOZ was added

set default compiler for icon-fesom correctly

9e6cdb8

set a reasonable value for pool_dir

1556af8

mandresm requested changes Nov 24, 2025

View reviewed changes

seb-wahl added 7 commits November 27, 2025 13:24

copying instead of linking namelists

ebfb48e

fixed bug in atm and land file tags

9773f5d

Merge remote-tracking branch 'origin/geomar_dev' into feature/echam6-moz

f757a43

fix duplicate entry atm_file_tags

7e58319

removed surf from moz land_file_tags

1252cd8

remove surf instead of yasso (see prev commit)

4c5fc2d

Updated settings for RESCUE

af9dc54

!!! nproma = 32 in runscripts/foci/RESCUE/foci-mops-lmu-scenario.yaml !!!

	choose_wam:
	1:
	wam_number: "1"

	add_input_files:
	wam_grid_tables: wam_grid_tables
	wam_subgrid_0: wam_subgrid_0
	wam_subgrid_1: wam_subgrid_1
	wam_subgrid_2: wam_subgrid_2
	uwavein: uwavein
	specwavein: specwavein
	sfcwindin: sfcwindin
	cdwavein: cdwavein

	post_run_commands = batch_system.get_post_run_commands(config)
	post_run_commands = batch_system.get_commands(config, "post_run_commands")

merge geomar dev into latest release #1352

Are you sure you want to change the base?

merge geomar dev into latest release #1352

Uh oh!

Conversation

seb-wahl commented May 14, 2025

Uh oh!

mandresm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants