Save gnu warnings in rt logfile. Add WARN_AS_ERROR build option // Update ORT input data directories (#3138) by DusanJovic-NOAA · Pull Request #3129 · ufs-community/ufs-weather-model

DusanJovic-NOAA · 2026-03-06T14:38:02Z

Commit Queue Requirements:

This PR addresses a relevant WM issue (if not, create an issue).
All subcomponent pull requests (if any) have been reviewed by their code managers.
Run the full Intel+GNU RT suite (compared to current baselines), preferably on Ursa (Derecho or Hercules are acceptable alternatives). Exceptions: documentation-only PRs, CI-only PRs, etc.
- Commit log file w/full results from RT suite run (if applicable).
- Verify that test_changes.list indicates which tests, if any, are changed by this PR. Commit test_changes.list, even if it is empty.
Fill out all sections of this template.

Description:

#3129

Update rt.sh to save number of gnu compile warnings to log files.
Add new compile option WARN_AS_ERROR (disabled by default) which, when enabled, will treat all warnings as error.
rt.conf is updated to enable this option for tests that currently build with 0 warnings.

#3138

Sync the ORT input data directories to match with rt.sh along with a typo fix within Jenkinsfile.ort.

Commit Message:

* UFSWM - Save gnu warnings in rt logfile. Add WARN_AS_ERROR build option
* UFSWM - update ORT input data directories and Jenkinsfile ORT typo fix

Priority:

Critical Bugfix: Reason
High: Reason
Normal

Git Tracking

UFSWM:

Sub component Pull Requests:

None

UFSWM Blocking Dependencies:

Blocked by #
None

Documentation:

Documentation update required.
- Relevant updates are included with this PR.
- A WM issue has been opened to track the need for a documentation update; a person responsible for submitting the update has been assigned to the issue (link issue).
Documentation update NOT required.
- Explanation:

Changes

Regression Test Changes (Please commit test_changes.list):

PR Adds New Tests/Baselines.
PR Updates/Changes Baselines.
No Baseline Changes.

Input data Changes:

None.
PR adds input data.
PR changes existing input data.

Library Changes/Upgrades:

Required
- Library names w/versions:
- Git Stack Issue (JCSDA/spack-stack#)
No Updates

Testing Log:

This build creates a warning (error) on GaeaC6

gspetro-NOAA · 2026-03-11T14:03:44Z

@DusanJovic-NOAA Could you sync your branch? I think this PR will be next.

gspetro-NOAA · 2026-03-11T14:42:52Z

We'd like to bring in changes from #3138 . Would you like me to do that?

DeniseWorthen · 2026-03-11T14:43:31Z

@DusanJovic-NOAA Do you recall what the warning was on C6 that made you have to turn off the warn-as-error in aac684d?

DusanJovic-NOAA · 2026-03-11T15:35:23Z

We'd like to bring in changes from #3138 . Would you like me to do that?

Sure.

DusanJovic-NOAA · 2026-03-11T15:38:54Z

@DusanJovic-NOAA Do you recall what the warning was on C6 that made you have to turn off the warn-as-error in aac684d?

Something about long symbol names ( module variables or subroutines). We have compile flag to suppress that specific warning, but for some reason on Gaea it does not work. It's probably specific to the compiler version used currently on Gaea.

…OAA/ufs-weather-model into log_gnu_w

on-behalf-of @ufs-community <ecc.platform@noaa.gov>

gspetro-NOAA · 2026-03-12T12:08:15Z

@DusanJovic-NOAA datm_cdeps_gnu has failed to compile twice on Derecho. The err file (/glade/derecho/scratch/gpetro/FV3_RT/rt_37587/compile_datm_cdeps_gnu/err) has a CICE-related error message:

CMake Warning:
  Manually-specified variables were not used by the project:

    MPI

+ OMP_NUM_THREADS=1
+ make -j 3 VERBOSE=1
/glade/derecho/scratch/gpetro/RTs/ufs-wm/3129a/CICE-interface/CICE/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90:1043:26:

 1043 |       call mpi_allreduce (i8_arr_lsum_level, i8_arr_gsum_level, &
      |                          1
......
 1381 |       call mpi_allreduce (arr_lsum_dd, arr_gsum_dd, nflds, &
      |                          2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/COMPLEX(8)). [-Werror]
/glade/derecho/scratch/gpetro/RTs/ufs-wm/3129a/CICE-interface/CICE/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90:1043:45:

 1043 |       call mpi_allreduce (i8_arr_lsum_level, i8_arr_gsum_level, &
      |                                             1
......
 1381 |       call mpi_allreduce (arr_lsum_dd, arr_gsum_dd, nflds, &
      |                                       2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/COMPLEX(8)). [-Werror]
f951: all warnings being treated as errors
make[2]: *** [CICE-interface/CMakeFiles/cice.dir/build.make:1169: CICE-interface/CMakeFiles/cice.dir/CICE/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:369: CICE-interface/CMakeFiles/cice.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

My latest test directory is at /glade/derecho/scratch/gpetro/RTs/ufs-wm/3129a/tests.

FYI @NickSzapiro-NOAA

DusanJovic-NOAA · 2026-03-12T12:52:21Z

@DusanJovic-NOAA datm_cdeps_gnu has failed to compile twice on Derecho. The err file (/glade/derecho/scratch/gpetro/FV3_RT/rt_37587/compile_datm_cdeps_gnu/err) has a CICE-related error message:

CMake Warning:
  Manually-specified variables were not used by the project:

    MPI

+ OMP_NUM_THREADS=1
+ make -j 3 VERBOSE=1
/glade/derecho/scratch/gpetro/RTs/ufs-wm/3129a/CICE-interface/CICE/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90:1043:26:

 1043 |       call mpi_allreduce (i8_arr_lsum_level, i8_arr_gsum_level, &
      |                          1
......
 1381 |       call mpi_allreduce (arr_lsum_dd, arr_gsum_dd, nflds, &
      |                          2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/COMPLEX(8)). [-Werror]
/glade/derecho/scratch/gpetro/RTs/ufs-wm/3129a/CICE-interface/CICE/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90:1043:45:

 1043 |       call mpi_allreduce (i8_arr_lsum_level, i8_arr_gsum_level, &
      |                                             1
......
 1381 |       call mpi_allreduce (arr_lsum_dd, arr_gsum_dd, nflds, &
      |                                       2
Error: Type mismatch between actual argument at (1) and actual argument at (2) (INTEGER(8)/COMPLEX(8)). [-Werror]
f951: all warnings being treated as errors
make[2]: *** [CICE-interface/CMakeFiles/cice.dir/build.make:1169: CICE-interface/CMakeFiles/cice.dir/CICE/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:369: CICE-interface/CMakeFiles/cice.dir/all] Error 2
make: *** [Makefile:136: all] Error 2

My latest test directory is at /glade/derecho/scratch/gpetro/RTs/ufs-wm/3129a/tests.

FYI @NickSzapiro-NOAA

I do not have access to Derecho. For now, turn WARN_AS_ERROR to OFF for that build, until that warning if fixed.

BrianCurtis-NOAA · 2026-03-12T13:23:34Z

https://github.com/NOAA-EMC/CICE/blob/def9c53b17d6722cc9991ba72ddae8ad2d5d25d9/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90#L1043-L1044

      call mpi_allreduce (i8_arr_lsum_level, i8_arr_gsum_level, &
                          veclth, MPI_INTEGER8, MPI_SUM, mpi_comm, ierr)

vs
https://github.com/NOAA-EMC/CICE/blob/def9c53b17d6722cc9991ba72ddae8ad2d5d25d9/cicecore/cicedyn/infrastructure/comm/mpi/ice_reprosum.F90#L1381-L1382

      call mpi_allreduce (arr_lsum_dd, arr_gsum_dd, nflds, &
                          MPI_COMPLEX16, mpi_sumdd, mpi_comm, ierr)

gspetro-NOAA

Testing completed successfully on all systems. Noting that runtimes on Ursa were highly variable, likely due to fairshare/queue issues.

NickSzapiro-NOAA · 2026-03-12T22:50:16Z

It's curious that the CICE warning shows up in datm but not cpld (?). As for fixes...

It looks like the CICE code may already have a specific workaround for this:

#if defined (NO_I8)
     ! Workaround for when i8 is not supported.
!      if (detailed_timing) call xicex_timer_start("repro_sum_allr_i4")
      call mpi_allreduce (i8_arr_lsum_level, i8_arr_gsum_level, &
                          veclth, MPI_INTEGER, MPI_SUM, mpi_comm, ierr)
!      if (detailed_timing) call xicex_timer_stop("repro_sum_allr_i4")
#else
!      if (detailed_timing) call xicex_timer_start("repro_sum_allr_i8")
      call mpi_allreduce (i8_arr_lsum_level, i8_arr_gsum_level, &
                          veclth, MPI_INTEGER8, MPI_SUM, mpi_comm, ierr)
!      if (detailed_timing) call xicex_timer_stop("repro_sum_allr_i8")
#endif

These MPI type mismatch interface warnings can also come from use mpi instead of better use mpi_f08

DusanJovic-NOAA added 2 commits March 2, 2026 18:14

Add 'gnu' compile warning logs, add WARN_AS_ERROR build option

ea0fcd6

Merge remote-tracking branch 'origin/develop' into log_gnu_w

e19de0b

github-project-automation bot added this to PRs to Process Mar 6, 2026

github-project-automation bot moved this to Evaluating in PRs to Process Mar 6, 2026

gspetro-NOAA moved this from Evaluating to Draft in PRs to Process Mar 6, 2026

Full regression test passed on Ursa

b38b5d5

DusanJovic-NOAA marked this pull request as ready for review March 6, 2026 22:28

gspetro-NOAA moved this from Draft to Evaluating in PRs to Process Mar 8, 2026

gspetro-NOAA added the No Baseline Change No Baseline Change label Mar 8, 2026

gspetro-NOAA moved this from Evaluating to Review in PRs to Process Mar 8, 2026

FernandoAndrade-NOAA and others added 5 commits March 9, 2026 08:52

Update opnReqTest

0881125

Update opnReqTest

c382186

Turn OFF compile option WARN_AS_ERROR for datm_cdeps_intel

aac684d

This build creates a warning (error) on GaeaC6

add ORT logs passed and fix typo

8401000

Update opnReqTest

ef3de06

Merge remote-tracking branch 'origin/develop' into log_gnu_w

970c0eb

gspetro-NOAA moved this from Review to Schedule in PRs to Process Mar 11, 2026

Merge branch 'develop' into fix/ort-resync

bac0dd0

Merge branch 'fix/ort-resync' of https://github.com/FernandoAndrade-N…

f783511

…OAA/ufs-weather-model into log_gnu_w

gspetro-NOAA changed the title ~~Save gnu warnings in rt logfile. Add WARN_AS_ERROR build option~~ Save gnu warnings in rt logfile. Add WARN_AS_ERROR build option // Update ORT input data directories (#3138) Mar 11, 2026

gspetro-NOAA added Ready for Commit Queue The PR is ready for the Commit Queue. All checkboxes in PR template have been checked. In Testing The PR that is currently in testing stages labels Mar 11, 2026

gspetro-NOAA added 2 commits March 11, 2026 13:55

hercules rt log - passed

827f3f0

orion rt log - passed

a9dc842

FernandoAndrade-NOAA added the ursa-ORT label Mar 11, 2026

BrianCurtis-NOAA and others added 3 commits March 11, 2026 20:23

Acorn RT: Passed

4391f7e

WCOSS2 RT Log: Passed

b00d84f

ORT Jobs Completed.

b6a34e4

on-behalf-of @ufs-community <ecc.platform@noaa.gov>

epic-cicd-jenkins removed the ursa-ORT label Mar 12, 2026

FernandoAndrade-NOAA and others added 2 commits March 12, 2026 02:31

add gaeaC6 RT log passed

432a79f

add ursa RT log passed

742392b

gspetro-NOAA and others added 3 commits March 12, 2026 07:42

turn off WARN_AS_ERROR for datm_cdeps_gnu

4330fa0

add ursa ORTs passed

4c0eec8

derecho rt log - passed

b42c446

gspetro-NOAA removed the In Testing The PR that is currently in testing stages label Mar 12, 2026

gspetro-NOAA requested review from BrianCurtis-NOAA, FernandoAndrade-NOAA, NickSzapiro-NOAA, dpsarmie and jkbk2004 March 12, 2026 20:03

gspetro-NOAA approved these changes Mar 12, 2026

View reviewed changes

jkbk2004 approved these changes Mar 12, 2026

View reviewed changes

FernandoAndrade-NOAA approved these changes Mar 12, 2026

View reviewed changes

dpsarmie approved these changes Mar 12, 2026

View reviewed changes

BrianCurtis-NOAA approved these changes Mar 12, 2026

View reviewed changes

FernandoAndrade-NOAA merged commit de0da94 into ufs-community:develop Mar 12, 2026
13 of 14 checks passed

github-project-automation bot moved this from Schedule to Done in PRs to Process Mar 12, 2026

FernandoAndrade-NOAA mentioned this pull request Mar 12, 2026

Update ORT input data directories #3138

Closed

21 tasks

DusanJovic-NOAA deleted the log_gnu_w branch March 13, 2026 14:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Save gnu warnings in rt logfile. Add WARN_AS_ERROR build option // Update ORT input data directories (#3138)#3129

Save gnu warnings in rt logfile. Add WARN_AS_ERROR build option // Update ORT input data directories (#3138)#3129
FernandoAndrade-NOAA merged 21 commits intoufs-community:developfrom
DusanJovic-NOAA:log_gnu_w

DusanJovic-NOAA commented Mar 6, 2026 •

edited by gspetro-NOAA

Loading

Uh oh!

gspetro-NOAA commented Mar 11, 2026

Uh oh!

gspetro-NOAA commented Mar 11, 2026

Uh oh!

DeniseWorthen commented Mar 11, 2026

Uh oh!

DusanJovic-NOAA commented Mar 11, 2026

Uh oh!

DusanJovic-NOAA commented Mar 11, 2026

Uh oh!

gspetro-NOAA commented Mar 12, 2026 •

edited

Loading

Uh oh!

DusanJovic-NOAA commented Mar 12, 2026

Uh oh!

BrianCurtis-NOAA commented Mar 12, 2026

Uh oh!

gspetro-NOAA left a comment

Uh oh!

Uh oh!

NickSzapiro-NOAA commented Mar 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Conversation

DusanJovic-NOAA commented Mar 6, 2026 • edited by gspetro-NOAA Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Commit Queue Requirements:

Description:

#3129

#3138

Commit Message:

Priority:

Git Tracking

UFSWM:

Sub component Pull Requests:

UFSWM Blocking Dependencies:

Documentation:

Changes

Regression Test Changes (Please commit test_changes.list):

Input data Changes:

Library Changes/Upgrades:

Testing Log:

Uh oh!

gspetro-NOAA commented Mar 11, 2026

Uh oh!

gspetro-NOAA commented Mar 11, 2026

Uh oh!

DeniseWorthen commented Mar 11, 2026

Uh oh!

DusanJovic-NOAA commented Mar 11, 2026

Uh oh!

DusanJovic-NOAA commented Mar 11, 2026

Uh oh!

gspetro-NOAA commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DusanJovic-NOAA commented Mar 12, 2026

Uh oh!

BrianCurtis-NOAA commented Mar 12, 2026

Uh oh!

gspetro-NOAA left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NickSzapiro-NOAA commented Mar 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

DusanJovic-NOAA commented Mar 6, 2026 •

edited by gspetro-NOAA

Loading

gspetro-NOAA commented Mar 12, 2026 •

edited

Loading