Merge pull request #376 from mvertens/feature/refactor_stream_usage_escomp

major refactor stream of stream usage

### Description of changes
Major refactor of stream usage

### Specific notes
All data mode files across the data components(except for `datm_datamode_gefs_mod.F90 `and `dice_datamode_cplhist_mod.F90`) now have explicit setting of streams rather than using the implicit copy used by `dfields`. Copies that used to be done implicitly using the naming convention assumed by dfields are
now done explictly. This makes it clear what export fields are direct copies and what export fields are
derived fields that have no corresponding stream field.

In each datamode module - there are now two sections in the module variable list:  `export (and sometimes import) state pointers` and  `stream pointer`.

The following changes have been made throughout the code: `character(*) => character(len=*)`,` trim(subname) => subname` (the latter is done since parameters, e.g. `subname`,  don't need trim functions)

The following new routines have been introduced or deleted:
 - **dwav**
   - dwav_datamode_copyall.F90: new
 - **drof**
   - drof_datamode_copyall.F90: new
   - introduction of new datatype (stream_pointer_type) for multilevel stream pointers
   - expanded list of stream fields needed for drof.cplhist and added new functionality (e.g. `drof_datamode_cplhist_mod.F90`) to support this addition.
 - **docn**
   - combined `docn_datamode_copyall_mod.F90` and `docn_datamode_iaf_mod.F90` into a new file     `docn_datamode_sstdata_mod.F90`. The two files `docn_datamode_copyall_mod.F90` and `docn_datamode_iaf_mod.F90` were effectively the same and two different datamodes are not needed since in both cases prescribed SST data was read in. (In addition, docn_datamode_iaf_mod.F90` set pointers to importState data which was never used and not needed).
   - ocn_comp_nuopc.F90: the datamode copyall is now sstdata and the datamode iaf is removed
   - changed cplhist and multilev_cplhist taxmode from extend to cycle
 - **dlnd**
   - introduction of new datatype (stream_pointer_type) for multilevel stream pointers
 - **dice**
   - dice_cplhist_data_mod.F90: still uses dfields since it is used by UFS
 - **dglc**
   - introduction of new datatype (stream_pointer_type) for multilevel stream pointers

**NOTE**: This PR also provides a new mapfile algorithm as a new option for mapping the streams input to the component model resolution. The new scheme does not have a default but is enabled via adding the following in the appropriate user_nl_XXX_streams where the user needs to fill in the entries in <> below:
<stream_name>:mapalgo = "mapfile:<path to mapping file>
As an example:
`rof.ryf8485_jra:mapalgo="mapfile:/cluster/shared/noresm/inputdata/cpl/cpl6/map_JRA025_to_tnx1v4_e1000r300_170928.nc"`. This new capability enables a significant speedup in stand-alone ocean simulations. See NorESMhub/BLOM#686 (comment).

**NOTE***: 

Contributors other than yourself, if any: None

CDEPS Issues Fixed: #377

Are there dependencies on other component PRs (if so list):

Are changes expected to change answers (bfb, different to roundoff, more substantial):
- docn cplhist mode can change answers due to changing taxmode from extend to cycle

Any User Interface Changes (namelist or namelist defaults changes):

Testing performed (will describe the noresm testing here)
- aux_cdeps_noresm:
  - compared to baseline cdeps1.0.87_noresm_v1 - all passed (#377) and generated new baseline cdeps1.0.87_noresm_v2)
- prealpha_noresm
  - compared to noresm3_0_beta09 - the only difference is SMS_Lm13.f19_f19_mtn14.I1850Clm50SpG.betzy_intel where atmImp_Faxa_ndep1 and atmImp_Faxa_ndep2 are different due to new new tag cdeps1.0.87_noresm_v2.

Hashes used for testing: noresm3_0_beta09 plus this CDEPS branch


Also tested with CESM prealpha tests and a few additional tests (see #376 for details.)

Author: billsacks

cime_config/stream_cdeps.py

@@ -344,7 +344,8 @@ def create_stream_xml(
 
                         # Check that key is valid
                         expect(
-                            mod_dict[var_key] in valid_values[var_key],
+                            (mod_dict[var_key] in valid_values[var_key]) or
+                            (var_key == 'mapalgo' and 'mapfile:' in mod_dict[var_key]),
                             "{} can only have values of {} for stream {} in file {}".format(
                                 var_key,
                                 valid_values[var_key],

datm/atm_comp_nuopc.F90

@@ -102,11 +102,7 @@
 
   <entry id="DATM_PRESNDEP">
     <type>char</type>
-    <valid_values>none,
-      clim_1850_cmip7,clim_2000_cmip7,clim_2010_cmip7,hist_cmip7,
-      clim_1850_cmip6,clim_2000_cmip6,clim_2010_cmip6,hist_cmip6,
-      SSP1-2.6,SSP2-4.5,SSP3-7.0,SSP5-3.4,SSP5-8.5,cplhist
-    </valid_values>
+    <valid_values>none,clim_1850_cmip7,clim_2000_cmip7,clim_2010_cmip7,hist_cmip7,clim_1850_cmip6,clim_2000_cmip6,clim_2010_cmip6,hist_cmip6,SSP1-2.6,SSP2-4.5,SSP3-7.0,SSP5-3.4,SSP5-8.5,cplhist</valid_values>
     <default_value>clim_2000</default_value>
     <values match="last">
       <value compset="^1850[CE]?_"  >clim_1850_cmip6</value>

datm/cime_config/config_component.xml

@@ -14,9 +14,6 @@
       <value datm_mode="CLM_QIAN$">
         CLM_QIAN.Solar,CLM_QIAN.Precip,CLM_QIAN.TPQW
       </value>
-      <value datm_mode="CLM_QIAN_WISO">
-        CLM_QIAN_WISO.Solar,CLM_QIAN_WISO.Precip,CLM_QIAN_WISO.TPQW
-      </value>
       <value datm_mode="CLMCRUJRA2024">
         CLMCRUJRA2024.Solar,CLMCRUJRA2024.Precip,CLMCRUJRA2024.TPQW
       </value>
@@ -329,19 +326,6 @@
     </values>
   </entry>
 
-  <entry id="flds_wiso">
-    <type>logical</type>
-    <category>datm</category>
-    <group>datm_nml</group>
-    <desc>
-      If true, prescribed water isotopes are sent from datm (must be true for running with CLM).
-    </desc>
-    <values>
-      <value>.false.</value>
-      <!-- <value datm_mode="CLM_QIAN_WISO">.true.</value> -->
-    </values>
-  </entry>
-
   <entry id="iradsw">
     <type>integer</type>
     <category>datm</category>

datm/cime_config/namelist_definition_datm.xml

@@ -45,7 +45,7 @@ module datm_datamode_core2_mod
   real(r8), pointer :: Sa_shum(:)    => null()
   real(r8), pointer :: Sa_pbot(:)    => null()
   real(r8), pointer :: Sa_pslv(:)    => null()
-  real(r8), pointer :: Faxa_lwdn(:)  => null()
+  real(r8), pointer :: Sa_dens(:)    => null()
   real(r8), pointer :: Faxa_rainc(:) => null()
   real(r8), pointer :: Faxa_rainl(:) => null()
   real(r8), pointer :: Faxa_snowc(:) => null()
@@ -55,13 +55,22 @@ module datm_datamode_core2_mod
   real(r8), pointer :: Faxa_swvdr(:) => null()
   real(r8), pointer :: Faxa_swvdf(:) => null()
   real(r8), pointer :: Faxa_swnet(:) => null()
+  real(r8), pointer :: Faxa_swdn(:)  => null()
+  real(r8), pointer :: Faxa_lwdn(:)  => null()
 
-  ! stream data
-  real(r8), pointer :: strm_prec(:)      => null()
-  real(r8), pointer :: strm_swdn(:)      => null()
-  real(r8), pointer :: strm_tarcf(:)     => null()
-
-  ! othe module arrays
+  ! required stream data points
+  real(r8), pointer :: strm_Faxa_prec(:)  => null()
+  real(r8), pointer :: strm_Faxa_swdn(:)  => null()
+  real(r8), pointer :: strm_Faxa_lwdn(:)  => null()
+  real(r8), pointer :: strm_Sa_pslv(:)    => null()
+  real(r8), pointer :: strm_Sa_tbot(:)    => null()
+  real(r8), pointer :: strm_Sa_shum(:)    => null()
+  real(r8), pointer :: strm_Sa_dens(:)    => null()
+  real(r8), pointer :: strm_Sa_u(:)       => null()
+  real(r8), pointer :: strm_Sa_v(:)       => null()
+  real(r8), pointer :: strm_tarcf(:)      => null()
+
+  ! other module arrays
   real(R8), pointer :: windFactor(:)
   real(R8), pointer :: winddFactor(:)
   real(R8), pointer :: qsatFactor(:)
@@ -159,31 +168,6 @@ subroutine datm_datamode_core2_init_pointers(exportState, sdat, datamode, factor
 
     rc = ESMF_SUCCESS
 
-    lsize = sdat%model_lsize
-
-    ! allocate module arrays
-    allocate(windFactor(lsize))
-    allocate(winddFactor(lsize))
-    allocate(qsatFactor(lsize))
-
-    call ESMF_MeshGet(sdat%model_mesh, spatialDim=spatialDim, numOwnedElements=numOwnedElements, rc=rc)
-    if (ChkErr(rc,__LINE__,u_FILE_u)) return
-    allocate(ownedElemCoords(spatialDim*numOwnedElements))
-    allocate(yc(numOwnedElements))
-    call ESMF_MeshGet(sdat%model_mesh, ownedElemCoords=ownedElemCoords)
-    if (ChkErr(rc,__LINE__,u_FILE_u)) return
-    do n = 1,numOwnedElements
-       yc(n) = ownedElemCoords(2*n)
-    end do
-
-    ! get stream pointers
-    call shr_strdata_get_stream_pointer( sdat, 'Faxa_prec'  , strm_prec  , rc)
-    if (ChkErr(rc,__LINE__,u_FILE_u)) return
-    call shr_strdata_get_stream_pointer( sdat, 'Faxa_swdn'  , strm_swdn  , rc)
-    if (ChkErr(rc,__LINE__,u_FILE_u)) return
-    call shr_strdata_get_stream_pointer( sdat, 'tarcf'      , strm_tarcf , rc)
-    if (ChkErr(rc,__LINE__,u_FILE_u)) return
-
     ! get export state pointers
     call dshr_state_getfldptr(exportState, 'Sa_z'       , fldptr1=Sa_z       , rc=rc)
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
@@ -205,6 +189,8 @@ subroutine datm_datamode_core2_init_pointers(exportState, sdat, datamode, factor
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
     call dshr_state_getfldptr(exportState, 'Sa_shum'    , fldptr1=Sa_shum    , rc=rc)
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call dshr_state_getfldptr(exportState, 'Sa_dens'    , fldptr1=Sa_dens    , rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
     call dshr_state_getfldptr(exportState, 'Faxa_rainc' , fldptr1=Faxa_rainc , rc=rc)
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
     call dshr_state_getfldptr(exportState, 'Faxa_rainl' , fldptr1=Faxa_rainl , rc=rc)
@@ -223,22 +209,63 @@ subroutine datm_datamode_core2_init_pointers(exportState, sdat, datamode, factor
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
     call dshr_state_getfldptr(exportState, 'Faxa_swnet' , fldptr1=Faxa_swnet , rc=rc)
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call dshr_state_getfldptr(exportState, 'Faxa_swdn'  , fldptr1=Faxa_swdn  , rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
     call dshr_state_getfldptr(exportState, 'Faxa_lwdn'  , fldptr1=Faxa_lwdn  , rc=rc)
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
 
-    if (.not. associated(strm_prec) .or. .not. associated(strm_swdn)) then
-       call shr_log_error(subname//'ERROR: prec and swdn must be in streams for CORE2', rc=rc)
+    ! get required stream pointers
+    call shr_strdata_get_stream_pointer( sdat, 'Faxa_prec'  , strm_Faxa_prec  , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Faxa_prec must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Faxa_swdn'  , strm_Faxa_swdn  , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Faxa_swdn must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Faxa_lwdn'  , strm_Faxa_lwdn  , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Faxa_lwdn must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Sa_pslv'    , strm_Sa_pslv    , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Sa_pslv must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Sa_tbot'    , strm_Sa_tbot    , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Sa_tbot must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Sa_u'       , strm_Sa_u       , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Sa_u must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Sa_v'       , strm_Sa_v       , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Sa_v must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Sa_shum'    , strm_Sa_shum    , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Sa_shum must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'Sa_dens'    , strm_Sa_dens    , requirePointer=.true., &
+         errmsg=subname//'ERROR: strm_Sa_dens must be associated for core2 datamode', rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    call shr_strdata_get_stream_pointer( sdat, 'tarcf', strm_tarcf, rc) ! required for CORE2_IAF
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    if (trim(datamode) == 'CORE2_IAF' .and. .not. associated(strm_tarcf)) then
+       call shr_log_error(subname//'tarcf must be associated for CORE2_IAF', rc=rc)
        return
     endif
 
-    if (trim(datamode) == 'CORE2_IAF' ) then
-       if (.not. associated(strm_tarcf)) then
-          call shr_log_error(subname//'tarcf must be in an input stream for CORE2_IAF', rc=rc)
-          return
-       endif
-    endif
+    ! create yc
+    call ESMF_MeshGet(sdat%model_mesh, spatialDim=spatialDim, numOwnedElements=numOwnedElements, rc=rc)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    allocate(ownedElemCoords(spatialDim*numOwnedElements))
+    allocate(yc(numOwnedElements))
+    call ESMF_MeshGet(sdat%model_mesh, ownedElemCoords=ownedElemCoords)
+    if (ChkErr(rc,__LINE__,u_FILE_u)) return
+    do n = 1,numOwnedElements
+       yc(n) = ownedElemCoords(2*n)
+    end do
+    deallocate(ownedElemCoords)
 
     ! create adjustment factor arrays
+    lsize = sdat%model_lsize
+    allocate(windFactor(lsize))
+    allocate(winddFactor(lsize))
+    allocate(qsatFactor(lsize))
     call datm_get_adjustment_factors(sdat, factorFn_mesh, factorFn_data, windFactor, winddFactor, qsatFactor, rc)
     if (ChkErr(rc,__LINE__,u_FILE_u)) return
 
@@ -278,11 +305,13 @@ subroutine datm_datamode_core2_advance(datamode, target_ymd, target_tod, target_
     cosfactor = cos((2.0_R8*SHR_CONST_PI*rday)/365 - phs_c0)
 
     do n = 1,lsize
+
+       !--- set Sa_z to a constant ---
        Sa_z(n) = 10.0_R8
 
        !--- correction to NCEP winds based on QSCAT ---
-       uprime = Sa_u(n)*windFactor(n)
-       vprime = Sa_v(n)*windFactor(n)
+       uprime = strm_Sa_u(n)*windFactor(n)
+       vprime = strm_Sa_v(n)*windFactor(n)
        Sa_u(n) = uprime*cos(winddFactor(n)*degtorad) - vprime*sin(winddFactor(n)*degtorad)
        Sa_v(n) = uprime*sin(winddFactor(n)*degtorad) + vprime*cos(winddFactor(n)*degtorad)
 
@@ -291,9 +320,12 @@ subroutine datm_datamode_core2_advance(datamode, target_ymd, target_tod, target_
        Sa_v10m(n) = Sa_v(n)
 
        !--- density and pslv taken directly from input stream, set pbot ---
+       Sa_pslv(n) = strm_Sa_pslv(n)
+       Sa_dens(n) = strm_Sa_dens(n)
        Sa_pbot(n) = Sa_pslv(n)
 
        !--- correction to NCEP Arctic & Antarctic air T & potential T ---
+       Sa_tbot(n) = strm_Sa_tbot(n)
        if      ( yc(n) < -60.0_R8 ) then
           tMin = (avg_c0 + avg_c1*yc(n)) + (amp_c0 + amp_c1*yc(n))*cosFactor + tKFrz
           Sa_tbot(n) = max(Sa_tbot(n), tMin)
@@ -304,7 +336,7 @@ subroutine datm_datamode_core2_advance(datamode, target_ymd, target_tod, target_
        Sa_ptem(n) = Sa_tbot(n)
 
        !---  correction to NCEP relative humidity for heat budget balance ---
-       Sa_shum(n) = Sa_shum(n) + qsatFactor(n)
+       Sa_shum(n) = strm_Sa_shum(n) + qsatFactor(n)
 
        !--- Dupont correction to NCEP Arctic air T  ---
        !--- don't correct during summer months (July-September)
@@ -315,34 +347,37 @@ subroutine datm_datamode_core2_advance(datamode, target_ymd, target_tod, target_
        end if
 
        ! PRECIPITATION DATA
-       strm_prec(n) = strm_prec(n)/86400.0_R8        ! convert mm/day to kg/m^2/s
+       strm_Faxa_prec(n) = strm_Faxa_prec(n)/86400.0_R8        ! convert mm/day to kg/m^2/s
        !  only correct satellite products, do not correct Serreze Arctic data
        if ( yc(n) < 58. ) then
-          strm_prec(n) = strm_prec(n)*1.14168_R8
+          strm_Faxa_prec(n) = strm_Faxa_prec(n)*1.14168_R8
        endif
        if ( yc(n) >= 58. .and. yc(n) < 68. ) then
           factor = MAX(0.0_R8, 1.0_R8 - 0.1_R8*(yc(n)-58.0_R8) )
-          strm_prec(n) = strm_prec(n)*(factor*(1.14168_R8 - 1.0_R8) + 1.0_R8)
+          strm_Faxa_prec(n) = strm_Faxa_prec(n)*(factor*(1.14168_R8 - 1.0_R8) + 1.0_R8)
        endif
        Faxa_rainc(n) = 0.0_R8               ! default zero
        Faxa_snowc(n) = 0.0_R8
        if (Sa_tbot(n) < tKFrz ) then        ! assign precip to rain/snow components
           Faxa_rainl(n) = 0.0_R8
-          Faxa_snowl(n) = strm_prec(n)
+          Faxa_snowl(n) = strm_Faxa_prec(n)
        else
-          Faxa_rainl(n) = strm_prec(n)
+          Faxa_rainl(n) = strm_Faxa_prec(n)
           Faxa_snowl(n) = 0.0_R8
        endif
 
        ! RADIATION DATA
        !--- fabricate required swdn components from net swdn ---
-       Faxa_swvdr(n) = strm_swdn(n)*(0.28_R8)
-       Faxa_swndr(n) = strm_swdn(n)*(0.31_R8)
-       Faxa_swvdf(n) = strm_swdn(n)*(0.24_R8)
-       Faxa_swndf(n) = strm_swdn(n)*(0.17_R8)
+       Faxa_swdn(n)  = strm_Faxa_swdn(n)
+       Faxa_swvdr(n) = strm_Faxa_swdn(n)*(0.28_R8)
+       Faxa_swndr(n) = strm_Faxa_swdn(n)*(0.31_R8)
+       Faxa_swvdf(n) = strm_Faxa_swdn(n)*(0.24_R8)
+       Faxa_swndf(n) = strm_Faxa_swdn(n)*(0.17_R8)
+
        !--- compute net short-wave based on LY08 latitudinally-varying albedo ---
        avg_alb = ( 0.069 - 0.011*cos(2.0_R8*yc(n)*degtorad ) )
-       Faxa_swnet(n) = strm_swdn(n)*(1.0_R8 - avg_alb)
+       Faxa_swnet(n) = strm_Faxa_swdn(n)*(1.0_R8 - avg_alb)
+
        !--- corrections to GISS sswdn for heat budget balancing ---
        factor = 1.0_R8
        if      ( -60.0_R8 < yc(n) .and. yc(n) < -50.0_R8 ) then
@@ -357,10 +392,12 @@ subroutine datm_datamode_core2_advance(datamode, target_ymd, target_tod, target_
        Faxa_swndr(n) = Faxa_swndr(n)*factor
        Faxa_swvdf(n) = Faxa_swvdf(n)*factor
        Faxa_swndf(n) = Faxa_swndf(n)*factor
+
        !--- correction to GISS lwdn in Arctic ---
+       Faxa_lwdn(n) = strm_Faxa_lwdn(n)
        if ( yc(n) > 60._R8 ) then
           factor = MIN(1.0_R8, 0.1_R8*(yc(n)-60.0_R8) )
-          Faxa_lwdn(n) = Faxa_lwdn(n) + factor * dLWarc
+          Faxa_lwdn(n) = strm_Faxa_lwdn(n) + factor * dLWarc
        endif
 
     enddo   ! lsize