Run a SURFEX-Crocus experiment with vortex¶
On Météo-France HPC supercomputers (belenos and taranis), s2m will automatically run experiments through the vortex toolbox.
Do never try to change this behaviour.
It means that you first need either to configure your vortex environment either to install vortex: https://redmine.umr-cnrm.fr/projects/snowtools_git/wiki/Install_VORTEX
This page gives you also all the informations for file transfert with Hendrix archive through the command ftmotpasse
The available options differ from the standard use of s2m without vortex.
Actions of the s2m command :
- Prepare vortex configuration file and running environment.
- Run the mkjob tool to create the transfer and computing jobs and submit the jobs.
- In the first transfer job, extract the necessary resources from hendrix or from the vortex cache in a working directory
- In the computing job, pre-process these resources to build the namelist
- In the computing job, run SURFEX
- In the last transfer job, save outputs on hendrix (with an asynchronous method) and on the vortex cache
Usage
USAGE module load python/3.7.6nomkl s2m -b begin_date -e end_date -m model -f forcing -r region [-o path_output] [-w workdir] [-n namelist] [-s surfex_exe_directory] [--nmembers=XX] [--ntasks=XX] [-x date_end_spinup] [-a threshold_1aout] [--escroc=XX] [--nmembers=XX] [--startmember=XX] [--dailyprep] [--walltime=h:mm:ss] [--writesx]
-m model -f forcing -r region
The forcing files you are going to use are defined by options -m model -f forcing -r region
If you want to use our reference SAFRAN reanalysis files you must use the following options:
-m s2m -f reanalysis2020.2@lafaysse -r region with region choosen in [alp_allslopes, alp_flat, pyr_allslopes, pyr_flat, cor_allslopes, cor_flat, postes] in the general case
-m safran -f reanalysis2020.2@lafaysse -r region with region choosen in [alp_allslopes, alp_flat, pyr_allslopes, pyr_flat, cor_allslopes, cor_flat, postes] only if you want to define your own slope angles
If you want to use your own forcing files, these forcing files must be put on hendrix in the directory /home/yourlogin/vortex/model/region/forcing/meteo (model, region, forcing correspond to the arguments of options -m, -r, -f)
If you want to use the forcing files from another hendrix account, use the following syntax: -f forcing@userlogin, these forcing files must be put on hendrix in the directory /home/userlogin/vortex/model/region/forcing/meteo (model, region, forcing correspond to the arguments of options -m, -r, -f)
Only yearly files are supported at the moment. The format of the file name is FORCING_YYYYMMDDHH_YYYYMMDDHH.nc and this can not be changed.
The 'meteo' block name can be modified in the task if necessary (not recommended for a standard application)
If you don't work on a standard geometry (region), you must always define a new geometry in a file $HOME/.vortexrc/geometries.ini which contains the following lines:
[name_of_new_geometry] info = Describe here your new geometry kind = unstructured area = name_of_new_geometry
Three cases can be distinguished:
- The forcing files correspond to the geometry you want to apply in your simulations
- The forcing files must be reduced to specific massifs, elevations, aspects or slopes, or extended on more slope angles
- The forcing files must be interpolated towards your geometry simulation:
For case 1, -r region corresponds to the new geometry you have defined in $HOME/.vortexrc/geometries.ini
case 2, is temporarily not implemented. You must preprocess the files on your side without vortex and apply case 1
not implemented If one of this option is activated, you must use the -r option with the following syntax: -r geometryin:geometryout
geometryin is the geometry of SAFRAN input (alp_flat, pyr_flat, alp_allslopes, etc.)
geometryout is the name of your target geometry (which must have been defined in $HOME/.vortexrc/geometries.ini and must be have a specific name.)
[-l list_slopes] not implemented Extract a reduced number of slope angles in FORCING files, or generate a larger number of slope angles for a flat FORCING file. Do not use this option if you are using a standard geometry (flat, allslopes).
Example: -l 0,20,40
Memory issues are expected for very large requests (for instance with more than 4 slope values on the whole Alps). In that case, please contact us.[-c nb_expo] not implemented Specify the number of aspect classes to extract or generate. Default:8. Do not use this option if you are using a standard geometry (flat, allslopes).
-c 8 : N, NE, E, SE, S, SW, W, NW
-c 4 : N, E, S, W
-c 2 : N, S
-c 135 : specify only 1 aspect in ° if multiple of 45°[-L Lower-altitude] not implemented Eliminate elevations lower than this level in FORCING files (in meters). Do not use this option if you are using a standard geometry (flat, allslopes).
[-U Upper-altitude] not implemented Eliminate elevations upper than this level in FORCING files (in meters). Do not use this option if you are using a standard geometry (flat, allslopes).
Ex : -L 1200 –U 2100 (select elevations from 1200 to 2100 m)
Default: keep all levels.For case 3:
The -r option must be used with the following syntax -r geometryin:geometryout:targetgeometry.nc
geometryin is the geometry of SAFRAN input (alp_flat, pyr_flat, etc.) and must not include several slope angles. Note that if your SAFRAN files have been preprocessed with s2m to obtain geometry geometryin, the preprocessing must have used -l 0 option.
geometryout is the name of your target geometry (as defined in $HOME/.vortexrc/geometries.ini)
targetgeometry.nc is a netcdf describing your new geometry. It must contain the massif_number and ZS variables. It can be 1d (Number_of_points) or 2d (y, x) in Lambert93 or 2d (latitude, longitude). 1d files can optionnally provide aspect and slope. These variables will be computed by the software for 2d files.
-b begin_date (format: YYYYMMDD or YYYYMMDDHH). Start at 06 UTC if HH is not defined.
-e end_date (format: YYYYMMDD or YYYYMMDDHH). Finish at 06 UTC if HH is not defined.
-o output. Define the name of your experiment. Default: output
Your outputs are saved on hendrix in the directories: /home/yourlogin/vortex/s2m/region/output/prep and /home/yourlogin/vortex/s2m/region/output/pro
The directories are created if they do not exist.
The outputs are also saved in the vortex cache. Therefore, if you develop a post-processing tool on beaufix, it will be much more efficient to access the resources with vortex toolbox.
Take care : Dash ( - ) is not allowed in directories names by the current version of vortex (change planned in v1.6)
Note that a default PGD file is taken in the CEN parameters directory if available for the region. Otherwise, the PGD file is taken in the hendrix output directory if it already exists. Otherwise, it will be generated using the informations provided in the namelist.
Note that if a PREP file of initial conditions is provided in the hendrix output directory (/home/yourlogin/vortex/s2m/region/prep/PREP_YYYYMMDD.nc), it will be used to restart the simulation. Alternatively, we look for an initial condition file in an experiment with the spinup xpid (/home/yourlogin/vortex/s2m/region/spinup/prep). Otherwise, they will be generated using the informations provided in the namelist and in the init_TG.nc file. A default init_TG.nc file is taken in the CEN parameters directory if available for the region. Otherwise, the init_TG.nc file is taken in the hendrix output directory (/home/yourlogin/vortex/s2m/region/prep/init_TG.nc) if it already exists. Otherwise, the code will crash, unless -g option is prescribed.
[-g] Generate a ground initialization file by computing a climatological average of air temperature on the provided period.
[-x date_end_spinup]
If a spinup simulation has been run, it is possible to force the model to start using the model state conditions at the end of this spinup even if it differs from the real starting date. If you use this options, it means that a PREP_YYYYMMDD.nc file is expected in the hendrix output directory or in the vortex cache at the date provided in this option.
[-a threshold_1aout]
Limit the snow water equivalent each 1st August to the provided value in kg/m2
[-w workdir]. Working directory (default $WORKDIR)
[-n namelist] SURFEX namelist path. Default: $SNOWTOOLS_CEN/DATA/OPTIONS_V8.1_NEW_OUTPUTS_NC.nam ( https://redmine.umr-cnrm.fr/projects/snowtools_git/repository/revisions/master/entry/snowtools/DATA/OPTIONS_V8.1_NEW_OUTPUTS_NC.nam )
You can define many SURFEX settings using a different namelist (output variables and timestep, physical options, soil and vegetation parameters, etc.)
If you need to define your own SURFEX namelist, we recommand to create a namelist from this default namelist and to modify it step by step.
You can define many SURFEX settings using a different namelist (output variables and timestep, physical options, soil and vegetation parameters, etc.)
More information at Define a namelist for gridded configurations
[-s surfex_exe_directory] Specify the directory where to find SURFEX binaries (PGD, PREP, OFFLINE)
For deterministic simulations, PGD, PREP and OFFLINE must have been compiled with MPI
For ensemble simulations, OFFLINE must have been compiled without MPI
If undefined, look for the content of the environment variable $EXESURFEX.
If -s and $EXESURFEX undefined, take default executables in the last available genv environment. (recommended if SURFEX does not need to be modified)
The default behaviour is to run OFFLINE using 1 nodes and 80 cores. This behaviour can be changed in the code with --ntasks and --nnodes options.
[--nnodes=X] Number of computing nodes (default 1).
[--ntasks=X] Number of tasks per node (default 80). For gridded MPI simulations, always respect ntasks * nnodes < number of lines of the grid
[--escroc=XX] Perform ensemble multiphysics simulations with the ESCROC system. XX must be one of the known subensemble :
E1 (7776-member full ensemble as defined in Lafaysse et al, TC, 2017)
E2 (35-member subensemble as defined in Lafaysse et al, TC, 2017)
E2MIP (35-member subensemble, fitted on MIP sites, unpublished)
E2tartes (35-member subensemble, same as E2 but with T17 radiative scheme, as used in Tuzet et al, TC, 2020)
E2MIPtartes (35-member subensemble, same as E2MIP but with T17 radiative scheme, as used in Dumont et al, JGR, 2020)
[--nmembers=XX] In the case of ESCROC, specify the number of members
[--startmember=XX] In the case of ESCROC, specify the number of the initial member.
[--dailyprep] Split the simulations daily in order to save model state each 24h. This is required to prepare initial conditions for a reforecast. Take care: this option is very time consuming, you must prescribe the walltime and should think to reduce outputs in PRO files.
[--walltime='h:mm:ss'] Change the default walltime of the computing job only. You cannot exceed 24 hours.
NB : if you exceed time limits in the transfer jobs : change the walltime of transfer jobs
[--lptr] Active the profiling of the code. Files are written localy and not in Hendrix. Link for folder of profiling files is inside step.03 text file.
[--writesx] Transfer PRO output files on sxcen.cnrm.meteo.fr : be careful with volumes and transfer times. Before using this option you must have created a vortex link in your home directory to not saturate the /home space :
ln -s /cnrm/cen/users/NO_SAVE/yourlogin/vortex $HOME/vortex
[--refill] Refill task to use in the case when a simulation went well and the output have been stored in the local cache but not on Hendrix.
Adding the --refill option to the initial s2m command allows to retrieves the ressources produced by the previous simulation on the local cache and put them on Hendrix as intended initially.