CrocO user doc¶

Software description¶

CrocO (Crocus-Observations) is a software enabling to assimilate snow observations into ensemble simulations of the snowpack model Crocus, using Particle Filters (PFs). This application is computationally intensive, and requires the parallelization of the ensemble members and an optimization of resource fluxes. For this reason, CrocO is tailored to Météo-France’s research HPC system (beaufix-hendrix).

Main Features¶

• several PF algorithms implemented or under development in SURFEX to assimilate reflectance, reflectance ratios and snow depth)
• VORTEX is used to fetch files from hendrix or beaufix cache, parallelize the computations and archive the outputs on hendrix
• snowtools_git is used as an interface
• More details in the technical doc : CrocO technical doc

Dependencies¶

CrocO depends on several open-source codes distributed by the CNRM (Centre National de Recherches Metéorologiques) using git :

• Crocus and the PFs are implemented within SURFEX model (fortran)
  [[https://redmine.umr-cnrm.fr/projects/surfex_git2]]
• the user interface, and pre-post processing are handled by snowtools_git code (python):
  [[https://redmine.umr-cnrm.fr/projects/snowtools_git]]
• the HPC environment and data fluxes rely on VORTEX (python):
  [[https://redmine.umr-cnrm.fr/projects/vortex]]

Prerequisites (not exclusive):¶

• access to beaufix and hendrix
• access to SURFEX code
• read snowtools_git wiki page
  [[https://redmine.umr-cnrm.fr/projects/snowtools_git/wiki/]]
• be familiar with Crocus snowpack model and its ensemble version ESCROC (Lafaysse et al. 2017):
  Multiphysics
• if you're going to assimilate reflectances (Tuzet et al., 2017):
  Explicit_representation_of_impurities
• CrocO doesn't handle the generation of initial conditions and forcings. You need to generate (or use preexisting) an ensemble of forcings and a spinup.

Installation¶

Standard users only need to install SURFEX on beaufix. snowtools_git and VORTEX are already installed for them on beaufix.

Install SURFEX¶

• If you're not developping in SURFEX, download the cen_dev branch directly on beaufix.
  Otherwise, you'd better install it locally and synchronize your local modifications to beaufix with rsync command :

  [[https://redmine.umr-cnrm.fr/projects/snowtools_git/wiki/Install_SURFEX]]

• Compile it in NOMPI-O2 configuration (sequential ensemble application case in the following link) :

  [[https://redmine.umr-cnrm.fr/projects/snowtools_git/wiki/Specific_compilations_%28beaufix_or_prolix_SOPRANO_old_compilers%29]]

Don't forget to frequently update your code version doing git pull in your code repository.

Set CrocO environment (standard users)¶

Developers in snowtools_git and VORTEX (for CrocO) must properly install it, (eventually with rsync) following :
Install and Install_VORTEX

Standard users don't need to install VORTEX nor snowtools_git. They will use versions of snowtools_git and VORTEX maintained by the main developer.

• To use snowtools and VORTEX, they just need to modify their .bash_profile on beaufix :

# vortex
export MTOOLDIR=$WORKDIR
export VORTEX=/home/cluzetb/common/vortex/vortex-cen_assim
export PYTHONPATH=$VORTEX
export PYTHONPATH=$PYTHONPATH:$VORTEX/bin
export PYTHONPATH=$PYTHONPATH:$VORTEX/site
export PYTHONPATH=$PYTHONPATH:$VORTEX/src
export PYTHONPATH=$PYTHONPATH:$VORTEX/project

# snowtools_git
export SNOWTOOLS_CEN=/home/cluzetb/common/snowtools_git
export PYTHONPATH=$PYTHONPATH:$SNOWTOOLS_CEN
alias s2m="python $SNOWTOOLS_CEN/tasks/s2m_command.py" 

• Setting new geometries. Any CrocO experiment should be associated with a given VORTEX geometry (location). In addition, they are used in the archive path and filenames. Some of them are defined in $VORTEX/conf/geometries.ini. If you need to define a new geometry (region), you must define it in a new file $HOME/.vortexrc/geometries.ini which contains the following lines. For example, Grandes-Rousses massif has a region_id of 12 and its region_name is grandes_rousses:
  

  [region_id]
  info       = Describe here your new geometry
  kind       = unstructured
  area       = region_name
  

• Some constant files necessary for SURFEX (covers, F06 parameters...) are stored on beaufix in /home/lafaysse/CONST_CEN/.
  To fetch it, you need to set some uget stuff :

mkdir $HOME/.vortexrc/hack/uget
ln -s /home/cnrm_other/cen/mrns/cluzetb/CONST_CEN $HOME/.vortexrc/hack/uget/CONST_CEN

How to use CrocO¶

Once the installation is done, you're almost ready to perform a CrocO experiment.

Definition of a CrocO experiment¶

CrocO allows you to perform 3 different types of experiments :

• openloop : no assimilation. Useful as a reference or to generate synthetic observations
• synthetic : assimilation of synthetic observations (e.g. from a previous openloop run)
  In that case, if the ensemble setup is the same as the openloop, you MUST remove the synthetic truth member from your ensemble (see below).
• real : assimilation of real observations

Assimilation scheme setup¶

The ensemble assimilation system is set up as follows (Cluzet et al. 2019, in review)
- a run is a unique combination of a forcing fxxxx and an ESCROC configuration mbyyyy
The configuration fxxxx->mbyyyy, is fixed during the experiment
- all the runs are initialized by the same spinup
- the total number of runs is defined by the parameter nmembers
- nmembers also defines the number of different ESCROC configurations mbyyyy
- nforcing (<=nmembers) defines the number of different forcings fxxxx to use
If nforcing < nmembers, the forcings are repeated until all runs have a forcing.

Note also that analyzed prep files and ESCROC members configurations are shuffled between assimilation dates, except for openloop experiments.

Prepare an experiment¶

• Remind that CrocO doesn't handle the generation of initial conditions (PGD, and PREP), forcings ensemble and observations files : you need to generate it and archive it on hendrix beforehand.
  Detailed instructions can be found in CrocO technical doc.

• Prepare a basic namelist. This namelist will be used to configure the PF algorithm, and SURFEX I/O behavior), and will be used as a base for ESCROC's multiphysics scheme (see Multiphysics).
  Bear in mind that the number of output variables considerably increases the time spent in writing (during computation) and transfer as well as storage needs. So carefully check the list of output variables in PRO files (see ex. below):
  

  &NAM_WRITE_DIAG_SURFn
  CSELECT = 'time','ASNOW_VEG','TALB_ISBA','TS_ISBA','WSN_T_ISBA','DSN_T_ISBA','SNOWDZ','WSN_VEG','SPECMOD','SNOWSSA','SNOWIMP1','SNOWIMP2'
  

• following changes_in_namelist in CrocO technical doc, set the assimilation parameters (choice of the PF algorithm, and SURFEX I/O behavior. You might also need to activate reflectance outputs (see. Explicit_representation_of_impurities).

• following CrocO technical doc, prepare an assimilation configuration file, used to set the assimilation dates and eventually the ids of the membres to run.
  If the ESCROC membersId are not specified in the configuration file (default cases for ESCROC subensemble E1*), they will be randomly drawn at the beginning of the simulation and written in a copy of the configuration file (which will be archived on hendrix to ensure traceability). You must specify it if you need to ensure reproducibility (for twin experiments for example).

Launch an experiment¶

CrocO assimilation sequence is ran through a single s2m command (snowtools_git command) on beaufix.

First of all, if you're not familiar with s2m command and vortex read the following page :
Run_a_SURFEX-Crocus_experiment_without_vortex

Among them, you will need only the following :

-m safran                  : set -m to safran.
-r <region_id> : your geometry, consistent with your vortex path and PGD filenames.
-b
-e
-f <xpid_forcing>          : name of the repertory where the forcings are stored on hendrix
-o <xpid>                  : name of the repertory where the outputs will be stored
-n <path_to_your_namelist> : default namelist is not provided

A few supplementary arguments are necessary to run CrocO. Optional are ().

--croco <your_path_to_assimilation_configuration_file> : give the path to the assimilation configuration file.
--escroc <escrocsubensemble>                             : specify the ESCROC subensemble to use 
                                                           ("E1tartes", "E1notartes", "E2")
--nmembers <N>                                           : Number of members to run/draw among the subensemble
--nforcings <Nf>                                         : Number of different forcings to use
--nnodes                                                 : number of nodes on which to parallelize
--walltime                                               : estimate of the time duration of the parallelized experiment (minutes). 
                                                           Your simulation will be terminated past that duration
--sensor                                                 : Name of the observations sensor/synthetical xp. (free, default is MODIS)

--openloop                                               : activate openloop mode
  OR
--synth <mbid>                                           : assimilation of synthetic data. Remove and replace the <mbid> member (synthetic truth)
  OR
--real                                                   : assimilation of real data

(--writesx <path>                                        : activate output to sxcen.cnrm, put it into <path>.)
(--grid                                                  : specify if your're performing gridded simulations)

Example :

s2m -b 20160801 -e 20161120 -f forcing_20162017B_31D_11_150 -m safran -r 12 --escroc E1tartes --croco /home/cluzetb/s2m_12.ini
 --nmembers 80 -n /home/cluzetb/OPTIONSV9_sd_beaufix.nam -o test_croco --nforcing 10 --nnodes 2 --walltime 120 --sensor MODIS --openloop

Simulation outputs¶

Once the simulation has finished, simulation outputs are stored in the vortex path on hendrix.
(see Simulation outputs storing in CrocO technical doc)
Now it's up to you to post-process it !