CNRM Open Source

Dear Crocus users,

The snowtools code have been migrated a few months ago to github. The code is now publicly available at the following address : https://github.com/UMR-CNRM/snowtools.

Today, we also migrate the ticket and assitance to our users. The assistance is now provided at https://github.com/UMR-CNRM/snowtools-tickets. This is a private repository, we grant access on demand. If you think you will need assistance or need to keep access to existing tickets, please ask to leo.viallon at meteo.fr and provide your github username associated with your professional email address (active developers at CEN already have been granted an access).

All tickets opened on https://redmine.umr-cnrm.fr/projects/snowtools_git/issues have been migrated on github and will be answered on this new interface.

The snowtools repository on https://redmine.umr-cnrm.fr/projects/snowtools_git will be soon closed and any new issue, code, wiki or issue update on this interface will not be taken into account.

Dear Crocus users,

The snowtools code have been migrated today to github. The code is now publicly available at the following address : https://github.com/UMR-CNRM/snowtools. No more contributions will be accepted through redmine.umr-cnrm.fr website. Support will only be provided if you use the version available on github.

As some major merges and modification of the git history append, it is recommended to start from a new fresh install of snowtools, by cloning the new repository from https://github.com/UMR-CNRM/snowtools.

Temporarily, the bug reports and technical assistance (issues/demandes) remain on redmine.umr-cnrm.fr web site : https://redmine.umr-cnrm.fr/projects/snowtools_git/issues/new. Do not hesitate to submit an issue in case you met technical issues with this migration.

Additional changes impacting users¶

This migration was an opportunity to provide some significant changes to the snowtools code base that will impact users. The major changes are reported below.

Bronx (https://github.com/UMR-CNRM/bronx) python library have been moved outside of the snowtools repository. However, it remains a strong requirement for snowtools. It is therefore necessary to install bronx separately (cloning the github repository and placing the src directory in your PYTHONPATH or use a python virtual environment and just run "pip install bronx").

A new visualisation tool have been provided for SURFEX-Crocus snow profiles, called proplotter. A tool for comparing profiles from two different files (in the same geometry) is also available under the name of procompare. The old version is no longer available or supported.

s2m command have been widely rewritten to simplify available options. Please run "s2m research --help" for more details.

Dear Crocus users,

Note that some technical updates have been applied to our branches of SURFEX code. They allow the compatibility of our code with Ubuntu 20.04 and replace some obsolete libraries by maintained equivalents.
Due to these modifications of libraries, after your next code update, running again the configure script will be mandatory before being able to compile the code.

Additionnaly, only for CNRM users still working on Mageia distributions, the following line has to be added in .bash_profile files:

unset GRIB_DEFINITION_PATH

This is not necessary in other cases.

Let us know in case you met any trouble due to this technical update: https://redmine.umr-cnrm.fr/projects/snowtools_git/issues/new

Thank you,
Matthieu

Dear snowtools users,

Please note a major change in the code architecture on master branch of snowtools repository these last days:

The main snowtools code is now in a snowtools folder (in which you will find all modules you are used to import). To import a snowtools module, from snowtools itself, from an other package or from your personal code, it is now compulsory to use absolute imports. e.g. :

from snowtools.utils.prosimu import prosimu

This have been done for three main reasons:

• It is the recommended import form in accordance with PEP8
• The code organization is now coherent with the way it is expected by vortex (python library that use snowtools to run experiment on Meteo-France supercomputers)
• The git repository will now contain at its root some useful files to describe the package (requirements, install instructions, README, etc.) and be in a near future directly installable with `pip` for users that are simple users (not for developers).

Impacts for you, after upgrading to the latest version:

• You have to adapt imports in your personal codes relying on snowtools imports
• Alias to s2m and proreader that may be defined in your ~/.bashrc or .bash_profile-like file have to point to the new location of files (note that SNOWTOOLS_CEN remains unchanged):
  

  alias s2m="python $SNOWTOOLS_CEN/snowtools/tasks/s2m_command.py" 
  alias proreader="python3 $SNOWTOOLS_CEN/snowtools/plots/GUI_Proreader.py"

Note that the python path does not have to be updated! If you think you need to modify your `PYTHONPATH`, please check you really have absolute imports everywhere (`from snowtools.[...] import`).

To update the code:

• Go into the folder containing the git repository snowtools_git
• Please check you do not have any uncommited file (with `git status` for instance).
• Pull the latest version of the code with

  git pull

• As git does not remove by itself folders that are no longer use, you have to delete them by hand or by doing

  git clean --force -x -d

  (check before with `git clean -n -x -d` that you do not have any personal uncommited or untracked file that would be deleted)

Despite the attention paid to required changes in code and several tests, it may remains some inconsistencies and bug linked to this change. Do not hesitate to open an issue on https://redmine.umr-cnrm.fr/projects/snowtools_git/issues.

Please also note the next major change in your timetables: python2 will not be supported in snowtools after the end of this year (2021). Please check in advance that your experiments and code run well on python3 and do not hesitate to open an issue if you encounter any problem, as snowtools is now considered to be fully working with python3.

Commit commit:1396635a

FA support requires cy46_t1_op1.05/cy46t1_bf.07 of libs4py.so

Contents since 1.4.9:¶

(to get the same contents without element [13] => get v1.4.12)
(to get the same contents without elements [12],[13] => get v1.4.11)

• Invoke python3 explicitly in shebang

• Add get_inputs (for gallery) target in doc Makefile

• domain_making: threshold Lambert/Mercator raised to 5deg from equator

• [12] Pruning of basemap-based plotting functionalities and .cmap user-defined colormaps (use .json instead, cf. examples in doc). Include pruning of epy_plot.py
• contourf_kw, colorbar_kw and pcolormesh_kw arguments added to cartoplot method

• [13] FA: Upgrade libs4py.so to cy46_t1_op1.05/cy46t1_bf.07, in order to be able to read GRIB2-encoded FA/LAM fields, and fields with missing values in FA
• LFA/DDHLFA better distinction
• netCDFMNH: support for MNH 5.5.0
• FA: upgrade gribapi.def files to v32

• epyweb: web.py removed, to be installed with pip: pip3 install web.py

• revision of tolerant_equality of RecursiveObject

And a whole lot of various fixes...

Dear Crocus users,

The cen branch (stable branch for Crocus users) has just been updated with numerous modifications including:

- The CrocO framework for data assimilation in Crocus with a Particle Filter: https://redmine.umr-cnrm.fr/projects/surfex_git2/wiki/CrocO_user_doc from Bertrand Cluzet

- Numerous numerical optimizations (order of loops, position of if statements, netcdf instructions, etc.) significantly improving the running times from Rafife Nheili

- Standalone version of Crocus for coupling with other numerical surface schemes: https://redmine.umr-cnrm.fr/projects/surfex_git2/wiki/Install_standalone_version_of_Crocus from Rafife Nheili

- Rewritting of microstructure prognostic variables with explicit names and removing of B92 metamorphism parameterization from Mathieu Fructus

- New functionalities of Crocus-RESORT module: https://redmine.umr-cnrm.fr/projects/surfex_git2/wiki/Crocus-RESORT from Carlo Carmagnola

- Other minor changes from various contributors

Many thanks to all contributors and to Mathieu Fructus and Rafife Nheili for the merging work and associated tests.

For standard users, only minor numerical differences are expected compared to previous version.

All branches from our group (cen cen_hotfix cen_release cen_dev) have converged today at a unique and common state before the beginning of a new cycle of developments.

There is no urge to update your code but note that we will ask all users to update their code in case of bugs in older versions.
(git pull and new compilation)

If despite all our tests, you meet troubles with this new version, please report them through the support interface: https://redmine.umr-cnrm.fr/projects/snowtools_git/issues/new

Thank you
Matthieu

Commit commit:478bf0ba

Contents since 1.4.8:¶

• Complete rewriting of Gallery of use-cases
• Doc index update, along with other minor changes

• More portable installation script: setup_epygram.py
• Deployment script update

• domain_making : allow for different X & Y resolutions in Regular LonLat geometries
• Rect/RegLL geometry plotting adapted to cartopy

• attach uvRelativeToGrid to field when relevant, and rely on this attribute as a default for wind plotting: may change default behaviour of wind plotting.
• field.scatter_with() improvement
• colormapping.get_ColormapHelper facility

• DiagnosticsResource: GRIB2 codes for graupel and pressure departure was not up to date

• new HDF5SAF format, used for some satellite products (e.g. SAF LAND)
• NetCDF: smarter guess of regular lon/lat
• FA: Update sfxflddesc_mod wrt SFXv8.1+

• make vgrid: new target in main Makefile for building vgrid binary

And a whole lot of various fixes...

Contents: only fixes (see commits !)

The first version of DPOLSIMUL is now available.
More information about this code and the way to download it can be found in the wiki.

Commit commit:b4880121

Contents since 1.4.3:¶

Formats:

• fix GRIB writing of constant fields
• netCDF CF-1.4 stereopolar projection decoding
• efficient iterator in GRIB (for f in g)

Installation:

• porting to belenos
• porting to MacOSX
• Makefile, deployment and installation updated (new directories/paths)

Miscellaneous:

• EPyGrAM no longer is in charge of setting matplotlib backend: now to be set in $HOME/.config/matplotlib/matplotlibrc
  e.g.
  backend: Agg is the recommended non-interactive backend
  backend: Qt5Agg is the recommended interactive backend for Mageia7
  cf. https://matplotlib.org/faq/usage_faq.html#what-is-a-backend for more information
• operations on Angles

And a whole lot of various fixes...