MTool : A Brief Tour¶

Introduction¶

MTOOL is mostly a non-intrusive way to build multi-steps and modular jobs. It could be restrict to a very few set of directives which are comments for the job language (either sh or python). The main features are therefore the step definition and the include mechanism.

The logical view of a job should not suffer from its actual execution sequence. One could imagine the design of a MTOOL job in such a shape that the job remains coherent without MTOOL processing. This processing is performed through a Perl filter provided in the distribution pack, altogether with some other utilities.

Crude Example¶

Here is a pseudo bash job with two parts and some common features:

  #MTOOL set jobname=foo
  echo "Part [this:id] started" 
  pwd

  #MTOOL step id=part1
  echo "This should be the pre-processing part" 

  #MTOOL step id=part2
  #MTOOL include files=arch.fortran.[this:target]
  echo "This should be the computing part" 

  #MTOOL common
  echo "This is the end" 

And some Python equivalent:

  #MTOOL set jobname=foo
  import os
  print "Part [this:id] started" 
  print os.getcwd()

  #MTOOL step id=part1
  print "This should be the pre-processing part" 

  #MTOOL step id=part2
  #MTOOL include files=arch.fortran.[this:target]
  print "This should be the computing part" 

  #MTOOL common
  print "This is the end" 

Execution Targets¶

MTool has been intensively used on large High Performance Computing systems such as NEC SX, IBM P6 and P7 clusters, BULL or CRAY supercomputers. It's highly recommended to process job scripts with MTool on these systems in order to defined sharp computing steps, moving any extra non-challenging computing operations to peripheral steps.

MTool on these targets has been interfaced with various batch and queuing systems : NQS, LoadLeveler and PBS Pro.

So far, more than 10 millions of jobs have gracefully been submitted through the MTool filter at Meteo-France for the last decade or so.

Distribution Package¶

MTool comes as a simple set of directories and files, including Perl modules and utilities to perform the full range of features, without any compilation or installation process. It could be configured either through environment variables or standard editing of default included files, as explained in the documentation.

The first version (2.2.0) comes with a targets subdirectory including default profiles and include files for BULL clusters at Meteo-France. Forthcoming versions will contain files for CRAY systems.

Changes¶

The first version published on this site is mtool-2.2.0

Changes will be tracked in this section.

Licence¶

MTool is distributed under the CECIL-C license (a French equivalent to the L-GPL license).

LICENSE-V1-fr.txt
LICENSE-V1-en.txt

Acknowledgements¶

Many improvements, fixes and testing have been carried out by the large community of users, but I am particularly grateful among other to : Antoinette Alias, Guillaume Beffrey, Michel Déqué, Ryad El Khatib, Philippe Marguinaud, Stéphane Martinez, Véronique Mathiot, Louis-François Meunier, Stéphane Sénési (to be completed)