Bring rubem calibrate to parity with the previous calibration workflow

doc/source/changelog.rst

@@ -13,6 +13,16 @@ The format follows `Keep a Changelog <https://keepachangelog.com/en/1.1.0/>`__.
 Added
 `````
 
+- ``rubem calibrate`` gains per-run bounds (``--bound NAME=MIN:MAX``, narrowing the
+  settings ranges), fixed parameters that leave the search (``--fix NAME=VALUE``),
+  the stations that enter the objective (``--stations``), the initialization,
+  strategy and polishing options, per-station goodness-of-fit (pairs, means,
+  standard deviations, Pearson correlation, RMSE, NSE), ``observed.csv``,
+  ``stations.csv`` and ``best_<variable>.csv`` in the run directory, a
+  ``started_at`` column in the evaluations, progress lines on the terminal and
+  the consolidation of the evaluations of an interrupted search. Negative
+  observed values are gaps, and time series files without their header are
+  refused (`#348 <https://github.com/LabSid-USP/RUBEM/issues/348>`__).
 - Added the calibration page of the documentation (the objective, the free
   parameters and their bounds, the derivation of ``w3``, the differential
   evolution settings and the evaluation budget, the observed series formats,

doc/source/userguide.rst

@@ -1182,38 +1182,66 @@ Use ``-h`` or ``--help`` to get a brief description of each command and its argu
      Calibrate the model parameters against an observed series.
 
    Options:
-     -c, --configfile <path>     Path to the configuration file (JSON).
-                                 [required]
-     --observed <file>           Observed series at the sample stations (CSV or
-                                 PCRaster TSS).  [required]
-     -o, --run-dir <path>        Directory for the calibration artifacts.
-                                 [required]
-     --variable <str>            Output variable compared with the observed
-                                 series.  [default: arn]
-     --spinup-steps <int range>  Leading time steps excluded from the NSE.
-                                 [default: 0; x>=0]
-     --maxiter <int range>       Maximum number of generations.  [default: 100;
-                                 x>=1]
-     --popsize <int range>       Population multiplier of the search.  [default:
-                                 15; x>=1]
-     --seed <int>                Seed of the differential evolution.
-     --workers <int range>       Parallel evaluations (default: all cores but
-                                 one).  [x>=1]
-     --temp-dir <path>           Parent of the per-evaluation output directories.
-     -h, --help                  Show this message and exit.
+     -c, --configfile <path>         Path to the configuration file (JSON).
+                                     [required]
+     --observed <file>               Observed series at the sample stations (CSV
+                                     or PCRaster TSS).  [required]
+     -o, --run-dir <path>            Directory for the calibration artifacts.
+                                     [required]
+     --variable <str>                Output variable compared with the observed
+                                     series.  [default: arn]
+     --spinup-steps <int range>      Leading time steps excluded from the NSE.
+                                     [default: 0; x>=0]
+     --maxiter <int range>           Maximum number of generations.  [default:
+                                     100; x>=1]
+     --popsize <int range>           Population multiplier of the search.
+                                     [default: 15; x>=1]
+     --seed <int>                    Seed of the differential evolution.
+     --workers <int range>           Parallel evaluations (default: all cores but
+                                     one).  [x>=1]
+     --temp-dir <path>               Parent of the per-evaluation output
+                                     directories.
+     --bound <str>                   Narrow the range of one parameter:
+                                     NAME=MIN:MAX.
+     --fix <str>                     Keep one parameter out of the search:
+                                     NAME=VALUE.
+     --stations <str>                Station ids of the objective, comma-
+                                     separated (default: every shared station).
+     --init <sobol|latinhypercube|halton|random>
+                                     Initial population of the search.  [default:
+                                     sobol]
+     --strategy <str>                Differential evolution strategy.  [default:
+                                     best1exp]
+     --polish / --no-polish          Refine the best candidate with a local
+                                     search.  [default: no-polish]
+     -h, --help                      Show this message and exit.
 
 ``calibrate`` needs SciPy, which comes with ``pip install
-"rubem[calibration]"``. It writes ``evaluations.csv``, ``result.json``
-and ``<config>-calibrated.json`` into the ``--run-dir``. The method itself (the
-objective, the parameter bounds, the derivation of ``w3``, how to budget the
-number of evaluations and what a run needs from the configuration) is
+"rubem[calibration]"``. It writes ``observed.csv`` (what the observations offer
+at each station, before the search), ``evaluations.csv`` (one row per
+evaluation), ``stations.csv`` (the goodness of fit of the best candidate at
+each station), ``best_<variable>.csv`` (its series beside the observed one),
+``result.json``, ``<config>-calibrated.json`` and the ``evaluations/``
+directory of per-evaluation records into the ``--run-dir``, which must be empty
+of earlier records. ``--bound`` and ``--fix`` are repeatable, one parameter
+each; ``--stations`` names the gauges the objective averages, and the others
+are still measured and reported. The method itself (the objective, the
+parameter bounds, the derivation of ``w3``, how to budget the number of
+evaluations, what the run prints and what it needs from the configuration) is
 documented in :doc:`Calibration </calibration>`.
 
 .. code-block:: console
 
-   $ rubem calibrate -c project-config.json --observed observed.csv -o calibration
+   $ rubem calibrate -c project-config.json --observed observed.csv -o calibration \
+       --spinup-steps 12
    Loading configuration and validating inputs...
    Calibration started...
+   The calibration compares 216 time step(s); the observed series covers 216 of them, the step(s) 13 to 228.
+   Station 1: 198 observed value(s) on the compared steps, 18 dropped as gaps.
+   Calibrating 8 free parameter(s) with 128 population member(s) per generation and at most 100 generation(s): up to 12928 model run(s), on 15 worker process(es).
+   Generation 1: best objective 1.60757e+06, best NSE 0.601240, 256 evaluation(s) recorded.
+   Generation 16: best objective 160757, best NSE 0.873210, 2048 evaluation(s) recorded.
+   Calibration finished after 2048 evaluation(s) in 16 generation(s): best objective 160757, best NSE 0.873210.
    Calibration finished successfully!
    Best NSE: 0.873210
    Best objective: 160757
@@ -1222,6 +1250,9 @@ documented in :doc:`Calibration </calibration>`.
    Evaluations table: /basins/ipojuca/calibration/evaluations.csv
    Result: /basins/ipojuca/calibration/result.json
    Calibrated configuration: /basins/ipojuca/calibration/project-config-calibrated.json
+   Observed summary: /basins/ipojuca/calibration/observed.csv
+   Stations table: /basins/ipojuca/calibration/stations.csv
+   Best candidate series: /basins/ipojuca/calibration/best_arn.csv
 
 Use ``-V`` or ``--version`` to get the version of the RUBEM.