Merge pull request #246 from TemoaProject/feat/stochastics

Temoa v4 stochastics implementation

Author: jdecarolis

docs/source/computational_implementation.rst

@@ -389,7 +389,7 @@ The Temoa model code is organized into clear, purpose-driven packages:
   * ``monte_carlo`` - :doc:`monte_carlo` (Uncertainty quantification)
   * ``myopic`` - Sequential decision making with limited foresight
   * ``single_vector_mga`` - Focused MGA on specific variables ([!] untested in v4.0)
-  * ``stochastics`` - Stochastic programming capabilities ([!] untested in v4.0)
+  * ``stochastics`` - :doc:`stochastics` (Stochastic programming capabilities)
 
 * ``temoa._internal`` - Internal utilities (not part of public API)
 

docs/source/index.rst

@@ -6,3 +6,4 @@ Temoa Project Documentation
    Documentation
    monte_carlo
    unit_checking
+   stochastics

docs/source/stochastics.rst

@@ -0,0 +1,107 @@
+
+.. _stochastics:
+
+Stochastic Programming
+======================
+
+The Stochastics extension in Temoa v4 provides support for stochastic programming using the `mpi-sppy <https://github.com/Pyomo/mpi-sppy>`_ library. This allows for decision-making under uncertainty by considering multiple scenarios simultaneously and finding an optimal "first-stage" decision that minimizes the expected cost over all scenarios.
+
+Stochastic programming is particularly useful for modeling uncertainties in future costs, demands, or resource availability.
+
+Dependencies
+------------
+
+The stochastics extension requires the ``mpi-sppy`` package. You can install it using ``uv``:
+
+.. code-block:: bash
+
+   uv add mpi-sppy
+
+Or using ``pip``:
+
+.. code-block:: bash
+
+   pip install mpi-sppy
+
+Configuration
+-------------
+
+To run Temoa in stochastic mode, you need to modify your main configuration TOML file and provide an additional stochastic configuration file.
+
+Main Configuration TOML
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Set the ``scenario_mode`` to ``"stochastic"`` and add a ``[stochastic]`` section:
+
+.. code-block:: toml
+
+   scenario_mode = "stochastic"
+
+   # ... other standard options ...
+
+   [stochastic]
+   # Path to the stochastic configuration file, relative to this file
+   stochastic_config = "stochastic_config.toml"
+
+Stochastic Configuration TOML
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The stochastic configuration file defines the scenarios, their probabilities, and the data perturbations associated with each scenario.
+
+.. code-block:: toml
+
+   # Define the scenarios
+   [scenarios]
+   # Each key is a scenario name, and the value is its probability
+   # Probabilities must sum to 1.0
+   low_cost = 0.5
+   high_cost = 0.5
+
+   # Define perturbations for a specific scenario
+   [[perturbations]]
+   scenario = "low_cost"
+   table = "cost_variable"
+   # Filter specifies which rows in the table to perturb
+   filter = { tech = "IMPHCO1" }
+   # Action can be "multiply", "add", or "set" (defaults to "set")
+   action = "multiply"
+   value = 0.5
+
+   [[perturbations]]
+   scenario = "high_cost"
+   table = "cost_variable"
+   filter = { tech = "IMPHCO1" }
+   action = "multiply"
+   value = 1.5
+
+Perturbation Options
+^^^^^^^^^^^^^^^^^^^^
+
+Currently, the following fields are required for each perturbation:
+
+* **scenario**: The name of the scenario to which this perturbation applies.
+* **table**: The Temoa parameter (database table) to perturb (e.g., ``cost_variable``, ``demand``, ``capacity_factor_process``).
+* **filter**: A dictionary of column-value pairs used to identify specific rows. Since the extension uses the dynamic manifest from ``HybridLoader``, any column belonging to the table's index can be used for filtering.
+* **action**: The operation to perform. Supported values:
+    * ``multiply``: Multiply the base value by ``value``.
+    * ``add``: Add ``value`` to the base value.
+    * ``set``: Replace the base value with ``value``.
+* **value**: The numeric value used in the perturbation action.
+
+How it Works
+------------
+
+When running in stochastic mode, Temoa:
+
+1. Loads the base data from the input database.
+2. Identifies the "first-stage" variables. In the current implementation, all decisions in the first time period are considered first-stage.
+3. Orchestrates multiple scenario runs using the ``mpi-sppy`` Extensive Form (EF) solver.
+4. For each scenario, the ``scenario_creator`` applies the specified perturbations to the base data and builds a Pyomo model instance.
+5. The EF solver binds the first-stage variables across all scenarios (non-anticipativity constraints) and optimizes the total expected cost.
+6. The terminal output reports the Stochastic Expected Value.
+
+Limitations
+-----------
+
+* **Two-Stage Only**: While ``mpi-sppy`` supports multi-stage stochastic programming, the current Temoa integration is tailored for two-stage problems where the first time period constitutes the first stage.
+* **Result Persistence**: Currently, only the expected objective value and summary logs are produced. Detailed per-scenario result persistence to the database is under development.

pyproject.toml

@@ -30,6 +30,7 @@ dependencies = [
     "rich>=14.2.0",
     "tomlkit>=0.12.0",
     "pint>=0.25.2",
+    "mpi-sppy>=0.12.1",
 ]
 
 

requirements-dev.txt

@@ -71,6 +71,8 @@ mdit-py-plugins==0.5.0
     # via myst-parser
 mdurl==0.1.2
     # via markdown-it-py
+mpi-sppy==0.12.1
+    # via temoa (pyproject.toml)
 multiprocess==0.70.18
     # via salib
 myst-parser==4.0.1
@@ -83,6 +85,7 @@ numpy==2.3.1
     #   contourpy
     #   highspy
     #   matplotlib
+    #   mpi-sppy
     #   pandas
     #   salib
     #   scipy
@@ -121,7 +124,9 @@ pygments==2.19.2
     #   rich
     #   sphinx
 pyomo==6.9.2
-    # via temoa (pyproject.toml)
+    # via
+    #   temoa (pyproject.toml)
+    #   mpi-sppy
 pyparsing==3.2.3
     # via matplotlib
 pytest==8.4.1
@@ -149,6 +154,7 @@ salib==1.5.1
 scipy==1.16.0
     # via
     #   temoa (pyproject.toml)
+    #   mpi-sppy
     #   salib
 seaborn==0.13.2
     # via temoa (pyproject.toml)

requirements.txt

@@ -37,6 +37,8 @@ matplotlib==3.9.2
     #   seaborn
 mdurl==0.1.2
     # via markdown-it-py
+mpi-sppy==0.12.1
+    # via temoa (pyproject.toml)
 multiprocess==0.70.18
     # via salib
 networkx==3.5
@@ -47,6 +49,7 @@ numpy==2.3.1
     #   contourpy
     #   highspy
     #   matplotlib
+    #   mpi-sppy
     #   pandas
     #   salib
     #   scipy
@@ -77,7 +80,9 @@ pygments==2.19.2
     #   pytest
     #   rich
 pyomo==6.9.2
-    # via temoa (pyproject.toml)
+    # via
+    #   temoa (pyproject.toml)
+    #   mpi-sppy
 pyparsing==3.2.3
     # via matplotlib
 pytest==8.4.1
@@ -97,6 +102,7 @@ salib==1.5.1
 scipy==1.16.0
     # via
     #   temoa (pyproject.toml)
+    #   mpi-sppy
     #   salib
 seaborn==0.13.2
     # via temoa (pyproject.toml)

temoa/_internal/temoa_sequencer.py

@@ -33,6 +33,7 @@
 from temoa.extensions.monte_carlo.mc_sequencer import MCSequencer
 from temoa.extensions.myopic.myopic_sequencer import MyopicSequencer
 from temoa.extensions.single_vector_mga.sv_mga_sequencer import SvMgaSequencer
+from temoa.extensions.stochastics.stochastic_sequencer import StochasticSequencer
 from temoa.model_checking.pricing_check import price_checker
 
 if TYPE_CHECKING:
@@ -67,6 +68,7 @@ def __init__(
         # for results catching for perfect_foresight or testing
         self.pf_results: pyomo.opt.SolverResults | None = None
         self.pf_solved_instance: TemoaModel | None = None
+        self.stochastic_sequencer: StochasticSequencer | None = None
 
     def _run_preliminary_checks(self) -> None:
         """Runs pre-flight system checks and (optionally) a non-fatal units check.
@@ -189,6 +191,10 @@ def start(self) -> None:
             case TemoaMode.MONTE_CARLO:
                 self._run_monte_carlo()
 
+            case TemoaMode.STOCHASTIC:
+                self.stochastic_sequencer = StochasticSequencer(config=self.config)
+                self.stochastic_sequencer.start()
+
             case _:
                 raise NotImplementedError(
                     f"The scenario mode '{self.temoa_mode}' is not yet implemented."

temoa/core/config.py

@@ -54,6 +54,7 @@ def __init__(
         myopic: dict[str, object] | None = None,
         morris: dict[str, object] | None = None,
         monte_carlo: dict[str, object] | None = None,
+        stochastic_config: Path | None = None,
         config_file: Path | None = None,
         silent: bool = False,
         stream_output: bool = False,
@@ -144,6 +145,7 @@ def __init__(
             )
         self.plot_commodity_network = plot_commodity_network and self.source_trace
         self.graphviz_output = graphviz_output
+        self.stochastic_config = stochastic_config
 
         # warn if output db != input db
         if self.input_database.suffix == self.output_database.suffix:  # they are both .db/.sqlite
@@ -228,6 +230,21 @@ def build_config(config_file: Path, output_path: Path, silent: bool = False) ->
         else:
             raise SolverNotAvailableError('No solver name specified in the configuration.')
 
+        if 'stochastic' in data:
+            stoch_data = data.pop('stochastic')
+            if 'stochastic_config' in stoch_data:
+                # Resolve relative to config file Path
+                sc = config_file.parent / stoch_data['stochastic_config']
+                data['stochastic_config'] = sc
+
+        # Pop other sections that are not top-level arguments in __init__
+        for section in ['execution', 'model', 'pricing', 'monte_carlo_tweaks']:
+            if section in data:
+                data.pop(section)
+
+        if 'output_path' in data:
+            data.pop('output_path')
+
         tc = TemoaConfig(output_path=output_path, config_file=config_file, silent=silent, **data)
         logger.info('Scenario Name:  %s', tc.scenario)
         logger.info('Data source:  %s', tc.input_database)

temoa/core/modes.py

@@ -17,3 +17,4 @@ class TemoaMode(Enum):
     CHECK = 6  # build and run price check, source trace it
     SVMGA = 7  # single-vector MGA
     MONTE_CARLO = 8  # MC optimization
+    STOCHASTIC = 9  # Stochastic optimization