Support for data indexed over PeriodPartitions

NEWS.md

@@ -2,9 +2,21 @@
 
 ## Unversioned
 
-### Bugfix
+### Bugfixes
 
 * Fixed a bug in the functionality `get_all_periods` when using `RepresentativePeriods` or `OperationalScenarios`.
+* Fix crash when reapplying the stored axis limits of plots of constant data (*e.g.*, a flat demand profile), as the stored limits can degenerate in Float32 precision.
+* Fix crash in `save_results` for models with variables indexed over `PeriodPartition`s.
+
+### Enhancements
+
+* Added support for visualizing data indexed over `PeriodPartition`s (introduced in *[`TimeStruct` v0.9.12](https://github.com/sintefore/TimeStruct.jl/releases/tag/v0.9.12)*), resolving [Issue #55](https://github.com/EnergyModelsX/EnergyModelsGUI.jl/issues/55):
+  * Both JuMP variables indexed over `PeriodPartition`s and `PartitionProfile` fields of elements (*e.g.*, `PeriodDemandSink` from `EnergyModelsFlex`) are available for plotting through the new *Partition* time axis.
+  * The *Partition* option in the time menu is only available for elements with data indexed over `PeriodPartition`s.
+  * The partitions of an element are by default deduced from its `period_duration` field; the function `period_partitions` can be specialized by packages using a different convention.
+    If the partitions of an element cannot be determined, a warning naming the element and the required method is issued and its data indexed over `PeriodPartition`s is not available for plotting.
+  * Variables indexed over `PeriodPartition`s are also supported when reading model results from CSV-files.
+    As the partition labels in the files are not unique across elements, the partitions are rebuilt per element through `period_partitions`.
 
 ## Version 0.7.2 (2026-08-04)
 

Project.toml

@@ -54,7 +54,7 @@ Pkg = "1"
 PrettyTables = "3"
 Printf = "1"
 SparseVariables = "0.7"
-TimeStruct = "0.9"
+TimeStruct = "0.9.12"
 XLSX = "0.12"
 YAML = "0.4"
 julia = "1.10, 1.11, 1.12"

docs/src/manual/quick-start.md

@@ -32,6 +32,14 @@ The results from a `JuMP` model can be visualized through the keyword argument `
 GUI(case; model=m)
 ```
 
+A complete overview of the keyword arguments available for the `GUI` functions is available *[in its docstring](@ref GUI(case::Case; kwargs...))*.
+
+!!! tip "Example"
+    The GUI and its functionality is described through *[an example](@ref man-exampl)*.
+    You can also load different examples from the example folder, if desired.
+
+### [Visualization of saved results](@id man-quick-saved)
+
 It is furthermore possible to visualize results from a saved model run.
 This however requires you to first save the results from a model run through the function [`save_results`](@ref).
 You can then visualize the results from a saved model run, again with the keyword argument `model`.
@@ -54,8 +62,31 @@ GUI(case; model = dir_save)
        It **cannot** be a saved `Case` as the pointers to specific instances of, *e.g.*, `Link`s are not recreated when loading a `Case`.
     2. You **must** use the function `save_results` for saving your results as we require the meta data when reading the CSV files for translating the data into the correct format.
 
-A complete overview of the keyword arguments available for the `GUI` functions is available *[in its docstring](@ref GUI(case::Case; kwargs...))*.
+### [Technologies with period partitions](@id man-quick-partitions)
 
-!!! tip "Example"
-    The GUI and its functionality is described through *[an example](@ref man-exampl)*.
-    You can also load different examples from the folder, if desired
+[`TimeStruct`](https://github.com/sintefore/TimeStruct.jl) (version 0.9.12 and later) allows for partitioning the operational periods of a time structure into `PeriodPartition`s.
+Technologies using them, *e.g.*, the `PeriodDemandSink` of [`EnergyModelsFlex`](https://github.com/EnergyModelsX/EnergyModelsFlex.jl), have `PartitionProfile` fields and `JuMP` variables indexed over these partitions.
+`EnergyModelsGUI` visualizes this data through the time axis option *Partition*, which is only available for elements with partitioned data.
+
+Partitions differ between elements and are not part of the time structure of the case.
+`EnergyModelsGUI` therefore has to construct the partitions of each element itself through the function [`period_partitions`](@ref EnergyModelsGUI.period_partitions).
+By default, it uses the field `period_duration` of the element, following the convention of `EnergyModelsFlex`.
+If your technology stores the duration of its partitions differently, you must provide a method for your type returning the vector of `PeriodPartition`s for the time structure `𝒯` of the case.
+Consider the case in which these are stored within the field `duation`, you must create a new method:
+
+```julia
+using EnergyModelsGUI, TimeStruct
+
+function EnergyModelsGUI.period_partitions(n::MyPeriodNode, 𝒯::TimeStructure)
+    return collect(partition_duration(𝒯, n.duration))
+end
+```
+
+The method can be defined in your script or in the package introducing the technology, *e.g.*, through a package extension on `EnergyModelsGUI`.
+It is our aim to include a default method within `EnergyModelsBase` which can be extended in a later stage.
+
+!!! warning "Requirements for period partitions"
+    1. Without a method of `period_partitions` for an element lacking the field `period_duration`, a warning naming the element and the required method is issued and the partitioned data of this element is not available for plotting.
+       All other data of the element and of the case remains available.
+    2. The method is also required when loading results from file.
+       The partition labels written by `save_results` are not unique across elements, so the partitions are rebuilt for each element through `period_partitions` when reading the CSV files.

src/datastructures.jl

@@ -349,6 +349,14 @@ const JuMPContainer = PlotContainer{:JuMP}
 const CaseDataContainer = PlotContainer{:CaseData}
 const GlobalDataContainer = PlotContainer{:GlobalData}
 
+# Define the time axes of the results axis and their labels in the time menu. The
+# partition axis (PARTITION_AXIS) is appended to the base options only for elements
+# with data indexed over `TS.PeriodPartition`s
+const TIME_AXES_LABELS = ["Strategic", "Representative", "Scenario", "Operational"]
+const TIME_AXES = [:results_sp, :results_rp, :results_sc, :results_op]
+const PARTITION_AXIS_LABEL = "Partition"
+const PARTITION_AXIS = :results_pt
+
 # Define standard colours in EMGUI
 const BLACK = RGBA{Float32}(0.0, 0.0, 0.0, 1.0)
 const WHITE = RGBA{Float32}(1.0, 1.0, 1.0, 1.0)

src/descriptive_names.yml

@@ -149,6 +149,16 @@ structures:
       cap_price:              "Price of capacity usage"
       cap_price_periods:      "The number of sub periods in an investment period"
 
+    ## sink/datastructures.jl
+    AbstractPeriodDemandSink:
+      period_duration:        "Duration of each demand period"
+      period_demand:          "Demand within each demand period"
+
+    StratPeriodDemandSink:
+      strat_demand:           "Demand within each strategic period"
+      period_min:             "Minimum fraction of the strategic demand satisfied in each demand period"
+      period_max:             "Maximum fraction of the strategic demand satisfied in each demand period"
+
 
 variables:
   # EnergyModelsBase
@@ -268,6 +278,10 @@ variables:
   solar_cap_use:            "Absolute capacity utilization"
 
   # EnergyModelsFlex
+  demand_sink_surplus:      "Surplus in demand within a demand period"
+  demand_sink_deficit:      "Deficit in demand within a demand period"
+  demand_sink_strat_surplus: "Surplus in demand within a strategic period"
+  demand_sink_strat_deficit: "Deficit in demand within a strategic period"
   input_frac_strat:         "Input resource fraction"
   load_shift_from:          "Load shift from"
   load_shift_to:            "Load shift to"

src/setup_GUI.jl

@@ -166,12 +166,14 @@ function GUI(
         :alpha => Observable(Float32(alpha)),
         :autolimits => Dict(
             :results_op => true,
+            :results_pt => true,
             :results_sc => true,
             :results_rp => true,
             :results_sp => true,
         ),       # Automatically adjust limits of the axis
         :finallimits => Dict(
             :results_op => GLMakie.HyperRectangle(Vec2f(0, 0), Vec2f(1, 1)),
+            :results_pt => GLMakie.HyperRectangle(Vec2f(0, 0), Vec2f(1, 1)),
             :results_sc => GLMakie.HyperRectangle(Vec2f(0, 0), Vec2f(1, 1)),
             :results_rp => GLMakie.HyperRectangle(Vec2f(0, 0), Vec2f(1, 1)),
             :results_sp => GLMakie.HyperRectangle(Vec2f(0, 0), Vec2f(1, 1)),
@@ -589,10 +591,7 @@ function create_makie_objects(vars::Dict, design::EnergySystemDesign)
     )
     time_menu = Makie.Menu(
         gridlayout_results_taskbar1[1, 2];
-        options = zip(
-            ["Strategic", "Representative", "Scenario", "Operational"],
-            [:results_sp, :results_rp, :results_sc, :results_op],
-        ),
+        options = zip(TIME_AXES_LABELS, TIME_AXES),
         halign = :left,
         width = 110 * vars[:fontsize] / 12,
         fontsize = vars[:fontsize],

src/utils_GUI/GUI_utils.jl

@@ -232,7 +232,14 @@ function initialize_available_data!(gui)
 
             for combination ∈ get_combinations(var, i_T)
                 selection = collect(combination)
-                field_data = extract_data_selection(var, selection, i_T, periods)
+                if type <: TS.PeriodPartition && isa(var, SparseVars)
+                    # Partitions may differ between elements; extract them from the
+                    # variable itself for the given selection
+                    periods_comb = get_var_periods(var, selection, i_T)
+                else
+                    periods_comb = periods
+                end
+                field_data = extract_data_selection(var, selection, i_T, periods_comb)
                 element = getfirst(x -> !isa(x, Resource), selection)
                 if !isa(element, AbstractElement) && !isnothing(element) # it must be a transmission
                     element = mode_to_transmission[element]
@@ -581,9 +588,11 @@ get_JuMP_dict(model::JuMP.Model) = object_dictionary(model)
     get_values(vals::SparseVariables.IndexedVarArray, ts::Vector)
     get_values(vals::JuMP.Containers.DenseAxisArray, ts::Vector)
     get_values(vals::DataFrame, ts::Vector)
+    get_values(vals::DataFrame, ts::Vector{<:TS.PeriodPartition})
 
 Get the values of the variables in `vals`. If a vector of time periods `ts` is provided, it
-returns the values for the times in `ts`.
+returns the values for the times in `ts`. For `TS.PeriodPartition`s read from CSV-files, the
+values are returned in the order of `ts`.
 """
 get_values(vals::SparseVars) = isempty(vals) ? [] : collect(Iterators.flatten(value.(vals)))
 get_values(vals::SparseVariables.IndexedVarArray) = collect(value.(values(vals.data)))
@@ -594,6 +603,8 @@ get_values(vals::SparseVariables.IndexedVarArray, ts::Vector) =
     isempty(vals) ? [] : value.(vals[ts])
 get_values(vals::JuMP.Containers.DenseAxisArray, ts::Vector) = Array(value.(vals[ts]))
 get_values(vals::DataFrame, ts::Vector) = vals[in.(vals.t, Ref(ts)), :val]
+get_values(vals::DataFrame, ts::Vector{<:TS.PeriodPartition}) =
+    [vals[findfirst(==(t), vals.t), :val] for t ∈ ts]
 get_values(vals::TimeProfile, ts::Vector) = vals[ts]
 
 """
@@ -677,6 +688,47 @@ function get_investment_times(gui::GUI, max_inst::Float64)
     end
 end
 
+"""
+    get_var_periods(var::SparseVars, selection::Vector, i_T::Int64)
+
+Get the sorted time indices available in `var` at axis `i_T` for the combination
+`selection` of the remaining indices. This is required for variables indexed over
+`TS.PeriodPartition`s as the partitions may differ between elements.
+"""
+function get_var_periods(var::SparseVars, selection::Vector, i_T::Int64)
+    sel = Tuple(selection)
+    pds = [
+        key[i_T] for
+        key ∈ keys(var.data) if (key[1:(i_T-1)]..., key[(i_T+1):end]...) == sel
+    ]
+    return sort(pds; by = partition_sort_key)
+end
+
+"""
+    get_time_periods(field_data::DataFrame)
+    get_time_periods(field_data)
+
+Get the time periods of the extracted data `field_data` of a JuMP variable, *i.e.*, the
+column `:t` if the model results are read from CSV-files and the first axis otherwise.
+"""
+get_time_periods(field_data::DataFrame) = field_data[!, :t]
+get_time_periods(field_data) = first(axes(field_data))
+
+"""
+    is_partition_data(container::PlotContainer)
+
+Return `true` if `container` holds data indexed over `TS.PeriodPartition`s, *i.e.*, a JuMP
+variable with a `TS.PeriodPartition` axis or a field of type `PartitionProfile`.
+"""
+is_partition_data(::PlotContainer) = false
+function is_partition_data(container::JuMPContainer)
+    return eltype(get_time_periods(get_field_data(container))) <: TS.PeriodPartition
+end
+function is_partition_data(container::CaseDataContainer)
+    field_data = get_field_data(container)
+    return isa(field_data, TimeProfile) && nested_eltype(field_data) <: PartitionProfile
+end
+
 """
     get_combinations(var::SparseVars, i_T::Int)
     get_combinations(var::JuMP.Containers.DenseAxisArray, i_T::Int)
@@ -708,6 +760,40 @@ function update_available_data_menu!(gui::GUI, element)
     container = available_data[element]
     container_strings = create_label.(container)
     get_menu(gui, :available_data).options = zip(container_strings, container)
+    update_time_menu!(gui, element)
+end
+
+"""
+    has_partition_data(gui::GUI, element)
+
+Return `true` if `element` has available data indexed over `TS.PeriodPartition`s.
+"""
+function has_partition_data(gui::GUI, element)
+    return any(is_partition_data, get_available_data(gui)[element])
+end
+
+"""
+    update_time_menu!(gui::GUI, element)
+
+Update the options of the time menu based on `element`: The option for plotting data over
+`TS.PeriodPartition`s is only made available if `element` has data indexed over such
+partitions.
+"""
+function update_time_menu!(gui::GUI, element)
+    time_menu = get_menu(gui, :time)
+    labels = copy(TIME_AXES_LABELS)
+    time_axes = copy(TIME_AXES)
+    if has_partition_data(gui, element)
+        push!(labels, PARTITION_AXIS_LABEL)
+        push!(time_axes, PARTITION_AXIS)
+    end
+    if length(collect(time_menu.options[])) != length(time_axes)
+        # Reset the selection if the partition axis is removed while being selected
+        if time_menu.selection[] ∉ time_axes
+            time_menu.i_selected = 1
+        end
+        time_menu.options = zip(labels, time_axes)
+    end
 end
 
 """
@@ -970,13 +1056,18 @@ function transfer_model(model::String, system::AbstractSystem)
 
             df = read_csv(file)
             col_names = names(df)
-            df[!, :t] = convert_array(df[!, :t], periods_dict)
             if "res" ∈ col_names
                 df[!, :res] = convert_array(df[!, :res], products_dict)
             end
             if "element" ∈ col_names
                 df[!, :element] = convert_array(df[!, :element], plotables_dict)
             end
+            if "pd" ∈ col_names
+                # Partitions may differ between elements and are reconstructed per element
+                df = convert_partitions(df, 𝒯, varname)
+            else
+                df[!, :t] = convert_array(df[!, :t], periods_dict)
+            end
 
             results[i] = varname => df
         end
@@ -989,6 +1080,49 @@ function transfer_model(model::String, system::AbstractSystem)
     return data
 end
 
+"""
+    convert_partitions(df::DataFrame, 𝒯::TimeStructure, varname::Symbol)
+
+Convert the string representations of the `TS.PeriodPartition`s in column `:pd` of `df` to
+the partitions themselves and store them in column `:t`.
+
+As the string representation of a partition does not include its operational periods and
+the partitions may differ between elements, the partitions are reconstructed for each
+element in column `:element` through [`period_partitions`](@ref). Rows whose partition
+cannot be reconstructed are removed with a warning highlighting the element.
+"""
+function convert_partitions(df::DataFrame, 𝒯::TimeStructure, varname::Symbol)
+    if !("element" ∈ names(df))
+        @warn "The variable `$varname` is indexed over `PeriodPartition`s without an " *
+              "element index. Its partitions cannot be reconstructed and it is skipped."
+        return DataFrame()
+    end
+    partitions_dicts = Dict{Any,Dict}()
+    unmatched_elements = Set()
+    partitions = Vector{Union{Nothing,TS.PeriodPartition}}(undef, nrow(df))
+    for (i, row) ∈ enumerate(eachrow(df))
+        element = row[:element]
+        partitions_dict = get!(partitions_dicts, element) do
+            get_repr_dict(period_partitions(element, 𝒯))
+        end
+        partitions[i] = get(partitions_dict, string(row[:pd]), nothing)
+        # Elements without any partitions are already highlighted by `period_partitions`
+        if isnothing(partitions[i]) && !isempty(partitions_dict)
+            push!(unmatched_elements, element)
+        end
+    end
+    for element ∈ unmatched_elements
+        @warn "The `PeriodPartition`s of element `$element` in the variable `$varname` do " *
+              "not match the partitions provided by `period_partitions`. The unmatched " *
+              "values are skipped."
+    end
+
+    keep = .!isnothing.(partitions)
+    df = df[keep, Not(:pd)]
+    df[!, :t] = TS.PeriodPartition[pd for pd ∈ partitions[keep]]
+    return select!(df, Not(:val), :val) # Keep the values in the last column
+end
+
 """
     read_csv(file::String)