Asymmetric line flow (Refined); READY FOR REVIEW !!!

CHANGELOG.md

@@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+### Changed
+- Modified `transmission.jl`, `investment_transmission.jl`, `load_network_data.jl`, 
+`write_transmission_flows.jl`, `write_transmision_losses.jl`, and `write_network_expansion.jl`
+for implementing asymmetric bidirectional flows. (#789)
+### Changed
+- Changed the output filename for RSV from `reg_dn.csv` to `reserves.csv` (#814).
+- Created link in the thermal doc page to fuels page for piecewise linear cost curve documentation (#820).
+- Doc rendering fix in fuel.jl (#821).
+
 ### Added
 - New resource type: Allam Cycle with Oxygen Storage (#772).
 - Caching of `derating_factor` to improve performance in model generation (#834).
@@ -25,11 +34,6 @@ Models running with, non-default, solvers Cbc and Clp will fail unless
 
 ## [0.4.3] - 2025-01-29
 
-### Changed
-- Changed the output filename for RSV from `reg_dn.csv` to `reserves.csv` (#814).
-- Created link in the thermal doc page to fuels page for piecewise linear cost curve documentation (#820).
-- Doc rendering fix in fuel.jl (#821).
-
 ### Fixed
 - Fixed capacity reserve margin formulation for asymmetric storage
 when OperationalReserves is on. (#818)

Project.toml

@@ -12,6 +12,7 @@ DataStructures = "864edb3b-99cc-5e75-8d2d-829cb0a9cfe8"
 Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
 Distances = "b4f34e82-e78d-54a5-968a-f98e89d6e8f7"
 HiGHS = "87dc4568-4c63-4d18-b0c0-bb2238e4078b"
+Infiltrator = "5903a43b-9cc3-4c30-8d17-598619ec4e9b"
 JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
 Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"

example_systems/1_three_zones_asymmetric_bidirectional/README.md

@@ -0,0 +1,32 @@
+# Three Zones
+
+**Three Zones**, a one-year example with hourly resolution, contains zones representing Massachusetts, Connecticut, and Maine. The ten represented resources include natural gas, solar PV, wind, and lithium-ion battery storage.
+
+To run the model, first navigate to the example directory:
+
+- Using a Julia REPL:
+
+```bash
+$ julia
+julia> cd("example_systems/1_three_zones/")
+```
+
+- Using a terminal or command prompt:
+```bash
+$ cd example_systems/1_three_zones/
+``` 
+
+Next, ensure that your settings in `settings/genx_settings.yml` are correct. The default settings use the solver `HiGHS`, time domain reduced input data (`TimeDomainReduction: 1`) and minimum capacity requirement policy (`MinCapReq: 1`) as specified in the `policies/Minimum_capacity_requirement.csv` file. Other optional policies include a capacity reserve margin, an energy share requirement (such as renewable portfolio standard (RPS) or clean electricity standard (CES) policies), a CO2 emissions cap, and a maximum capacity requirement policy (see the documentation for more details). For this example, a rate-based carbon cap of 50 gCO<sub>2</sub> per kWh is specified in the `policies/CO2_cap.csv` input file.
+
+Once the settings are confirmed, run the model with the `Run.jl` script in the example directory:
+
+- Using a Julia REPL (recommended)
+```julia
+julia> include("Run.jl")
+```
+- Using a terminal or command prompt:
+```bash
+$ julia Run.jl
+```
+
+Once the model has completed, results will write to the `results` directory.

example_systems/1_three_zones_asymmetric_bidirectional/Run.jl

@@ -0,0 +1,3 @@
+using GenX
+
+run_genx_case!(dirname(@__FILE__))

example_systems/1_three_zones_asymmetric_bidirectional/policies/CO2_cap.csv

@@ -0,0 +1,4 @@
+,Network_zones,CO_2_Cap_Zone_1,CO_2_Cap_Zone_2,CO_2_Cap_Zone_3,CO_2_Max_tons_MWh_1,CO_2_Max_tons_MWh_2,CO_2_Max_tons_MWh_3,CO_2_Max_Mtons_1,CO_2_Max_Mtons_2,CO_2_Max_Mtons_3
+MA,z1,1,0,0,0.05,0,0,0.018,0,0
+CT,z2,0,1,0,0,0.05,0,0,0.025,0
+ME,z3,0,0,1,0,0,0.05,0,0,0.025

example_systems/1_three_zones_asymmetric_bidirectional/policies/Minimum_capacity_requirement.csv

@@ -0,0 +1,4 @@
+MinCapReqConstraint,ConstraintDescription,Min_MW
+1,MA_PV,5000
+2,CT_Wind,10000
+3,All_Batteries,6000

example_systems/1_three_zones_asymmetric_bidirectional/resources/Storage.csv

@@ -0,0 +1,4 @@
+Resource,Zone,Model,New_Build,Can_Retire,Existing_Cap_MW,Existing_Cap_MWh,Max_Cap_MW,Max_Cap_MWh,Min_Cap_MW,Min_Cap_MWh,Inv_Cost_per_MWyr,Inv_Cost_per_MWhyr,Fixed_OM_Cost_per_MWyr,Fixed_OM_Cost_per_MWhyr,Var_OM_Cost_per_MWh,Var_OM_Cost_per_MWh_In,Self_Disch,Eff_Up,Eff_Down,Min_Duration,Max_Duration,Reg_Max,Rsv_Max,Reg_Cost,Rsv_Cost,region,cluster
+MA_battery,1,1,1,0,0,0,-1,-1,0,0,19584,22494,4895,5622,0.15,0.15,0,0.92,0.92,1,10,0,0,0,0,MA,0
+CT_battery,2,1,1,0,0,0,-1,-1,0,0,19584,22494,4895,5622,0.15,0.15,0,0.92,0.92,1,10,0,0,0,0,CT,0
+ME_battery,3,1,1,0,0,0,-1,-1,0,0,19584,22494,4895,5622,0.15,0.15,0,0.92,0.92,1,10,0,0,0,0,ME,0

example_systems/1_three_zones_asymmetric_bidirectional/resources/Thermal.csv

@@ -0,0 +1,4 @@
+Resource,Zone,Model,New_Build,Can_Retire,Existing_Cap_MW,Max_Cap_MW,Min_Cap_MW,Inv_Cost_per_MWyr,Fixed_OM_Cost_per_MWyr,Var_OM_Cost_per_MWh,Heat_Rate_MMBTU_per_MWh,Fuel,Cap_Size,Start_Cost_per_MW,Start_Fuel_MMBTU_per_MW,Up_Time,Down_Time,Ramp_Up_Percentage,Ramp_Dn_Percentage,Min_Power,Reg_Max,Rsv_Max,Reg_Cost,Rsv_Cost,region,cluster
+MA_natural_gas_combined_cycle,1,1,1,0,0,-1,0,65400,10287,3.55,7.43,MA_NG,250,91,2,6,6,0.64,0.64,0.468,0.25,0.5,0,0,MA,1
+CT_natural_gas_combined_cycle,2,1,1,0,0,-1,0,65400,9698,3.57,7.12,CT_NG,250,91,2,6,6,0.64,0.64,0.338,0.133332722,0.266665444,0,0,CT,1
+ME_natural_gas_combined_cycle,3,1,1,0,0,-1,0,65400,16291,4.5,12.62,ME_NG,250,91,2,6,6,0.64,0.64,0.474,0.033333333,0.066666667,0,0,ME,1

example_systems/1_three_zones_asymmetric_bidirectional/resources/Vre.csv

@@ -0,0 +1,5 @@
+Resource,Zone,Num_VRE_Bins,New_Build,Can_Retire,Existing_Cap_MW,Max_Cap_MW,Min_Cap_MW,Inv_Cost_per_MWyr,Fixed_OM_Cost_per_MWyr,Var_OM_Cost_per_MWh,Reg_Max,Rsv_Max,Reg_Cost,Rsv_Cost,region,cluster
+MA_solar_pv,1,1,1,0,0,-1,0,85300,18760,0,0,0,0,0,MA,1
+CT_onshore_wind,2,1,1,0,0,-1,0,97200,43205,0.1,0,0,0,0,CT,1
+CT_solar_pv,2,1,1,0,0,-1,0,85300,18760,0,0,0,0,0,CT,1
+ME_onshore_wind,3,1,1,0,0,-1,0,97200,43205,0.1,0,0,0,0,ME,1

example_systems/1_three_zones_asymmetric_bidirectional/resources/policy_assignments/Resource_minimum_capacity_requirement.csv

@@ -0,0 +1,6 @@
+Resource,Min_Cap_1,Min_Cap_2,Min_Cap_3
+MA_solar_pv,1,0,0
+CT_onshore_wind,0,1,0
+MA_battery,0,0,1
+CT_battery,0,0,1
+ME_battery,0,0,1

example_systems/1_three_zones_asymmetric_bidirectional/settings/clp_settings.yml

@@ -0,0 +1,14 @@
+# Clp Solver parameters https://github.com/jump-dev/Clp.jl
+# Common solver settings
+Feasib_Tol: 1e-5              # Primal/Dual feasibility tolerance
+TimeLimit: -1.0               # Terminate after this many seconds have passed. A negative value means no time limit
+Pre_Solve: 0                  # Set to 1 to disable presolve
+Method: 5                     # Solution method: dual simplex (0), primal simplex (1), sprint (2), barrier with crossover (3), barrier without crossover (4), automatic (5)
+
+#Clp-specific solver settings
+DualObjectiveLimit: 1e308     # When using dual simplex (where the objective is monotonically changing), terminate when the objective exceeds this limit
+MaximumIterations: 2147483647 # Terminate after performing this number of simplex iterations
+LogLevel: 1                   # Set to 1, 2, 3, or 4 for increasing output. Set to 0 to disable output
+InfeasibleReturn: 0           # Set to 1 to return as soon as the problem is found to be infeasible (by default, an infeasibility proof is computed as well)
+Scaling: 3                    # 0 -off, 1 equilibrium, 2 geometric, 3 auto, 4 dynamic(later)
+Perturbation: 100             # switch on perturbation (50), automatic (100), don't try perturbing (102)

example_systems/1_three_zones_asymmetric_bidirectional/settings/cplex_settings.yml

@@ -0,0 +1,10 @@
+# CPLEX Solver Parameters
+Feasib_Tol: 1.0e-05 # Constraint (primal) feasibility tolerances.
+Optimal_Tol: 1e-5 # Dual feasibility tolerances.
+Pre_Solve: 1 # Controls presolve level.
+TimeLimit: 110000 # Limits total time solver.
+MIPGap: 1e-3 # Relative (p.u. of optimal) mixed integer optimality tolerance for MIP problems (ignored otherwise).
+Method: 2 # Algorithm used to solve continuous models (including MIP root relaxation).
+BarConvTol: 1.0e-08 # Barrier convergence tolerance (determines when barrier terminates).
+NumericFocus: 0 # Numerical precision emphasis.
+SolutionType: 2 # Solution type for LP or QP.

example_systems/1_three_zones_asymmetric_bidirectional/settings/genx_settings.yml

@@ -0,0 +1,14 @@
+NetworkExpansion: 1 # Transmission network expansionl; 0 = not active; 1 = active systemwide
+Trans_Loss_Segments: 1 # Number of segments used in piecewise linear approximation of transmission losses; 1 = linear, >2 = piecewise quadratic
+EnergyShareRequirement: 0 # Minimum qualifying renewables penetration; 0 = not active; 1 = active systemwide
+CapacityReserveMargin: 0 # Number of capacity reserve margin constraints; 0 = not active; 1 = active systemwide
+CO2Cap: 2 # CO2 emissions cap; 0 = not active (no CO2 emission limit); 1 = mass-based emission limit constraint; 2 = demand + rate-based emission limit constraint; 3 = generation + rate-based emission limit constraint
+StorageLosses: 1 # Energy Share Requirement and CO2 constraints account for energy lost; 0 = not active (DO NOT account for energy lost); 1 = active systemwide (DO account for energy lost)
+MinCapReq: 1  # Activate minimum technology carveout constraints; 0 = not active; 1 = active
+MaxCapReq: 0  # Activate maximum technology carveout constraints; 0 = not active; 1 = active
+ParameterScale: 1 # Turn on parameter scaling wherein demand, capacity and power variables are defined in GW rather than MW. 0 = not active; 1 = active systemwide
+WriteShadowPrices: 1 # Write shadow prices of LP or relaxed MILP; 0 = not active; 1 = active
+UCommit: 2 # Unit committment of thermal power plants; 0 = not active; 1 = active using integer clestering; 2 = active using linearized clustering
+TimeDomainReduction: 1 # Time domain reduce (i.e. cluster) inputs based on Demand_data.csv, Generators_variability.csv, and Fuels_data.csv; 0 = not active (use input data as provided); 0 = active (cluster input data, or use data that has already been clustered)
+OutputFullTimeSeries: 1
+AsymmetricalTransFlowLimit: 1 # Switch to indicate if asymmetrical bidirectional lines are included in the system or not

example_systems/1_three_zones_asymmetric_bidirectional/settings/gurobi_settings.yml

@@ -0,0 +1,15 @@
+# Gurobi Solver Parameters
+# Common solver settings
+Feasib_Tol: 1.0e-05           # Constraint (primal) feasibility tolerances.
+Optimal_Tol: 1e-5             # Dual feasibility tolerances.
+TimeLimit: 110000             # Limits total time solver.
+Pre_Solve: 1                  # Controls presolve level.
+Method: 4                     # Algorithm used to solve continuous models (including MIP root relaxation).
+
+#Gurobi-specific solver settings
+MIPGap: 1e-3                  # Relative (p.u. of optimal) mixed integer optimality tolerance for MIP problems (ignored otherwise).
+BarConvTol: 1.0e-08           # Barrier convergence tolerance (determines when barrier terminates).
+NumericFocus: 0               # Numerical precision emphasis.
+Crossover: -1                 # Barrier crossver strategy.
+PreDual: 0                    # Decides whether presolve should pass the primal or dual linear programming problem to the LP optimization algorithm.
+AggFill: 10                   # Allowed fill during presolve aggregation.

example_systems/1_three_zones_asymmetric_bidirectional/settings/highs_settings.yml

@@ -0,0 +1,11 @@
+# HiGHS Solver Parameters
+# Common solver settings
+Feasib_Tol: 1.0e-05        # Primal feasibility tolerance # [type: double, advanced: false, range: [1e-10, inf], default: 1e-07]
+Optimal_Tol: 1.0e-05       # Dual feasibility tolerance # [type: double, advanced: false, range: [1e-10, inf], default: 1e-07]
+TimeLimit: 1.0e23             # Time limit # [type: double, advanced: false, range: [0, inf], default: inf]
+Pre_Solve: choose          # Presolve option: "off", "choose" or "on" # [type: string, advanced: false, default: "choose"]
+Method: ipm           #HiGHS-specific solver settings # Solver option: "simplex", "choose" or "ipm" # [type: string, advanced: false, default: "choose"]
+
+# run the crossover routine for ipx
+# [type: string, advanced: "on", range: {"off", "on"}, default: "off"]
+run_crossover: "on"

example_systems/1_three_zones_asymmetric_bidirectional/settings/time_domain_reduction_settings.yml

@@ -0,0 +1,151 @@
+#####
+#
+#  TIME DOMAIN REDUCTION SETTINGS
+#
+#  Set parameters here that organize how your full timeseries
+#   data will be divided into representative period clusters.
+#   Ensure that time_domain_reduction is set to 1 in GenX_settings.yml
+#   before running. Run within GenX or use PreCluster.jl to test and
+#   examine representative period output before proceeding.
+#   Specify your data input directory as inpath within Run_test.jl
+#   or PreCluster.jl.
+#
+#####
+
+  #   - TimestepsPerRepPeriod
+  #   Typically 168 timesteps (e.g., hours) per period, this designates
+  #   the length of each representative period.
+TimestepsPerRepPeriod: 168
+
+  #   - ClusterMethod
+  #   Either 'kmeans' or 'kmedoids', this designates the method used to cluster
+  #   periods and determine each point's representative period.
+ClusterMethod: 'kmeans'
+
+  #   - ScalingMethod
+  #   Either 'N' or 'S', this designates directs the module to normalize ([0,1])
+  #   or standardize (mean 0, variance 1) the input data.
+ScalingMethod: "S"
+
+  #   - MaxPeriods
+  #   The maximum number of periods - both clustered periods and extreme periods -
+  #   that may be used to represent the input data. If IterativelyAddPeriods is on and the
+  #   error threshold is never met, this will be the total number of periods.
+MaxPeriods: 11
+
+  #   - MinPeriods
+  #   The minimum number of periods used to represent the input data. If using
+  #   UseExtremePeriods, this must be at least the number of extreme periods requests. If
+  #   IterativelyAddPeriods if off, this will be the total number of periods.
+MinPeriods: 8
+
+  #   - IterativelyAddPeriods
+  #   Either 'yes' or 'no', this designates whether or not to add periods
+  #   until the error threshold between input data and represented data is met or the maximum
+  #   number of periods is reached.
+IterativelyAddPeriods: 1
+
+  #   - IterateMethod
+  #   Either 'cluster' or 'extreme', this designates whether to add clusters to
+  #   the kmeans/kmedoids method or to set aside the worst-fitting periods as a new extreme periods.
+  #   The default option is 'cluster'.
+IterateMethod: "cluster"
+
+  #   - Threshold
+  #   Iterative period addition will end if the period farthest (Euclidean Distance)
+  #   from its representative period is within this percentage of the total possible error (for normalization)
+  #   or ~95% of the total possible error (for standardization). E.g., for a threshold of 0.01,
+  #   every period must be within 1% of the spread of possible error before the clustering
+  #   iterations will terminate (or until the max number of periods is reached).
+Threshold: 0.05
+
+  #   - nReps
+  #   The number of times to repeat each kmeans/kmedoids clustering at the same setting.
+nReps: 100
+
+  #   - DemandWeight
+  #   Default 1, this is an optional multiplier on demand columns in order to prioritize
+  #   better fits for demand profiles over resource capacity factor profiles.
+DemandWeight: 1
+
+  #   - WeightTotal
+  #   Default 8760, the sum to which the relative weights of representative periods will be scaled.
+WeightTotal: 8760
+
+  #   - ClusterFuelPrices
+  #   Either 1 (yes) or 0 (no), this indicates whether or not to use the fuel price
+  #   time series in Fuels_data.csv in the clustering process. If 0, this function will still write
+  #   Fuels_data_clustered.csv with reshaped fuel prices based on the number and size of the
+  #   representative weeks, assuming a constant time series of fuel prices with length equal to the
+  #   number of timesteps in the raw input data.
+ClusterFuelPrices: 1
+
+  #   - UseExtremePeriods
+  #   Either 'yes' or 'no', this designates whether or not to include
+  #   outliers (by performance or demand/resource extreme) as their own representative periods.
+  #   This setting automatically includes the periods with maximum demand, minimum solar cf and
+  #   minimum wind cf as extreme periods.
+UseExtremePeriods: 1
+
+  #   - MultiStageConcatenate
+  #   (Only considered if MultiStage = 1 in genx_settings.yml)
+  #   If 1, this designates that the model should time domain reduce the input data
+  #   of all model stages together. Else if 0, the model will time domain reduce each
+  #   stage separately
+MultiStageConcatenate: 0
+
+# STILL IN DEVELOPMENT - Currently just uses integral max demand, integral min PV and wind.
+#   - ExtremePeriods
+#   Use this to define which periods to be included among the final representative periods
+#   as "Extreme Periods".
+#   Select by profile type: demand ("Demand"), solar PV capacity factors ("PV"), and wind capacity factors ("Wind").
+#   Select whether to examine these profiles by zone ("Zone") or across the whole system ("System").
+#   Select whether to look for absolute max/min at the timestep level ("Absolute")
+#      or max/min sum across the period ("Integral").
+#   Select whether you want the maximum ("Max") or minimum ("Min") (of the prior type) for each profile type.
+ExtremePeriods:
+   Demand:
+      Zone:
+         Absolute:
+            Max: 0
+            Min: 0
+         Integral:
+            Max: 0
+            Min: 0
+      System:
+         Absolute:
+            Max: 1
+            Min: 0
+         Integral:
+            Max: 0
+            Min: 0
+   PV:
+      Zone:
+         Absolute:
+            Max: 0
+            Min: 0
+         Integral:
+            Max: 0
+            Min: 1
+      System:
+         Absolute:
+            Max: 0
+            Min: 0
+         Integral:
+            Max: 0
+            Min: 0
+   Wind:
+      Zone:
+         Absolute:
+            Max: 0
+            Min: 0
+         Integral:
+            Max: 0
+            Min: 1
+      System:
+         Absolute:
+            Max: 0
+            Min: 0
+         Integral:
+            Max: 0
+            Min: 0