macroenergy / dolphyn.jl

DOLPHYN: Decision Optimization for Low Carbon Power and Hydrogen Nexus

DOLPHYN

Overview

DOLPHYN is a configurable, open source energy system optimization model developed to explore interactions between multiple energy vectors and emerging technologies across their supply chains as part of a future integrated low-carbon energy system.

The DOLPHYN model evaluates investments and operations across the electricity and Hydrogen (H2) supply chains, including production, storage, transmission, conditioning, and end-use consumption. Importantly, the model is able to capture interactions between electricity and hydrogen infrastructure through: a) using hydrogen for power generation and b) production of hydrogen using electricity. The model is set up as a single-stage investment planning model and determines the least-cost mix of electricity and H2 infrastructure to meet electricity and H2 demand subject to a variety of operational, policy and carbon emission constraints. The DOLPHYN model incorporates the GenX electricity system model to characterize electricity system operations and investments (v0.3.6). Periodically, the electricity system representation is regularly updated to the latest GenX version.

DOLPHYN is designed to be highly flexible and configurable, for use in a variety of applications from academic research and technology evaluation to public policy and regulatory analysis and resource planning. We are currently working to add biofuel supply chains and carbon capture, transport, and storage to the model.

We welcome you to add new features and resources to DOLPHYN for use in your own work and to share them here for others. If you have issues using DOLPHYN please let us know by opening an issue.

Requirements

DOLPHYN is written in Julia. We recommend using the latest version of Julia to run DOLPHYN but will support all versions back to the current long term stable version.

You need a mathematical programme solver to run DOLPHYN. DOLPHYN comes packaged with HiGHS, a free open-source solver. This will be installed automatically by the HiGHS.jl package when you use DOLPHYN.

DOLPHYN also works with several other open-source and commercial solvers via the JuMP.jl package. DOLPHYN is most extensively tested using Gurobi, a commercial solver requiring a paid commercial license or free academic license

DOLPHYN also has limited support for:

• CPLEX - a commercial solver requiring a paid commercial license or free academic license
• Clp - a free open-source solver
• Cbc - a free open-source solver

Installing and Running DOLPHYN

DOLPHYN is available as a Julia package. To install DOLPHYN, open the Julia REPL and run:

using Pkg

Pkg.add("Dolphyn")

Running your first example

Download the example systems by downloading or cloning the Dolphyn.jl repository by following the instruction here. Navigate to one of the example systems, e.g.:

julia> cd("Example_Systems/SmallNewEngland/OneZone")

Here you will find:

• Input data contained in several CSV files
• A settings folder containing:
  • global_model_settings.yml: settings across all sectors
  • gens_settings.yml: setting for the electricity sector
  • hsc_settings.yml: settings for the hydrogen sector
  • solver settings, e.g. highs_settings.yml
• A Run.jl file which constructs, runs, and outputs the model

To run the model, ensure you are not in the package manager by hitting the backspace key, then execute the Run.jl file:

julia> include("Run.jl")

Once the model has completed running, results will be written into the "Results" folder. The hydrogen sector results are saved inthe Results/Results_HSC folder.

Example Systems

SmallNewEngland: OneZone is a one-year example with hourly resolution representing Massachusetts. A rate-based carbon cap of 50 gCO₂ per kWh is specified in the CO2_cap.csv input file. Expect a run time of ~5 seconds.

SmallNewEngland: ThreeZones is similar to the above example but contains zones representing Massachusetts, Connecticut, and Maine. Expect a run time of ~1 minute.

NorthSea_2030 is a combined power and hydrogen model for the EU for the year 2030. It contains a power model with hourly resolution, contains zones representing Belgium, Germany, Denmark, France, Great Britain, the Netherlands, Sweden, and Norway. The model also includes a CO2 constraint representing 30% of 2015 power sector CO2 emissions applied to the hydrogen and power sector jointly. Expect a run time of ~10 minutes.

Creating Your Own System

The DOLPHYN model is designed to be highly flexible and configurable. We recommend using the example input files and Run.jl file as templates. The Model Inputs / Outputs Documentation details the input files and data required for different sectors, resources, and policies.

Installing Gurobi or an alternative solver

HiGHS will be automatically downloaded and installed when you install DOLPHYN so you do not need to download it separately. However if you would like to use a specific version of have a separate copy, it can be downloaded from: https://highs.dev/.

If you wish to use another solver you will need to install it separately and add the corresponding Julia package to your Julia environment. Most solvers supported by JuMP.jl should be compatible. The instructions below are for Gurobi, but the process is similar for other solvers.

To use Gurobi instead of HiGHS to solve the .../SmallNewEngland/OneZone example:

• Download and install the latest version of the Gurobi Optimizer from: https://www.gurobi.com/downloads/gurobi-software/
• Install the Gurobi Julia package by running the following commands in the Julia REPL:
  • using Pkg
  • Pkg.add("Gurobi")
  • Pkg.build("Gurobi")
• In the Settings/global_model_settings.yml file, change the solver to Gurobi by changing the line Solver: HiGHS to Solver: Gurobi
• Add the following line to the Run.jl file:
  • using Gurobi

If the step to build Gurobi fails, the most likely cause is that the Gurobi installation cannot be found. Use the following instructions to define the "GUROBI_HOME" and "GRB_LICENSE_FILE" environment variables on your computer. For example, for Gurobi 10.0 on Ubuntu they should point to:

• GUROBI_HOME = ...path to Gurobi install/gurobi1000/linux64
• GRB_LICENSE_FILE = ...path to Gurobi install/gurobi1000/gurobi.lic

Once you have successfully built Gurobi.jl, run the model as described above. The model should now run using Gurobi instead of HiGHS.

Installing DOLPHYN as a Developer

Intalling DOLPHYN via the package manager will allow you to use Dolphyn and it is possible to install specific versions from branches or forks of the Dolphyn.jl repo by specifying the URL when adding the package.

However, if you would like to extend, modify or contribute to DOLPHYN then we recommend cloning the repository and installing it as a developer. This will allow you to make changes to the code, test them, and contribute them back to the main repository.

If you are doing a fresh install

ZIP download

If you would like a one-time download of DOLPHYN which is not set up to pull updates using git, then simply download and unzip the files using this link.

Fresh Install Using GitHub Desktop

Use the File -> Clone Respository -> URL dropdown menu to clone the DOLPHYN repository from:

https://github.com/macroenergy/Dolphyn.jl.git

Fresh Install Using GitHub via your terminal / command line

In the top-level folder where you want to place DOLPHYN, run:

git clone https://github.com/macroenergy/Dolphyn.jl.git

If you are working from an existing project

Existing Project Using GitHub Desktop

Pull the latest version of DOLPHYN from the main branch. If you previously used the verison of DOLPHYN where GenX was a submodule (as opposed to ordinary folder as it is now) then some of the submodule config files may remain in your project. The easiest solution is to delete your entire DOLPHYN folder from your computer and re-clone the repository.

Existing Project Using GitHub via your terminal / command line

In your top-level folder (generally DOLPHYN or DOLPHYN-DEV), run:

git pull

git checkout main

Setting up the Julia environment

In order to run DOLPHYN from the cloned repo, several Julia packages must be downloaded and installed. To help users install the correct packages and versions, we have created a Julia environment file. This file is located in the top-level DOLPHYN folder and is called Project.toml.

First time running DOLPHYN

The first time you run DOLPHYN, you must instantiate the Julia environment. This will download and install all the required packages.

In your command terminal (not the Julia REPL), navigate to your DOLPHYN folder then run the following commands:

• julia --project=. (this starts the Julia REPL using the environment file found in the current directory)
• julia> ] (Enter ']' at the prompt)
• (DOLPHYN) pkg> instantiate (you should see DOLPHYN project name here, if not, enter activate .)

If you plan to use Gurobi or another solver other than HiGHS, you should add the corresponding Julia package to your Julia environment, as described in the section above.

Here is a snapshot for you to see the commands in action. The user instantiates the environment and then builds Gurobi. Not shown, is that they added the Gurobi package to the environment using add Gurobi before instantiating.

You can now press backspace to exit the Julia package manager and start using DOLPHYN by running your first example.

Second+ time running DOLPHYN:

In your command terminal (not the Julia REPL), navigate to your DOLPHYN folder then run the following commands:

julia --project=.

julia> ]

(DOLPHYN) pkg> st (this is for checking the status of packages installed for DOLPHYN)

DOLPHYN Team

The model was originally developed by Guannan He while at the MIT Energy Initiative, and is now maintained by a team contributors at MITEI led by Dharik Mallapragada and Ruaridh Macdonald as well as Guannan He's research group at Peking University. Key contributors include Dharik S. Mallapragada, Ruaridh Macdonald, Guannan He, Mary Bennett, Shantanu Chakraborty, Anna Cybulsky, Michael Giovanniello, Jun Wen Law, Youssef Shaker, Nicole Shi and Yuheng Zhang.

dolphyn.jl's Issues

Output File Zone Value Incorrect

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

After ran the Run.jl file from Example_Systems/SmallNewEngland/ThreeZones, I discovered that the file HSC_h2_pipeline_flow.csv has incorrect data for Zone.

This is a bug and should be fixed.

Expected Behavior

The value for Zone should be 1, 2, 3 these integers for the ThreeZones output file.

Steps To Reproduce

1. Follow the steps to run Example cases for DOLPHYN from readme.md
2. Choose Example_Systems/SmallNewEngland/ThreeZones/Run.jl to run
3. After step 2 runs successfully, open HSC_h2_pipeline_flow.csv from the Results_HSC folder and check the values for Zone.

Environment

- OS: MacBook Pro Ventura 13.6
- Julia version: 1.6.7
- Solver: Gurobi
- Solver version: 10.0.3

Manifest.toml file

Manifest.zip

Anything else?

No response

ModelH2Truck always on?

I have noticed that no matter I set ModelH2Trucks to 0 or 1, it will always run and produce H2 trucks outputs.

Add data documentation of MACRO_v0 (BESC) to main

What new feature do you propose?

MACRO_v0 is a branch of Dolphyn and has content of its own. With a github's standard way, MACRO_v0 can not publish its documentation from the same site. This is an alternative way to publish data documentation of MACRO_v0 within Dolphyn's documentation.

Please link any issues this feature would resolve

No response

What suggestions do you have for implementing this feature?

Will add BESC from MACRO_v0 a section of its own on Dolphyn's documentation main page. It only includes the data not functions from MACRO_v0. Will add text such that where to locate the functions and the rest of MACRO_v0.

Please add any relevant labels using the drop-down menu on the right.

• I have added all relevant labels.

Use forked version of GenX

We’re currently using an older, frozen, version of GenX in Dolphyn. This means we’re not benefitting from updates being made to GenX and there’s increasing amounts of duplication of work.

To resolve this, I’ll add a forked version of GenX as a submodule.

This issue relates to getting a minimum working version of this. It’ll rely on some hardcoded patching between the two. We have started an effort to rewrite GenX and Dolphyn to work together better and be more extensible to other sectors.

[Feature]: Convert HSC units from mass-based units (e.g. tonne H2) to energy-based units (MWh based on HHV of H2)

What new feature do you propose?

[WIP]

It will be advantageous for Dolphyn to manage H2 resources and demand in units of energy rather than mass. Let's convert all the mass units to energy units based on the higher heat value of H2.

• Change input handling
• Change output writing
• Change internal logic for energy conversions

Please link any issues this feature would resolve

No response

What suggestions do you have for implementing this feature?

Tell us your way to implement this feature. Any suggestions?

Please add any relevant labels using the drop-down menu on the right.

• I have added all relevant labels.

Missing GenX documentation in main branch

1. When reading GenX submodule, we are missing GenX documentation.
2. GenX power balance documentation should include terms for electricity consumption related to H2 production

Index.md update

Update Index.md file to reflect description of DOLPHYN model with appropriate acknowledgement of the source code adopted from GenX github repository.

Transmission Expansion parameter scaling issue in output file

When using parameter scaling = 1, the network_expansion.csv file is written in GW instead of MW (i.e. the scaling is not re-applied to the outputs).

Load-based Emissions Limit

https://github.com/macroenergy/DOLPHYN/blob/4dbb41c7163eeb58196dee198761aa54799f7153/src/multisector/genx_hsc/co2_cap_power_hsc.jl#L61-L80

Should energy-based emissions limit take into account demand by electrolyzers?

In the context of CSC and SF, should the energy demand include power demand by DAC and SF units?

H2 Storage incorrect set names

https://github.com/MacroEnergyProject/DOLPHYN-dev/blob/1a8c3bf3dfd92c82aca1a37546f9e66d2726dbd8/src/HSC/model/storage/h2_storage_all.jl#L142

this constraints indexes over H2_STOR_SHORT_DURATION, this should be H2_STOR_LONG_DURATION instead. Moreover, the H2_STOR_LONG_DURATION is not defined. Instead the following line at the beginning of the function should be added H2_STOR_LONG_DURATION = inputs["H2_STOR_LONG_DURATION"]

Additionally, incorrect set names in https://github.com/MacroEnergyProject/DOLPHYN-dev/blob/1a8c3bf3dfd92c82aca1a37546f9e66d2726dbd8/src/HSC/model/storage/h2_long_duration_storage.jl#L98 and https://github.com/MacroEnergyProject/DOLPHYN-dev/blob/1a8c3bf3dfd92c82aca1a37546f9e66d2726dbd8/src/HSC/model/storage/h2_long_duration_storage.jl#L102

Should be H2_STOR_LONG_DURATION instead of STOR_LONG_DURATION

Liquefaction documentation

Need to add documentation to describe new features related to liquefaction modeling

Documentation Building Errors

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

1. Warnings about unexpected Julia interpolation of type Symbol in the Markdown:

Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :\
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215
┌ Warning: Unexpected Julia interpolation of type Symbol in the Markdown.
│   value = :/
└ @ Documenter.HTMLWriter ~/.julia/packages/Documenter/nQAq5/src/html/HTMLWriter.jl:2215

2. A list of functions did not get documented successfully when documentation is being built. Here is the output:

[ Info: CheckDocument: running document checks.
┌ Warning: 120 docstrings not included in the manual:
│└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
[ Info: Populate: populating indices.
[ Info: RenderDocument: rendering document.

3. These warnings are about duplicate docs:

Warning: duplicate docs found for 'DOLPHYN.hydro_inter_period_linkage!' in src/hydro_res.md:2-8
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = [
│     "GenX/src/model/resources/hydro/hydro_res.jl",
│     "GenX/src/model/resources/hydro/hydro_inter_period_linkage.jl"
│ ]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.cap_reserve_margin!' in src/slack_variables_overview.md:60-63
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["cap_reserve_margin.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.load_cap_reserve_margin!' in src/slack_variables_overview.md:60-63
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["cap_reserve_margin.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.load_cap_reserve_margin_trans!' in src/slack_variables_overview.md:60-63
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["cap_reserve_margin.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.load_co2_cap!' in src/slack_variables_overview.md:67-70
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["co2_cap.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.energy_share_requirement!' in src/slack_variables_overview.md:74-77
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["energy_share_requirement.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.load_energy_share_requirement!' in src/slack_variables_overview.md:74-77
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["energy_share_requirement.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.load_minimum_capacity_requirement!' in src/slack_variables_overview.md:81-84
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["minimum_capacity_requirement.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.minimum_capacity_requirement!' in src/slack_variables_overview.md:81-84
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["minimum_capacity_requirement.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.configure_solver' in src/solvers.md:2-5
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["configure_solver.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.configure_clp' in src/solvers.md:8-11
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["configure_clp.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.configure_cbc' in src/solvers.md:14-17
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["configure_cbc.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.configure_highs' in src/solvers.md:20-23
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["configure_highs.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.configure_gurobi' in src/solvers.md:26-29
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["configure_gurobi.jl"]
│ ```
│   :autodocs_block = :autodocs_block
└ @ Documenter ~/.julia/packages/Documenter/nQAq5/src/utilities/utilities.jl:46
┌ Warning: duplicate docs found for 'DOLPHYN.configure_cplex' in src/solvers.md:32-35
│ ```@autodocs
│ Modules = [DOLPHYN]
│ Pages = ["configure_cplex.jl"]
│ ```
│   :autodocs_block = :autodocs_block

Expected Behavior

Expect all functions with docstring should be included in the manual. They should not show up in the output of documentation building as "not included in the manual".

Error: "Unexpected Julia interpolation of type Symbol in the Markdown." should not be in the output.

Make sure all functions have docstring to show up in the final documentation.

Steps To Reproduce

1. In DOLPHYN folder, run julia --project=docs docs/make.jl
2. In the output at the end of this documentation building, you will see a warning about such functions.

Environment

- OS: MacBook Pro Ventura 13.6
- Julia version: 1.9
- Solver: Gurobi
- Solver version: 10.0.3

Manifest.toml file

Manifest.toml.txt

Anything else?

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Automatically test examples for new PRs

@Betristor

• Create a workflow in GitHub Actions to test the three SmallNewEngland cases using HiGHs after each PR
• Base off of genx_as_submodule
• Use Documentation.yml as a template

Select the zone and time data from input data to keep static input files while create multiple scenarios

Discussed in #89

^{Originally posted by Betristor October 26, 2022}
For now, we need to create separate examples in our branch for testing, debuging or running cases each time when we need a new scenario. But these actions could be implemented by the code automatically. Basically I listed some of the scenarios that could be controled through setting files if we add some code.

• select zones from a larger zone set to create a sub-area
• select time from time series to specify to a certain period Time selection will corrupt the time reduction function we have for now
• select generators according to tech types or names

Above three could all be implemented by adding filters when we load the data. And we could control these actions from settings. Besides, based on this, we could establish a library of generators for later standard test examples establishment.

Problem with TDR

When I use TDR to solve the model of three zone example system, the power demand of each zone decreases with each repeated runs of the same conditions despite nothing being changed. In the folder TDR_Results the load_data get replaced with a smaller value with each repeated run. The only way to get around this is to delete existing results and TDR results folder before doing a new run. Without TDR, the power balance and demand are normal and unchanged with repeated run.

Proposal of applying ModelScalingFactor in writing output part instead of in constraints and expressions

In GenX, ModelScalingFactor is used to convert power unit from MW into GW which are both commonly used units in power system. But this conversion happens too many times in expressions and constraints. In code logic it's right but in development process it's annoying. A basic unit should be fixed at input stage and we could change it into many forms once the results are given.

Replace sum with sum_expression in HSC/model

What new feature do you propose?

This request is to replace sum with sum_expression for better performance. Take a line that has sum from h2_investment.jl as an example:
@expression(EP, eTotalH2GenCFix, sum(EP[:eH2GenCFix][k] for k in H2_GEN))

We will use sum_expression from src/model/expression_manipulation.jl:

function sum_expression(expr::AbstractArray{C, dims}) :: AffExpr where {C,dims}
    # check_addable_to_expr(C,C)
    total = AffExpr(0.0)
    for i in eachindex(expr)
        add_to_expression!(total, expr[i])
    end
    return total
end

Please link any issues this feature would resolve

No response

What suggestions do you have for implementing this feature?

No response

Please add any relevant labels using the drop-down menu on the right.

• I have added all relevant labels.

PR #115 may have reintroduced old code

Hi @gn-he and @Betristor

PR #115 may have accidentally reintroduced some old code when you were merging it with main.

I was working with h2_truck_all and there are a mix of constraints using zones and routes. I think the zone-based indexing was reintroduced in this commit: "Merge branch 'main' into truck_route_index"

Could you check if the code should be removed and push a PR with the fixes?

Thanks

Hourly energy share requirement modeling

In branchH2_PTC_hourly_modeling , we have modified the energy_share_requirement function to model a constraint that is applied at the hourly level rather than at the annual level. However this implementation needs to be generalized in the following ways:

• We should update the code to allow for toggling between annual or hourly energy share requirement constraint based on user specification ("EnergyShareRequirement" flag e.g. flag could be 1 for annual and 2 for hourly matching). Based on this flag an If condition can be implemented into the energy_share_requirement function to activate the relevant constraint.
• The current implementation of the hourly energy share requirement constraint needs to be generalized to be useful for systems with multiple zones. Right now constraint hard coded for z=1
• The functions for writing outputs related to energy share requirement, in particular prices (dual of constraint) and revenues from meeting this requirement, is commented out since it is not compatabile with writing ouputs for the hourly form of the constraint. This needs to be revised and generalized to work for both annual and hourly constraints.

Update objective function documentation with CO2 and liquid fuel terms

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

The objective function section of the documentation is currently missing documentation of terms related to newly added sectors: CO2 and liquid fuels. https://macroenergy.github.io/Dolphyn.jl/dev/objective_function/

Expected Behavior

We should describe all the terms added to objective function from these two sectors in the objective function documentation.

Steps To Reproduce

No response

Environment

- OS:
- Julia version:
- Solver:
- Solver version:

Manifest.toml file

No response

Anything else?

No response

[BUG] Documentation for functions in DOLPHYN and GenX not showing properly

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Some documentation for functions with versions in both GenX and DOLPHYN are not being built properly, e.g. the solver setup documentation. This leaves blank pages in the documentation

Expected Behavior

All documentation to be built and displayed correctly

Steps To Reproduce

No response

Environment

- OS:
- Julia version:
- Solver:
- Solver version:

Manifest.toml file

No response

Anything else?

No response

separate HSC_costs by technology

Write now, HSC costs lumps everything together, including G2P.
G2P needs to be separated, because it is not hydrogen generation but electrical generation.
It would also be nicer to have costs for each sub-sector / technology. For example, separating hydrogen production from hydrogen storage.
Also, the NetworkExpansion costs still don't show up in each zone (only the Total shows up).

CO2 emissions accounting - issue with emissionsperPlant for CCS plants

In branch syn_fuel_model_addition_new_csc, The current formulation inaccurately accounts for the emissions from point sources with carbon capture. See explanation below.

Issue

Emissions_power.jl

• Line 33-37: Emissions for power plant is being defined based on parameter dfGen[!,:CO2_per_MWh][y] that is defined in load_generators_data.jl (lines 196-204) to be fuel_CO2_intensity x heat rate. If we set the same fuel type for CCGT-CCS and CCGT, then both CCGT-CCS plant will have higher CO2 emissions than CCGT, that will be reflected in the expression eEmissionsByZone that is used in the emissions cosntraints

Emissions_Hsc.jl
• Line 37-49: same issue as above for emissions_power. SMR with CCS is treated to have higher emissions than SMR once the fuel type is changed.

For emissions budget constraints in the various files
(co2_cap_power_hsc.jl, co2_cap_hsc.jl, co2_cap_power.jl), the equation is generally of the form:

Emissionsby zone <= some limit determined by other parameters where the LHS is computed as sum of respective expressions related to emissions from power, HSC and CSC MINUS emissions from DAC. However, we are not accounting for emissions captured for individual technologies on the LHS.

Proposed fix

• Add a column to generators_data and HSC_generators_data to specify CO2 capture rate for each technology.
• When computing emissions per plant in emissions_power, emissions_HSC, account for capture rate, so that the resulting expressions actually reflect the physically correct values
• To avoid inputing multiple parameters related to capture rates, eliminate parameters CCS_tonne_per_tonne in HSC_generators_data.csv and CCS_tonne_per_MWh in generators_data.csv respectively. Once we know the capture rate and CO2 generated from fuel combustion (fuelc_cO2 x time heat rate) we can compute the capture rate in the model.

Write log to text file

It would be useful to have the Julia / Gurobi log written to a text file so that it was always saved with the results to have the history. This would allow to compare performance and solver methods between runs or different types of problems.

Truck model revision - allow for truck transport on selected routes

The truck module currently needs to be revised to address a few aspects:

• eliminate a few unnecessary variables e.g. vH2TruckEnergy and vH2RetTruckEnergy that are never used in the code.
• Avoid reference to "charge" and "energy" in set definition since they do not mean anything for trucks.
• Allow for truck transport only a for pair of defined routes similar to the implementation for pipelines. This will allow for user to consider selected routes for where truck transport is eligible rather than considering the full combination of routes. We can do this by having a 0,1 matrix of size z x z where sources and sinks are for each route are marked as 1 and all the matrix cell entries are zero.
• Make a note that maximum travel delay cannot exceed number of hours for each representative period.

"Model is infeasible" using CPLEX as solver

Hello, I've run into an error where it claims there are zero solutions to the model and I'm not sure why as I am using example data (ERCOT_2021_Annual_1GW). I will post the full error below. If anyone has any suggestions (especially if you've successfully run the model using CPLEX), please reach out with suggestions. Thank you.

Full error:

[ Info: MILP solved for primal
ERROR: LoadError: Result index of attribute MathOptInterface.VariablePrimal(1) out of bounds. There are currently 0 solution(s) in the model.
Stacktrace:
[1] check_result_index_bounds
@ ~/.julia/packages/MathOptInterface/wrorD/src/attributes.jl:207 [inlined]
[2] get(model::CPLEX.Optimizer, attr::MathOptInterface.VariablePrimal, x::MathOptInterface.VariableIndex)
@ CPLEX ~/.julia/packages/CPLEX/Vd1Gd/src/MOI/MOI_wrapper.jl:2974
[3] get(b::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Bridges ~/.julia/packages/MathOptInterface/wrorD/src/Bridges/bridge_optimizer.jl:1195
[4] get(model::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/wrorD/src/Utilities/cachingoptimizer.jl:911
[5] _moi_get_result(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, ::MathOptInterface.VariablePrimal, ::Vararg{Any})
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:640
[6] get(model::JuMP.Model, attr::MathOptInterface.VariablePrimal, v::JuMP.VariableRef)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:680
[7] value(v::JuMP.VariableRef; result::Int64)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/variables.jl:1080
[8] value
@ ~/.julia/packages/JuMP/ptoff/src/variables.jl:1079 [inlined]
[9] #1563
@ ./none:0 [inlined]
[10] iterate
@ ./generator.jl:47 [inlined]
[11] _all(f::Base.var"#343#345", itr::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"}, #unused#::Colon)
@ Base ./reduce.jl:1250
[12] all
@ ./reduce.jl:1246 [inlined]
[13] Dict(kv::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"})
@ Base ./dict.jl:131
[14] fix_integers(jump_model::JuMP.Model)
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:40
[15] solve_model(EP::JuMP.Model, setup::Dict{Any, Any})
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:89
[16] top-level scope
@ ~/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:86
in expression starting at /Users/mlozano/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:86

caused by: Result index of attribute MathOptInterface.VariablePrimal(1) out of bounds. There are currently 0 solution(s) in the model.
Stacktrace:
[1] check_result_index_bounds
@ ~/.julia/packages/MathOptInterface/wrorD/src/attributes.jl:207 [inlined]
[2] get(model::CPLEX.Optimizer, attr::MathOptInterface.VariablePrimal, x::MathOptInterface.VariableIndex)
@ CPLEX ~/.julia/packages/CPLEX/Vd1Gd/src/MOI/MOI_wrapper.jl:2974
[3] get(b::MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Bridges ~/.julia/packages/MathOptInterface/wrorD/src/Bridges/bridge_optimizer.jl:1195
[4] get(model::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, attr::MathOptInterface.VariablePrimal, index::MathOptInterface.VariableIndex)
@ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/wrorD/src/Utilities/cachingoptimizer.jl:911
[5] _moi_get_result(::MathOptInterface.Utilities.CachingOptimizer{MathOptInterface.Bridges.LazyBridgeOptimizer{CPLEX.Optimizer}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}, ::MathOptInterface.VariablePrimal, ::Vararg{Any})
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:640
[6] get(model::JuMP.Model, attr::MathOptInterface.VariablePrimal, v::JuMP.VariableRef)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/optimizer_interface.jl:680
[7] value(v::JuMP.VariableRef; result::Int64)
@ JuMP ~/.julia/packages/JuMP/ptoff/src/variables.jl:1080
[8] value
@ ~/.julia/packages/JuMP/ptoff/src/variables.jl:1079 [inlined]
[9] #1563
@ ./none:0 [inlined]
[10] iterate
@ ./generator.jl:47 [inlined]
[11] Dict{JuMP.VariableRef, Float64}(kv::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"})
@ Base ./dict.jl:103
[12] dict_with_eltype
@ ./abstractdict.jl:575 [inlined]
[13] dict_with_eltype
@ ./abstractdict.jl:582 [inlined]
[14] Dict(kv::Base.Generator{Vector{JuMP.VariableRef}, DOLPHYN.var"#1563#1564"})
@ Base ./dict.jl:129
[15] fix_integers(jump_model::JuMP.Model)
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:40
[16] solve_model(EP::JuMP.Model, setup::Dict{Any, Any})
@ DOLPHYN ~/Library/CloudStorage/OneDrive-RMI/Documents/GitHub/DOLPHYN/src/solve_model.jl:89
[17] top-level scope
@ ~/Library/CloudStorage/OneDrive-RMI/Documents/45V/Dolphyn_Examples/Time_matching/DOLPHYN-LB-Annual/DOLPHYN-ERCOT/ERCOT_2021_Annual_1GW/Run.jl:86

VERSION check in write outputs

In many of the write outputs files (in the GenX side of the model), the code checks for the Julia version. Usually, this is to create a new dataframe column. If running with Julia 1.7.x (as I was), you will get an error. If I change the condition to <1.8, the model runs fine and this condition does not exist in the standalone GenX model. The condition is also verified in some other cases, when calculating a totals value. In this case, the user would not get an error even if using a Julia version of 1.7.x, and therefore the written result output could be false. See photos for code examples (in one example I changed the version to 1.8 instead of 1.7).

Non-served energy for Liquid H2 demand not modeled

LIquid H2 demand and supply balance does not include an non-served energy term even though input parameters related to non-served energy are included. Need to create a liquid version of non-served energy variables/constaints, similar to h2_non_served.jl

Gurobi mismatch within a new environment

After gurobi releashed gurobi10.0.0, the newest [email protected] points to gurobi10.0.0. Users with gurobiv9 license will encounter the problem when establishing and solving model as below:

Gurobi Error 10009: Version number is 10.0, license is for version 9.0

Either we bump our license to the newest version on gurobi website or we downgrade the Gurobi.jl package. Version 0.10.3 is the most used version in my case. After downgrading to 0.10.3, it works fine. Version 0.11.3 is OK with gurobiv9, but it seems some package update Gurobi.jl to 0.11.4 after it's installed.

Recommended that gurobi users bump their license to v10 or try to fix Gurobi.jl at 0.11.3.

TDR Scales up load if extreme periods not included

In a recent simulation result, it was realized that if Extreme Periods are not used for Time Domain Reduction, then the load is scaled up. For instance, in my simulation, I am trying to do time domain reduction on 7 years of data to obtain 20 representative weeks for a year. Previously, when I did not Use Extreme Periods, I was getting a load value (after accounting for time_weights) that represented the sum of 7 years instead of 1 year. After enabling extreme periods this issue was resolved.

Can the time domain reduction script be updated such that irrespective of whether extreme periods are used or not, the load is not scaled up?

Characterizing DOLPHYN performance

• @shakesy94 Fill in comparison table of Elec, H2, CSC, SF models with different settings
• [ ]

@dharik13 @gn-he Any other metrics to add?

GenX Policies - Capacity Reserve Margin

The capacity reserve margin does not account for hydrogen storage and electrolyzer power consumption (flexible load).

In the last update on the GenX policies for Capacity Reserve Margin, there was a term that accounted for asymmetric storage. This could be extended to account for hydrogen storage. However, this assumption needs to be validated.

Additionally, the complete power balancing equation (that includes conversion and storage of hydrogen and electrolyzer power consumption) needs to be reviewed to ensure that all flexible loads and storage resources are accounted for in the balancing equation.

[!WIP] Add syn fuels module

The aim of this issue is to track additional features related to synfuel and transportation demand modelling. These require a functional CSC module, and as such, the PR related to this issue cannot be closed until the CSC branch is merged. While there are multiple CSC branches, CSC_JW is considered the latest.

Liquid Fuels HSC Output Writing Files -- Merge and Optimization

What new feature do you propose?

This request does two things:

1. Merge HSC/write_outputs folder jl files from Liquid_Fuels branch to Liquid_Fuels_Merge branch;
2. Add optimizations codes to these same files.

Please link any issues this feature would resolve

#178

What suggestions do you have for implementing this feature?

N/A

Please add any relevant labels using the drop-down menu on the right.

• I have added all relevant labels.

Structure reformation of namespace for expressions, variables and constraints

Adopted from GenX repo, basic name space for expression, variables and constraints has some patterns like starting with e for expressions, with v for variables and with c for constraints. Balance constraints will have balance in name space and common constraints will have con basically. But now we need more energy sectors being modeled and this name space is expanding fast out of easy-going understanding area.
A more detailed name space should be designed and applied to take future and current developing energy sectors into consideration.

Generally we need to design name space for different energy sectors, energy flow nodes/joints inside each energy sector and different technical levels. This is hierarchical structure extraction from physical modeling. Besides this in code level, different macros in JuMP should firstly be distinguished for referencing them. And at the same time model name space should be combined.
In this structure, typically the name space will be like

• vPPower for power output (Power) in electricity sector (P), note that it involves multi energy nodes (generation and storage) so this level is missing.
• ePPowerBalance for generic power balance (PowerBalance) in electricity sector (P) of different zones at any time. Still, it involves multi energy nodes (generation and storage) so this level is missing. Since we know it's for electricity, we may use Balance instead of PowerBalance.
• cPPowerBalance for generic power balance (PowerBalance) in electricity sector (P) of different zones at any time given demand. Still, it involves multi energy nodes (generation and storage) so this level is missing.

And some examples using full name space will be like

• vPGenThemalCommit (maybe vPGenThemalOnline) for thermal unit commit. P for sector, Gen for energy flow nodes, Commit for technical level.
• ePPowerBalanceGenThermalCommit and ePPowerBalanceGenThermalNoCommit for balance sub terms and ePPowerBalanceGenThermalAll for adding them. The name space is long while readable and easy to understand.
• though we have many anonymous constraints, this name space is useful when constructing them. Like cPGenThermalRampingUp and cPGenThermalRampingDn for ramping constraints. Also, for unit commitment constraints, cPGenThermalUCStart and cPGenThermalUCInter.

But practice is more complicated, I think more familiar abbreviation could help reduce the length and construct them. Another problem to consider is the usage of capital form or lower form. It's not a big deal but affects reading experience.

Error in writing HSC costs

There seemed to be an error with writing the HSC cost using 3 zone example system, and it will specify that "eTotalCFixH2Charge not found". This is strange as the storage module is running well with outputs but somehow not the cost.

Error in loading h2_g2p.jl

Following error occurs when running three zone example system using DOLPHYN about H2_G2P. DimensionMismatch("dimensions must match: a has dims (Base.OneTo(3), Base.OneTo(168)), b has dims (Base.OneTo(168), Base.OneTo(3)), mismatch at 1") on line 54 of h2_g2p.jl

Keep input and output in seperate and specific folders

Discussed in #90

^{Originally posted by Betristor October 29, 2022}
With the expansion of coupled energy sectors, the inputs files and output files are getting more and messy. To keep the inputs in seperate folders like power, hsc and csc could help identify which file is used in which sector. And by subfolders seperating them, we could have identical file names for inputs and even identical column names for specific data, which will bring convenience while adding new sectors. And to keep the output files in seperate folders will have same effect in saving our time developing new sectors.

TDR not working in constant power and H2 load

I have noticed that TDR does not work well when the H2 load is a constant, and will generate an empty spreadsheet in the TDR folder. For a constant power load, TDR will not even run and produces an error of "ArgumentError: column name :Load_MW_z1 not found in the data frame".

Porting GenX output-writing functions

• Port over GenX output-writing functions, from most -> least expensive based on Yuheng's benchmarking
• Add some best practices for writing outputs to the wiki
• Add short instructions on benchmarking using @Btime macro from BenchmarkTools.jl

Pipelines data file input does not include FOM

The pipelines input file does not have a separate column for Fixed O&M (or Variable O&M). Since we currently don't have any pipelines, I added FOM as part of my CAPEX, but in the long term it might be good to have separate columns - so that we are consistent across our data inputs.

Pipeline cost not reported in HSC_costs.csv

The row "cNetworkExpansion" in the HSC_costs.csv output file comes out with missing values, even though some pipelines have been built.

Capacity reserve margin constraint documentation missing H2 sector terms

Documentation for capacity reserve margin constraint missing terms related to electricity use by other sectors (e.g. H2). Need to be updated.

Missing H2 transmission flows output file

Similar to write_transmission_flows.jl for GenX, a script to generate H2 flows across various of zones of the model across the time steps would be useful.

[BUG] Blue_H2 and Grey_H2 error iterating over empty sets

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

In write_h2_balance_zone.jl at line 42 and 43, if on generators were designated as Blue_H2 = 1 or Grey_H2 = 1, then there is an error trying to iterate over an empty set.

42. Blue_H2_Generation = sum(sum(inputs["omega"].* value.(EP[:vH2Gen])[y,:] for y in BLUE_H2))
43. Grey_H2_Generation = sum(sum(inputs["omega"].* value.(EP[:vH2Gen])[y,:] for y in GREY_H2))

Expected Behavior

There should be no attempt to iterate over these (empty) sets. To fix, change to the code below, which is using the exact logic as used later in the file for other variables.

Steps To Reproduce

No response

Environment

- OS:MIT Supercloud
- Julia version:9.12
- Solver:Gurobi
- Solver version:10.0

Manifest.toml file

Manifest.txt

Anything else?

No response

[Failed to precompile DOLPHYN] No such file or directory related to Gurobi

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

I am new to using DOLPHYN, but I am facing problem when instantiating and running the DOLPHYN. The error message is something like this:

✗ Gurobi
✗ DOLPHYN
0 dependencies successfully precompiled in 18 seconds. 308 already precompiled.

ERROR: The following 2 direct dependencies failed to precompile:

DOLPHYN [2a466132-4b02-429f-9994-5df6ddd3977f]

Failed to precompile DOLPHYN [2a466132-4b02-429f-9994-5df6ddd3977f] to "C:\Users\Gabriel\.julia\compiled\v1.9\DOLPHYN\jl_E75E.tmp".
ERROR: LoadError: SystemError: opening file "C:\Users\Gabriel\.julia\packages\Gurobi\vStLK\deps\deps.jl": No such file or directory
Stacktrace:
[1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
@ Base .\error.jl:176
[2] #systemerror#82
@ .\error.jl:175 [inlined]
[3] systemerror
@ .\error.jl:175 [inlined]
[4] open(fname::String; lock::Bool, read::Nothing, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing)
@ Base .\iostream.jl:293
[5] open
@ .\iostream.jl:275 [inlined]
[6] open(f::Base.var"#418#419"{String}, args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Base .\io.jl:393
[7] open
@ .\io.jl:392 [inlined]
[8] read
@ .\io.jl:473 [inlined]
[9] _include(mapexpr::Function, mod::Module, _path::String)
@ Base .\loading.jl:1959
[10] include(mod::Module, _path::String)
@ Base .\Base.jl:457
[11] include(x::String)
@ Gurobi C:\Users\Gabriel.julia\packages\Gurobi\vStLK\src\Gurobi.jl:7
[12] top-level scope
@ C:\Users\Gabriel.julia\packages\Gurobi\vStLK\src\Gurobi.jl:14
[13] include
@ .\Base.jl:457 [inlined]
[14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
@ Base .\loading.jl:2049
[15] top-level scope
@ stdin:3
in expression starting at C:\Users\Gabriel.julia\packages\Gurobi\vStLK\src\Gurobi.jl:7
in expression starting at stdin:3
ERROR: LoadError: Failed to precompile Gurobi [2e9cd046-0924-5485-92f1-d5272153d98b] to "C:\Users\Gabriel\.julia\compiled\v1.9\Gurobi\jl_51A.tmp".
Stacktrace:
[1] error(s::String)
@ Base .\error.jl:35
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
@ Base .\loading.jl:2300
[3] compilecache
@ .\loading.jl:2167 [inlined]
[4] _require(pkg::Base.PkgId, env::String)
@ Base .\loading.jl:1805
[5] _require_prelocked(uuidkey::Base.PkgId, env::String)
@ Base .\loading.jl:1660
[6] macro expansion
@ .\loading.jl:1648 [inlined]
[7] macro expansion
@ .\lock.jl:267 [inlined]
[8] require(into::Module, mod::Symbol)
@ Base .\loading.jl:1611
[9] include
@ .\Base.jl:457 [inlined]
[10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
@ Base .\loading.jl:2049
[11] top-level scope
@ stdin:3
in expression starting at C:\Users\Gabriel\DOLPHYN\src\DOLPHYN.jl:17
in expression starting at stdin:3

Gurobi [2e9cd046-0924-5485-92f1-d5272153d98b]

Failed to precompile Gurobi [2e9cd046-0924-5485-92f1-d5272153d98b] to "C:\Users\Gabriel\.julia\compiled\v1.9\Gurobi\jl_DB64.tmp".
ERROR: LoadError: SystemError: opening file "C:\Users\Gabriel\.julia\packages\Gurobi\vStLK\deps\deps.jl": No such file or directory
Stacktrace:
[1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
@ Base .\error.jl:176
[2] #systemerror#82
@ .\error.jl:175 [inlined]
[3] systemerror
@ .\error.jl:175 [inlined]
[4] open(fname::String; lock::Bool, read::Nothing, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing)
@ Base .\iostream.jl:293
[5] open
@ .\iostream.jl:275 [inlined]
[6] open(f::Base.var"#418#419"{String}, args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Base .\io.jl:393
[7] open
@ .\io.jl:392 [inlined]
[8] read
@ .\io.jl:473 [inlined]
[9] _include(mapexpr::Function, mod::Module, _path::String)
@ Base .\loading.jl:1959
[10] include(mod::Module, _path::String)
@ Base .\Base.jl:457
[11] include(x::String)
@ Gurobi C:\Users\Gabriel.julia\packages\Gurobi\vStLK\src\Gurobi.jl:7
[12] top-level scope
@ C:\Users\Gabriel.julia\packages\Gurobi\vStLK\src\Gurobi.jl:14
[13] include
@ .\Base.jl:457 [inlined]
[14] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
@ Base .\loading.jl:2049
[15] top-level scope
@ stdin:3
in expression starting at C:\Users\Gabriel.julia\packages\Gurobi\vStLK\src\Gurobi.jl:7
in expression starting at stdin:3

Stacktrace:
[1] pkgerror(msg::String)
@ Pkg.Types C:\Users\Gabriel\AppData\Local\Programs\Julia-1.9.3\share\julia\stdlib\v1.9\Pkg\src\Types.jl:69
[2] precompile(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; internal_call::Bool, strict::Bool, warn_loaded::Bool, already_instantiated::Bool, timing::Bool, kwargs::Base.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
@ Pkg.API C:\Users\Gabriel\AppData\Local\Programs\Julia-1.9.3\share\julia\stdlib\v1.9\Pkg\src\API.jl:1581
[3] precompile(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Pkg.API C:\Users\Gabriel\AppData\Local\Programs\Julia-1.9.3\share\julia\stdlib\v1.9\Pkg\src\API.jl:156
[4] precompile(pkgs::Vector{Pkg.Types.PackageSpec})
@ Pkg.API C:\Users\Gabriel\AppData\Local\Programs\Julia-1.9.3\share\julia\stdlib\v1.9\Pkg\src\API.jl:145
[5] precompile(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Pkg.API C:\Users\Gabriel\AppData\Local\Programs\Julia-1.9.3\share\julia\stdlib\v1.9\Pkg\src\API.jl:171
[6] precompile()
@ Pkg.API C:\Users\Gabriel\AppData\Local\Programs\Julia-1.9.3\share\julia\stdlib\v1.9\Pkg\src\API.jl:162
[7] top-level scope
@ REPL[12]:1

I believe this happened because I have not installed the Gurobi properly. However, I don't have a Gurobi license and don't have a plan to use Gurobi as my solver. Is there a way to make Gurobi not a dependent package with the DOLPHYN?

Expected Behavior

DOLPHYN could run without Gurobi as a dependent package.

Steps To Reproduce

No response

Environment

- OS: Windows 11
- Julia version: 1.9.3
- Solver:
- Solver version:

Manifest.toml file

No response

Anything else?

No response

Proposal of a standard test example for code dev

It's sometimes confusing to justify code quality in lack of a standard test example especially when new codes are introduced.
In a system where couple effect like electricity and hydrogen and carbon is the point we care about, as later energy carriers and forms come in, it will be harder to distinguish whether our code could still keep logistic and correct. New couple joints could break everything down if it's wrong.
Could we have a tested and minimum example which will be verified to be correct on a specific point on development timeline so that we could trace back when code development seems wrong. And this example could be updated as code development moves on as milestones.

package activation errors

• problems activating julenv.jl and Project.toml when setting up on new local devices: specifically on Mac M1 Max (needs Julia 1.8.5)
• process should also be streamlined so it is easily configurable on multiple devices

Proposed changes to write_HSC_outputs.jl

Propose removing line 55 if setup["H2CO2Cap"] == 1 condition because write_h2_emissions should be turned on always regardless if H2CO2Cap = 1 to output the CO2 emissions of HSC.