microsoft / times-excel-reader

Development has moved to https://github.com/samwebster/times-excel-reader

times-excel-reader's Issues

Position of the FI_T tag

Originally in TIMES models the FI_T tag was always positioned above the last column containing indices, i.e. right before the first column containing scalar values. However this is no longer the case and the tag may appear above any column.

Incorrect ~ActivePDef in input Excel files?

The active time period (~ActivePDef) in https://github.com/samwebster/times-ireland-model/blob/main/SysSettings.xlsx is P15, but I think it should be P22. (@samwebster @olejandro ?)

I think the tables B, M, and E correspond to the start/mid/end years shown in this worked example (page 50 of https://iea-etsap.org/docs/TIMES%20Documentation%20PART%20IV_final.pdf):

Note that the length of these tables should be equal to the length of the time period definition. P15 has length 15, and P22 has 22. The output DD files have length 22. I also tweaked the input to P22 and ran our script, and the tables matched (except for one extra row in B which I'll look into next). This bumped our progress up 0.1%:

42.0% of ground truth rows present in output (11902/28320)

Processing rules in TODO need to be updated

@Antti-L noticed that the following assumptions in TODO.md are incorrect:

• PrimaryCG: if missing, use the output commodity
• If Attribute=EFF: Change Comm-IN to ACT; Change Attribute to CEFF
• If Attribute=OUTPUT: Change Comm-IN to Comm-OUT-A; Change Attribute to CEFF

Tags missing

@Antti-L noticed that the tag list is missing the following tables: TFM_MIG and TFM_DINS_TS.
Is it the list in times_mapping?

Which data set should we use for testing?

If we could get output (DD) files for the TIM (https://github.com/MaREI-EPMG/times-ireland-model) then we would have an open-source test data set

Mappings does not include all use cases

@Antti-L noticed that e.g. the only mapping for ACT_EFF is ACT_EFF[REG,DATAYEAR,PRC,COM,TSLVL,VALUE] = ~FI_T(Region,Year,TechName,Comm-IN,TimeSlice,Attribute:CEFF). However, ACT_EFF should be supported for inputs, outputs and commodity groups in Other_indexes.

[RFC] Use black formatter?

I propose formatting the codebase using https://pypi.org/project/black/ . It's easy to set up locally to format-on-save, and I can add it to the CI to check pushed commits/PRs. From the doc, benefits of using black:

Black is the uncompromising Python code formatter. By using it, you agree to cede control over minutiae of hand-formatting. In return, Black gives you speed, determinism, and freedom from pycodestyle nagging about formatting. You will save time and mental energy for more important matters.

Blackened code looks the same regardless of the project you're reading. Formatting becomes transparent after a while and you can focus on the content instead.

Black makes code review faster by producing the smallest diffs possible.

Thoughts, @samwebster @tminka @niloyroot ?