One goal of iiasa/ixmp#182 / #249 was to clarify what core functionality of the message_ix modeling framework currently resides in the Java code of ixmp_source (i.e. behind ixmp's JDBCBackend).

As the name suggests, it is not the task of a storage backend to provide core model/framework logic; this logic should instead live in this repo. (In the current situation, new, non-Java backends would each be forced to re-implement that logic, which would be duplicative and increase points of failure.)

This issue is for:

• @danielhuppmann @gidden @zikolach myself and possibly others to identify these pieces, and
• a broader set of people to divide and coordinate (in further issues/PRs) the work of porting them from ixmp_source to message_ix.

Pieces

(@khaeru to flesh these out/keep them updated based on discussion below; others feel free to edit)

1. Scenario initialization. message_ix.Scenario(..., version='new') returns a Scenario pre-populated with certain sets and parameters.
   • Connect MACRO explicitly to MESSAGEix Scenarios #223 sets an example here: the MACRO sets and parameters are visible in the Python code. As a further iteration @gidden @khaeru feel they could be moved to a YAML file.
   • Add Model.initialize ixmp#212 adds this functionality to ixmp.
2. Mapping sets. message_ix.Scenario.cat_list(...), .cat(...) and .add_cat(...) operate by creating and manipulating related sets named type_* and map_*.
   • More intuitive treatment of type_* and cat_* mappings #21 could be addressed at the same time.
3. GDX output. The ixmp_source method Scenario.toGDX() writes Scenario data to a GDX file.
   • Depends on item 2.
4. GDX input. The ixmp_source method Scenario.readSolutionFromGDX() reads the solution (levels of specific variables and marginals of specific equations) from a GDX file. It does not read any structure (i.e., sets) or input data (i.e., parameters).

Notes

• @zikolach pointed out that some of the Java code is used by other builds from ixmp_source, e.g. behind the Scenario Explorer. So, re-implementing the functionality in Python does not always mean it can be removed from ixmp_source. On a case-by-case basis, we will need to decide this; and if the code is in both places, then there must be clear warnings to future developers that the behaviour must be kept in sync, perhaps by marking one or the other as authoritative.
• Items 3 and 4 are needed to make ixmp's GAMSModel non-backend-specific. Currently it only works with JDBCBackend.
• See also Update MESSAGEix scheme defaults in the ixmp core ixmp#70.