Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some backend component attributes need to be serialised before saving to NetCDF #461

Closed
brynpickering opened this issue Apr 22, 2023 · 2 comments
Milestone

Comments

@brynpickering
Copy link
Member

Problem description

Each component in the backend dataset (e.g., the energy_cap variable dataarray) has its own attributes; a combination of useful metadata (the description of the variable) and internally useful stuff, like whether the variable has coordinate data attached to the string representation of objects in its array. One thing which may be considered as useful metadata, but was initially conceived for internal processing, is "references". This is a set of the other components in the backend in which the component in question is referenced. "cost" is referenced in the objective, "carrier_prod" is referenced in the "cost" expression, etc.

These attributes make their way out of the backend dataset and into the model results dataset by way of variables and global expressions.

So, this "references" set needs to be serialised manually for every array before the netcdf can be saved to file, OR it is dropped since it is loses its relevance once it is outside the backend dataset.

I would be more in favour of dropping it, but I don't know if others can see value in keeping it in the results. When loading a model with results from file, it would allow you to see e.g., which constraints carrier_prod was used in, even though you would no longer have access to the constraint objects (until you run model.build()).

Calliope version

0.7.0-dev.

@brynpickering brynpickering added this to the 0.7.0.b1 milestone Apr 23, 2023
@brynpickering
Copy link
Member Author

The particular case of references would be solved by it being a list rather than a set. However, the issue still exists that we do not look into individual array attrs when serialising data types that cannot be handled by NetCDF.

@brynpickering
Copy link
Member Author

Fixed in #489

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant