Shrink reference data to support execution on GHA runners

[FriederikeHanssen]

Edited issue description (SW)

• currently reference data required to run the mini simulated test data remains at 100+ GB
• this far exceeds disk capacity of GHA runners and because of this we cannot currently run non-stub tests for CI
• shrinking reference data will likely take several weeks and this work is to be deferred until after the 1.0.0 release

For initial discussions, please see #29 (comment)

---

Original issue description (FH) [click to show]

          I've been unable to run an end-to-end test for the CI since the required reference is too large for the GHA runners. And you're right that the 30.GB request is also too high for the default runners. Instead, this is something I run locally for testing.

The current test executed by the GHA workflow is only a stub test. This is something I planned to improve after the initial release - I suspect it will take a significant amount of time to reduce reference data size so that end-to-end tests work on the restrictive default runners

Originally posted by @scwatts in #29 (comment)

[scwatts]

Thanks for creating opening this @FriederikeHanssen! I've made some edits above to give some more context for when we return to the issue