You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Adding a sublibrary to the test data means republishing all of it #4138
tools/ci/download-xs.sh fetches the HDF5 test data as one tarball. It carries
neutron, photon, thermal and WMP, unpacked about 1.16 GB.
There is no way to add a sublibrary to it except to regenerate and rehost the
whole thing. The cost of adding data is set by the size of the existing library
rather than by the size of what is being added, and it needs whoever controls
the hosting to act before any test that depends on the new data can run in CI.
Photonuclear is the case I have run into. The library is roughly 150 MB, and
there is nowhere to put it short of a 1.3 GB republish, or committing binaries
under tests/.
Proposal
Keep download-xs.sh as the entry point. The developer workflow and docs/source/devguide/tests.rst do not change. Split what sits behind it into
one artifact per sublibrary, each with its own URL and cache entry, so that
adding one is additive:
artifact
contents
size
neutron
neutron + thermal + WMP
~1.12 GB
photon
photoatomic
45 MB
(new)
e.g. photonuclear
~150 MB
Thermal and WMP stay with neutron deliberately. Splitting them off saves 3% of
that artifact, which does not justify two more of everything. Thermal
S(alpha,beta) is useless without neutron data. WMP is a fit to the pointwise
resonance cross section rather than an independent dataset, and the transport
switches between the two representations at the edges of the multipole window,
so keeping them in one artifact means they cannot drift apart.
Questions
Is this a direction the project wants, or is the single tarball deliberate?
Who owns where these artifacts live?
Would an additive first step be acceptable, publishing a new sublibrary as
its own artifact alongside the existing tarball and leaving that tarball
alone? That decouples this from any decision about the existing data.
Happy to do the script, CI and docs work if there is interest.
Related
Implement photoneutron physics #1941 asks for photoneutron physics. Whatever implementation lands, it will
need photonuclear data available in CI.
Activity