Something were most network testbeds fail fairly miserably is keeping images up to date. We have basic CI infrastructure set up to crank out images whenever we need to update them for a particular reason. However, were not in a terribly good position to crank out and deliver images that keep up with the upstreams, either on a nightly basis or based on upstream events.
At a basic level, there are two sides to the problem, the production pipeline and the consumption pipeline. However, we also must consider version history and the potential for breaking experiments.
At one end of the spectrum, is to publish an image associated with a name, say ‘the’ Debian 10 image and never touch it until a concrete problem is identified. This is the approach commonly taken in network testbed land and is fairly dumb. It creates an artificial static landing spot that does not represent reality or any semantically meaningful level of fidelity for an experiment, it’s just what an experiment happens to land on because it’s what the testbed builders have provided. So we land in the very bad place of experiments being built to support the testbed rather than the other way around, which is exactly the wrong thing. If an experiment has an explicit requirement for Debian 10.0-2 for a reason that is specific to their experiment that’s fine and they should be able to specify that. However the Debian 10 image should always reflect what Debian 10 currently is.
At the other end of the spectrum, an experimenter may materialize when the current Debian image is at version 10.0-2 and then re-materialize after 10.0-3 has been published and something breaks. Thus the image constraints in an experiment definition need to be able to capture minor versions. In the absence of a minor version, the default should likely be to nudge the experiment along over minor releases (not major). But if the experiment is actually sensitive to these minor changes (i.e. the experiment has kernel modules pegged to a specific version) then there needs to be a way to specify minor version constraints in the experiment.