Hi everyone,
As most of you knows, Freedesktop has seen increased infrastructure
cost due to ever increasing egress traffic in GitLab CI. GStreamer was
the first major project to start using this infrastructure for testing,
followed by Mesa. GStreamer is also the biggest consumer of egress
traffic in the Freedesktop community. In order to keep in place the
benefit of having pre-commit CI to our work, we have started optimizing
our CI. This optimization work is a recurring task, but this time, we
have moved our focus from "build time" only optimization to also
consider egress size.
So far, we already started reducing the size of the artifacts used by
test jobs. The runners cache has been improved to drastically reduce
the number of time docker images are downloaded. We are still looking
for solutions to some of the egress consumer, like the cerbero pre-
built dependency cache. More news to come about this.
Today, we are about to merge another piece of improvement into
GStreamer CI. The goal is to reduce how many pipelines we run per
changes. To achieve this, the per branch pipelines has been modifier so
that they need to be triggered manually. The gst-indent job will still
run automatically, as this is handy when preparing a patchset as it's
hard to locally have the right gst-indent version that matches. We are
also marking the jobs as interruptible. This will allow gitlab to
cancel jobs that are no longer relevant. Not that this is done in a
garbage collection way, so don't expect immediate effect. It should
improve our CI performance, and reduce cost when short force push are
used on an MR branch.
As soon as you create an MR, a "detached" pipeline will be creates and
run automatically. In a normal workflow, you should not have to do
anything else then send an MR as you would have usually done. For user
branches, or if you need to run a test on gstreamer namespace, you can
click the "play" button on the manifest job, and then pending pipeline
will then run.
To make this change, we had to make important changes in our scripts.
Whenever we make changes, we take a risk of regressing. Please let me
know if you encounter any related regression, we'll try and fix these
as soon as we can.
regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Locked
