The obvious next step after building a nice CI pipeline around Bazel is Continuous Deployment. So no surprise that one of the frequent questions on Bazel slack is
How do I release the artifacts built by Bazel?
and the answer is really not well documented anywhere. Here's what I've learned.
Bazel is mostly unaware of version control, and that's good because coupling causes intended feature interactions. But sometimes you want the git SHA to appear in the binary so your monitoring system can tell which version is crash-looping. This is where stamping is used. Bazel keeps two files sitting in bazel-out all the time, stable-status.txt and volatile-status.txt, which are populated from local environment info like the hostname, and can be inputs to build actions.
The files are just sitting there in the output tree after any build:
You can fill in more values in this file by adding --workspace_status_command=path/to/my_script.sh to your .bazelrc and writing a script that emits values, often by calling git. Note that adding this flag to every build can mean slow git operations slowing down developers, so you might want to include this flag only on CI.
As an aside, instead of just a git SHA let me recommend https://twitter.com/jakeherringbone/status/1324871225898749953
The "stable" statuses are meant to be relatively constant over rebuilds on your machine. So your username is stable. The stable status file is part of Bazel's cache key for actions. So if your value of --embed_label changes, it will be reflected in the BUILD_EMBED_LABEL line of stable-status.txt and you'll get a cache miss for every stamped action. They will be re-run to find out the new value.
The "volatile" statuses change all the time, like the timestamp. These are not part of an action key, as that would make the cache useless.
Bazel only rebuilds an artifact if the stable stamp or one of the declared inputs changes. Otherwise you can get a cache hit, with a stale value of a volatile stamp.
Due to using a volatile stamp, we had a bug when we made Angular's release process. As a workaround, to make sure all the artifacts were versioned together, we had to do a clean build when releasing. I always felt bad for whoever was doing the push on their laptop and waiting.
This was the wrong approach, it should have used stable stamping.
Bazel has a flag --stamp. Very sadly, it is not exposed to Bazel rules in any consistent way, and so many rules have a fixed boolean attribute stamp = True|False. This inconsistency is too bad, and causes a lot of friction around correct stamping.
You should not enable --stamp on your CI builds. When any stable status value changes, you'll bust the cache and re-do a lot of work. Even if you don't use stable status values, some ruleset you depend on might.
This is also a key element of how we'll find the changed artifacts later. We don't want any stamp info in them at all, so their content hash is deterministic.
Use a custom Bazel rule to describe your release artifacts. Delivery styles vary a lot, so I haven't seen one of these that works for everyone. The custom rule can produce a manifest file of whatever info your continuous delivery system needs to know.
After a green CI build and test step, your pipeline should use bazel query to find all of the release artifacts.
We could release everything all the time, but
Here's the recipe:
Since most of the actions in the dependency graph shouldn't be stamp-aware, the last step here should still be fairly incremental.