In a recent thread on Fedora’s development list, Miroslav Suchý asked about what Fedora packagers found most time-consuming, that could be simplified or better automated.

For me, personally, it’s rebuilding a stack of packages that Facebook develop - due to a lack of stable ABI (so it’s partly our own doing, yes), the entire stack has to be rebuilt each time.

There are some Fedora tooling that make this easier – rpmdev-bumpspec can automate the process of updating the version and changelog on simple spec files, and fedpkg request-side-tag lets any packager get a ‘side tag’ based off of an existing release, so packages can be built individually then merged back as a set, so there is no window in which the set of packages in, say, Rawhide, are inconsistent with each other, and there is no need to use buildroot overrides (which exposes the unreleased packages to other developers when they might still be buggy).

It’s still a tedious process though, and will get worse the more packages get added to this set. And I want the process to be reproducible so I can hand it off to someone else.

Enter bulk-rebuild. This is currently hard-coded to rebuild the set of Facebook packages I care about, but I hope to generalize it so it can be useful for others. Going to share this to the mailing list where it can hopefully drive some discussions.

From the README.md:

bump-stack.sh

This script is currently specific to bumping the FB packages that depend on Folly. It can be generalized but probably doesn’t make much sense unless the packages have matching version numbers.

src/fedora/fb-pkgs
❯ ./bulk-rebuild/bump-stack.sh 2020.12.28.00
/home/michel/src/fedora/fb-pkgs/folly
~/src/fedora/fb-pkgs/folly ~/src/fedora/fb-pkgs
Switched to branch 'master'
Current branch master is up to date.
Downloading: https://github.com/facebook/folly/archive/v2020.12.28.00/folly-2020.12.28.00.tar.gz
|   3.1 MiB Elapsed Time: -1 day, 16:00:01
Downloaded: folly-2020.12.28.00.tar.gz
Could not execute new_sources: Request is unauthorized.
Password for salimma@FEDORAPROJECT.ORG:
Uploading: folly-2020.12.28.00.tar.gz
######################################################################## 100.0%
Source upload succeeded. Don't forget to commit the sources file
Done updating folly. Make any manual adjustments now then press ENTER

[master 91ba7c3] Update to 2020.12.28.00
 3 files changed, 6 insertions(+), 3 deletions(-)
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 614 bytes | 204.00 KiB/s, done.
Total 5 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Emitting a message to the fedora-messaging message bus.
remote: * Publishing information for 1 commits
remote: Sending to redis to log activity and send commit notification emails
remote: * Publishing information for 1 commits
remote:   - to fedora-message
To ssh://pkgs.fedoraproject.org/rpms/folly
   1c2166c..91ba7c3  master -> master
# other packages being updated ...

build-stack.sh

Wraps around fedpkg request-side-tag and fedpkg chain-build. If the release requested is not f34 it would rebase from master.

If a side tag is passed as the second argument, it skips requesting and waiting for one, and assumes the side tag is ready to use. wait-repo without specifying a build would always wait for the next repo build so it doesn’t make sense to wait.

The list of packages is currently hardcoded too, but it’s easy to fix.

src/fedora/fb-pkgs
❯ ./bulk-rebuild/build-stack.sh f32
Waiting for side tag: f32-build-side-35423
Successfully waited 3:18 for a new f32-build-side-35423 repo
# repos being rebased to master
Chain building watchman-2020.12.28.00-1.fc32 + [['git+https://src.fedoraproject.org/rpms/folly.git#d77413a204d813852e15a203e9298596db4e9cb9'], ['git+https://src.fedoraproject.org/rpms/fizz.git#489e3fad9c84b2ad9c4e0ac5a1999707fd615c7d'], ['git+https://src.fedoraproject.org/rpms/wangle.git#556764f3fffc18e862de5ca59fb6df7ddd325c4a'], ['git+https://src.fedoraproject.org/rpms/fbthrift.git#f526a69f4d0c24fa28097d32b45984e9c62d83da']] for f32-build-side-35423
Created task: 58576348
Task info: https://koji.fedoraproject.org/koji/taskinfo?taskID=58576348
Watching tasks (this may be safely interrupted)...
58576348 chainbuild (f32-build-side-35423): free

At the end of execution the user is prompted to create the Bodhi update. Once these made it to updates-testing the side tag is automatically retired.

58576348 chainbuild (f32-build-side-35423) completed successfully
Done! Run this to create your update:
bodhi updates new --type enhancement --severity low --stable-karma 1 --from-tag f32-build-side-35423 --notes "Updating FB packages to ..."

Of note: the CLIs for fedpkg and koji is currently meant for human, not machine, consumptions. Invoking them from Bash scripts and trying to use the output (e.g. getting the name of that side tag) involves some brittle parsing. I plan to rewrite this in Python anyway, but it might be useful to add an output formatter to some commands where it makes sense (e.g. request-side-tag or chain-build).

This post is day 9 of my #100DaysToOffload challenge. Visit https://100daystooffload.com to get more info, or to get involved.

Have a comment on one of my posts? Start a discussion in my public inbox by sending an email to ~michel-slm/public-inbox@lists.sr.ht [mailing list etiquette]

Posts are also tooted to @michel_slm@floss.social