Farewell ShipIt, Hello Merge Queue!

In my article Avoid Breaking Main Branch I talked about ShipIt 🐿️, our solution to the productivity issues caused by enabling the requirement that all PullRequest have to be up-to-date before merging. We had enabled checks on PullRequest to avoid logical conflicts breaking the build on main. Shipit allowed us to avoid queuing manually to get our changes merged and instead would do it automatically.

Today, I want to explain to you how you can achieve similar results by using GitHub’s own feature, the Merge Queue.

What’s the merge queue

A merge queue helps increase velocity by automating pull request merges into a busy branch and ensuring the branch is never broken by incompatible changes. The merge queue provides the same benefits as the Require branches to be up to date before merging branch protection, but does not require a pull request author to update their pull request branch and wait for status checks to finish before trying to merge.

GitHub Managing a merge queue Documentation Reference

This is fantastic news as the reason we built the CIManager back at Intuit was to ensure everyone ran the CI pipeline with an up-to-date branch so that we could avoid breaking the default branch. This was causing a lot of pain to the team as it was never clear who broke what, and even if we didn’t want to point a finger at anyone, this information would help get the code fixed.

How does it work?

Let’s say your team is working on changes, and they want to merge into the default branch so that they can release it to your customers. The Pull Requests are A, B and C and the default branch is main. The git graph would look like something like this:

---
config:
  theme: base
  gitGraph:
    showCommitLabel: false
---
gitGraph TB:
    commit
    branch A
    checkout A
    commit
    checkout main
    branch B
    checkout B
    commit
    checkout main
    branch C
    checkout C
    commit
    checkout main
    merge A
    merge B
    merge C

There are several ways this can go, so first let’s assume that the PRs are not conflicting with each other.

Non-conflicting changes

Assuming this is the order of the changes main <- A <- B <- C, GitHub will trigger the following builds:

• main + A
• main + A + B
• main + A + B + C

If they all pass, it means that you will merge all three PRs at once in main as individual changes.

ShipIt being purely random the order is not guaranteed, but it would trigger an extra build for each change merged into main to every PR in the queue. So it would look like this (assuming the same order as above):

• main + A
• main + B
• main + C
• main + A + B
• main + A + C
• main + A + B + C

Comparing both approaches, we run 3 builds less with the Merge Queue, which means we would save a lot of money on CI costs in the long run.

Conflicting changes

Assuming we keep the same order main <- A <- B <- C and assuming that the PR A will break PR B, GitHub would trigger the following builds:

• main + A
• main + A + B (this build would fail)
• main + A + B + C
• main + A + C (B would be removed from the queue and C would be pushed forward but another build would be triggered as we wouldn’t need to build against B)

Using ShipIt, we would get the same number of builds as in the previous scenario (non-conflicting changes) except for the build where we check main + A + B + C as B was not merged.

How to configure it

I would recommend you to read the GitHub documentation but in the end, this should be the only thing you would need to do:

on:
  pull_request:
  merge_group:

However, remember to remove the requirement for branches to be up-to-date before merging, otherwise, it will block the merge queue from working. (Ideally, GitHub would do this for you, but it’s not the case at the moment)

Conclusions

You can achieve better results with the merge queue than with ShipIt, especially in the most common case when you don’t have conflicting changes. You also get access to improved features, such as greater visibility of the queue and the ability to prioritise some PRs over others.

I’m now archiving my repository for ShipIt and moving all my projects to use the merge queue instead.

Thanks

I’d like to thank my wife Marie Dziubich for helping me review the post and my colleague Nghia (Steve) Trinh for insisting I wrote another article!.