Note on Rails World 2025, the Event Horizon, the Ruby Embassy via brandur.org

70 seconds test times for a large real world app. That’s better than Google, better than Apple, better than Dropbox, better than Netflix, and better than Stripe, likely by 10-100x. A test suite that fast keeps developers happy and productivity high. And all in Ruby! One of the world’s slowest programming languages.

The event horizon

Broadly, there are three stages in the long ark of a company’s CI trajectory:

1. Early on, the test suite is run only locally.
2. CI is set up. Test suite is runnable both locally or in CI.
3. Test suite gets too big, or too custom, or has too many dependencies. Test suite is runnable only in CI.

I’ve never actually been at a company where they transition from 2 → 3. I was pretty early at Stripe, but it still happened before I got there. I can’t say exactly how senior engineers rationalize the change, but I can imagine how the conversation goes. Something along the lines of, “We’re just too smart/too sophisticated/have too many special requirements to be running things on laptops now. We are web scale. We have to use the cloud.” 2

After the transition to stage 3 there’s a brief moment where things are still theoretically recoverable, like if a small team of dedicated engineers worked day and night for a few weeks they could walk things back from the brink, maybe. But generally speaking, you’re caught in the gravity well. After crossing the event horizon, there’s no going back. The overwhelming default will be to descend further into the black hole. The build continues to get more custom and requires more configuration and leverages more cloud constructs. There’s ~0 performance feedback now, so engineers don’t even notice half the time when they write slow tests, further degrading the morass.

In this developer’s opinion, it’s not only important to keep an eye out for that 2 → 3 transition and avoid it, but absolutely crucial to do so. A test suite that can still run on one machine can be shaped and sped up. Once you’re cloud only, all bets are off.

FROM:

brandur.org

Rails World 2025, the Event Horizon, the Ruby Embassy

Source