Release management

Releases are where good engineering culture meets its stress test. The temptation is to treat release management as a paperwork problem (“who signs the doc?”). It’s an engineering problem — a design that either survives contact with production or doesn’t.

Release windows: weekday mornings, no Fridays, no holidays

Not because Fridays are technically different. Because the human system around the code isn’t. On Friday afternoon, half the team is mentally on the weekend, on-call is thinner, and the person who wrote the change won’t be around to fix what breaks at 6pm.

Weekday mornings give the team a full workday to catch and fix regressions with the whole team available. Anyone who insists on a Friday release should also be volunteering to be on call for the weekend, in writing, before the deploy button is pushed.

Sign-off chain, explicit and short

1. QA on staging — smoke suite + the specific scenarios the change touches.
2. Deploy to prod.
3. QA on prod — same smoke suite, real credentials, real payment provider (test mode if applicable), on the actual domain.
4. Announce close — one message in the release channel: what shipped, who to ping if it burns.

Skipping step 3 is the most common failure I see. Staging and prod are different environments. Env vars drift. Feature flags default differently. Third-party sandboxes behave differently from live. If you didn’t verify on prod, you didn’t verify.

“No rollback plan” is the smell, not the paperwork

A change without a defined rollback path is a change nobody has thought hard enough about. The rollback plan is not a formality — it’s the moment you find out that the migration you’re about to run is not actually reversible, or that the feature flag you thought would disable the code path doesn’t cover the background job.

For every non-trivial release, I want an answer to:

• What specifically breaks if this is wrong, and for whom? Not “what could go wrong” in the abstract — the concrete failure, the concrete affected users.
• Can we roll back the code alone, or does the data need touching too? Those are two very different rollback paths, and it’s worth knowing which one you’re in before you need it.
• If data did change, what’s the recovery? A dry-run of the reverse migration, ideally executed against a staging clone of prod.
• What alert surfaces the failure inside ten minutes rather than ten hours? If nothing catches it fast, you’re relying on customers to tell you.

If the answer to any of those is “we’ll figure it out,” the change isn’t ready. The fix isn’t more process — it’s more thinking, done before the change is deployed and not after.

Feature flags for anything you’re not 100% sure of

Not “for anything risky” — for anything you’re not 100% sure of. The flag is what lets you roll back the behavior without rolling back the code, which decouples the two decisions. A flag lets you turn the new checkout flow off at 3am from a phone; a code rollback needs a build, a deploy, and someone at a laptop.

The tradeoff is honest: every flag is technical debt with a shelf life. Delete them when you’re done proving the feature works. A flag that’s been on for 100% of users for six months is a landmine pretending to be a safety net.

Observability before launch, not after

The first version of a feature ships with the logs, metrics, and alerts you need to know whether it’s working. Not the second version. Not the version after the first outage. The first version. Otherwise the first time you hear about a problem is from a customer, and by then the graph you needed doesn’t have any data in it.