This website uses cookies

Read our Privacy policy and Terms of use for more information.

The problem and the scale

In September 2020, Philip Müller described Shopify's core monolith as "over 2.8 million lines of Ruby code and 500,000 commits", in continuous development since at least 2006 by hundreds of developers. Eighteen months earlier, Kirsten Westeinde had written that more than a thousand developers had worked on it over a decade. This is the application that serves merchant admin, checkout, orders, shipping, billing and taxes.

The pain was not throughput. It was change. Westeinde lists three symptoms: changes in one area had unexpected effects elsewhere (her example is a tax calculation change altering shipping rate results), tests were slow and hard to write because of coupling, and a new developer had to understand orders and payments before shipping a first feature in an unrelated domain. Maple Ong's Packwerk article adds the code-level picture: classes over 2,000 lines, and boundaries that were only "an agreed upon social contract that can be easily broken".

The obvious answer in 2016 was microservices. Shopify said no, and the reasoning is the most reusable part of the story. Westeinde names the costs: a test and deploy pipeline per service, infrastructure overhead, network latency and failure on every cross-service call, and large refactors that need coordinated deploys. Ong puts it in one sentence: "Distributing code over multiple machines is a topology change, not an architectural change." If the domains are tangled, splitting them into services turns in-process tangles into network tangles.

The architecture before, and what broke

Before 2017, the code was organized like any Rails app: by software concept. Models in one folder, controllers in another, views in a third. Nothing in the layout said which code belonged to orders and which to shipping, and Ruby's open constant lookup meant any class could reach any other class.

The first fix was a reorganization. Starting in early 2017, a small team moved roughly 6,000 Ruby classes into domain folders (orders, shipping, inventory, billing and so on), with each class assigned by hand to a component. The list of components was negotiated with stakeholders across the company. The move shipped as one large pull request generated by scripts. It contained only file moves, and it still broke Git history tracking on GitHub.

The folders gave ownership but not isolation. To measure isolation, the team built Wedge: it hooked Ruby tracepoints during CI, recorded every method call, kept the calls crossing component boundaries, and scored each component. By 2020, Müller reports that Wedge took over an hour to run, produced noisy results that were hard to attribute, and the dependency graph it revealed was bad: every component depended on more than half of the others, with many cycles. The Shop class sat in the middle, because almost everything needs a shop.

So the state in 2019 was 37 components with owners and public entrypoints on paper, and a runtime analysis too slow and too noisy to block anything. That is the gap the three mechanisms below close.

Mechanism 1: domain components with a declared owner for every file

How it works. Each component is a folder that looks more and more like a Rails engine: its own models, controllers and jobs, with a public entrypoint that other components are supposed to call. Every file in the monolith has an owning team. Exceptions raised in production are triaged automatically to the owning component, so the team that can fix them sees them first.

The trade-off. Shopify chose to draw component lines from a vision of the future domain model (Müller cites Domain-Driven Design) rather than from today's code. That gives a target, but it means most components start with weak boundaries: in 2020, Müller writes that almost no component had a complete, strong boundary yet. Some other Shopify apps took the opposite route and carved out a few perfectly clean components first, leaving a larger unstructured remainder.

What it costs. One disruptive pull request, lost blame history, and months of negotiation over where 6,000 classes live. In return, chores that touch the whole codebase, such as Rails upgrades, become distributable by owner, and the exception routing works from day one, before any boundary is enforced.

Mechanism 2: a static checker with a to-do list that only shrinks

How it works. Packwerk, open-sourced in 2020, replaces Wedge's runtime tracing with static analysis of constant references. Each package declares itself in a package.yml at its root. Packwerk resolves every constant reference to the package that defines it, using the same file-location conventions as Rails' Zeitwerk autoloader (Some::Nested::Model lives in models/some/nested/model.rb). A reference from package A to package B is valid only if A declares a dependency on B and the constant is public in B, meaning it lives in B's app/public folder. Packwerk also rejects cyclic dependencies between packages, citing the Acyclic Dependencies Principle.

A package declaration in the 2020 format looks like this (an illustrative file, keys as described in the Packwerk documentation of the time):

# components/shipping/package.yml
enforce_dependencies: true
enforce_privacy: true
dependencies:
  - components/platform
  - components/orders

The ratchet is the important part. On a legacy codebase, Packwerk records the existing violations of each package in a list of deprecated references. Those are tolerated. Any new violation fails packwerk check in CI. The list becomes a per-team to-do list. Ong is explicit about the intent: "The purpose of Packwerk would be defeated if we merely added to the list of violations." The one accepted exception is when the correct dependency direction is not yet possible: recording a violation is preferred over adding a dependency in the wrong direction.

The trade-off. Static analysis of Ruby is incomplete. Packwerk ignores constants that are not autoloaded, which avoids false positives at the price of false negatives. Shopify judged that catching most references was enough to change behaviour. It also solves detection, not design: Ong notes there was no established pattern for turning a two-way dependency into a one-way one.

What it costs. The analysis runs in a few minutes on the largest codebase, against over an hour for Wedge, which is what made it possible to run on every pull request. In 2020 it covered six Rails applications, with 48 packages and 30 boundary enforcements in the core codebase, and about a third of the 37 main components. Adoption was voluntary and team by team. The Shipping team used its to-do list to extract a domain into its own service, which is the strongest evidence that the list is actionable.

Mechanism 3: break the dependency graph with inversion, cohesion and contracts

How it works. A checker can stop new edges, but someone still has to remove the old ones. Müller's priority list is concrete. First, cut dependencies on core components, starting with Shop, by moving responsibilities out of Shop into the components that need them. Second, invert dependencies where the direction is wrong, for example with publish and subscribe through ActiveSupport::Notifications, so a low-level component emits an event and a high-level one reacts, instead of the low-level component calling up. Third, type the boundary: Sorbet replaced an earlier dry-schema based Component::Schema for the input and output contracts of component entrypoints.

Shopify also changed what "good component" means. The team favours functional cohesion (code that performs a task together) over informational cohesion (code that touches the same tables). Grouping by table is the default trap of a Rails app, because models map to tables. To measure it, the team tracks change locality: how many components a typical pull request touches.

The trade-off. In-process events keep the deploy simple but hide the control flow: a reader of the emitting component no longer sees who reacts. Typed contracts slow down the first version of an entrypoint. And Müller lists a failure: pushing one consistent interface pattern across all components added rules, and sometimes led developers to reshape the problem to fit the pattern.

What it costs. People more than code. Shopify built an Architecture Guild with about 400 members and 54 documented meetups, a Developer Handbook, and embedded the componentization team inside product teams. Müller frames adoption with the Fogg behaviour model (ability, motivation, prompt): Packwerk on the pull request is the prompt, and the guild and handbook raise ability. His retrospective says he would start the forum from day one and invest in class-level design before component boundaries.

What this changes compared to the principles you already apply

Domain-Driven Design tells you to find bounded contexts. It does not tell you how to make 2.8 million lines respect them. Shopify's contribution is to separate the three things usually bundled into "microservices": the boundary (a declared package with a public folder), the enforcement (a CI check with a shrinking violation list), and the deployment unit (still one process). You can adopt the first two without the third, and pay for a network hop only where it buys something: Müller names storefront rendering and credit card vaulting as justified extractions.

It also reframes event-driven architecture. Inside Shopify's monolith, publish and subscribe is first a dependency-inversion tool, used to fix arrow direction, before it is an integration or scalability tool. And it puts cohesion next to coupling: a component with no outgoing violations but whose features always change together with another component is still the wrong cut. Change locality is a cheap metric for that.

Finally, the ratchet is an architecture technique on its own. A baseline of tolerated violations plus a hard gate on new ones turns a multi-year migration into many small, owned, measurable tasks. The same pattern works for lint rules, type coverage and deprecated APIs.

What to steal for your architecture

Adopt it when your pain is change coupling (unrelated tests breaking, onboarding across domains, every pull request touching five folders) and your deployment and runtime scale are fine. Do not adopt it as a substitute for scaling the runtime: if one domain needs a different language, a different data store or a separate failure domain for compliance, as card vaulting does, extract that domain as a service. Do not adopt it either for a young product: Westeinde's own advice is that "no architecture is often the best architecture in the early days of a system", and that the best time to re-architect is as late as possible.

For one team, skip the guild and the tooling program. Organize the code by domain folder from the start, declare dependencies between folders with the checker your stack offers (Packwerk for Rails, import-linter for Python, dependency-cruiser for TypeScript, ArchUnit for Java, internal packages for Go), and fail CI on cycles. Cost: an afternoon.

For ten teams, add ownership and the ratchet. Give every folder an owner in a CODEOWNERS file, route exceptions by owner, record today's violations as a baseline file per component, and block new ones on pull requests. Track two numbers per component each month: remaining baseline violations and change locality. Use in-process events only where an arrow points the wrong way.

For a hundred teams, do what Shopify did and treat it as a behaviour program: typed contracts on public entrypoints, a forum with a named owner, a handbook, and a short, written list of reasons that justify extracting a service. Expect years, not quarters, and measure progress by the shrinking to-do lists rather than by the number of services.

Sources