What is system design? The questions to ask before choosing an architecture

Most expensive software problems were decided early and noticed late. A portal that cannot survive month-end load, an integration that breaks when a vendor is down, a database nobody can afford to change: each one traces back to a question that nobody asked during design. This guide gives you those questions in a form you can use in a meeting.

What is system design?

System design is the work of deciding what the parts of a system are, how data moves between them, and which tradeoffs you accept, before the system is built. It sits between the requirements (what the business needs) and the code (how a developer writes it). A good design fits on a few pages and answers questions such as where the data lives, what happens when a part fails and who runs it.

Design matters because it is cheap to change on paper and costly to change in production. It also forces trade-offs into the open. Making a system faster, safer or more available usually makes it more expensive or more complex, and someone should choose that consciously. AWS describes its Well-Architected Framework as a way to understand the pros and cons of decisions you make while building systems, and each pillar page in Microsoft's equivalent framework links to a page of tradeoffs.

Which questions should you ask before choosing an architecture?

Ask these ten, in order, and write down the answers. Each row shows what a good answer looks like and the design choice it changes.

If you cannot answer a row, that is the finding. Go back to the people who own the process, and use a requirements brief to capture what you learn.

What is a service level objective, and how do you set one?

A service level objective, or SLO, is a target for a measured quality of the service. Google's site reliability engineering book defines a service level indicator (SLI) as a carefully defined quantitative measure such as request latency, error rate or availability. It defines an SLO as a target value or range for that indicator, and a service level agreement (SLA) as a contract with consequences if you miss the SLO.

Targets turn directly into design and cost. For example, 99.9% availability over a 30-day month allows about 43 minutes of downtime (43,200 minutes x 0.1%), while 99.99% allows about 4 minutes. The second target usually needs redundant components, automated failover and a rota of people on call, so ask what an hour of downtime costs the business before you pay for it.

Google's guidance is practical. Do not pick a target just because it matches current performance, because that can lock you into supporting a system that needs heroic effort to hit its numbers. Avoid absolutes such as "always available," keep the number of SLOs small and start loose, then tighten. A target your users would call good enough is better than a perfect one.

What should happen when a dependency fails?

Question 5 is the one most teams skip. Every system depends on others, such as a payment provider, an ERP, an email service or a cloud region, and each one will eventually be slow or down. Decide the behavior in advance, per dependency.

Illustrative example: Consider a regional distributor whose customer portal takes orders and writes them into an ERP that runs a nightly backup from 1 to 2 AM. Without a design decision, orders placed in that hour fail with an error page. With one, the portal accepts the order, stores it in a queue, tells the customer it is received, and writes it to the ERP when the ERP returns. The same questions apply to the payment provider, where the right answer may instead be to stop the order and say so plainly, because a payment cannot be queued honestly.

The pattern generalizes. For each dependency, choose between queueing the work, serving a stale copy, degrading a feature or failing visibly, and write down who gets alerted. For wider context on connecting systems safely, see our guide to build, buy or integrate.

Monolith or microservices: what should a smaller team choose?

A monolith is a single application that is built and deployed as one unit. Microservices split the system into many small services that are deployed and run separately, and communicate over a network. Martin Fowler, a software author, wrote in 2015 that successful microservice systems almost always began as a monolith, and reported that many colleagues argue against starting a new project with microservices, because early on speed of learning matters most and good service boundaries are hard to see. He adds that refactoring functionality between services is much harder than inside a monolith.

Microsoft's architecture guidance lists the costs plainly. A microservices system has more moving parts than the equivalent monolithic application, and its challenges include data consistency, versioning and the need for a mature DevOps culture. It tells teams to carefully evaluate whether they have the skills and experience to succeed with a highly distributed system.

For a company with a small engineering group, the verdict is clear. Start with one application organized into clearly separated modules, with one database and one deployment. Keep the module boundaries clean so a part can be lifted out later. Split a module into its own service only for a measured reason: it needs to scale very differently, a separate team owns it, or it must be released on a different schedule. Your answers to questions 1, 7 and 9 tell you whether any of those reasons exist. Most 20 to 200 person companies will not reach them for years.

How do the AWS and Azure frameworks help?

Both cloud providers publish review frameworks that work as checklists after you have your first design. The AWS Well-Architected Framework names six pillars: operational excellence, security, reliability, performance efficiency, cost optimization and sustainability. The Azure Well-Architected Framework names five: reliability, security, cost optimization, operational excellence and performance efficiency. Azure's page pairs each pillar with design principles and a page of tradeoffs.

The ten questions map onto these pillars. Questions 3 and 5 are reliability, 6 is security, 8 is cost, 7 is operational excellence and 1 is performance. Use the framework reviews to catch what you missed, and use the questions to start the conversation with non-engineers.

Which decisions are one-way doors?

Some choices can be reversed in a week, and others lock you in for years. Amazon's 2016 shareholder letter puts it this way: many decisions are reversible, two-way doors. The opposite is a one-way door, a decision that is costly or impossible to undo once made.

In system design, the usual one-way doors are the main database and its data model, the cloud provider's proprietary services that your code calls directly, the legal home of customer data, and any contract that locks in a vendor for years. Two-way doors include a logging tool, a front-end library or the choice of one queue product over another behind a clean interface. Move fast on the two-way doors, and give the one-way doors a written comparison and a small prototype first.

What is the verdict?

Answer the ten questions in writing before choosing technology, and keep each answer to a number or a named person. Choose the simplest architecture that meets the targets you set, which for a small team usually means one application, one database and managed cloud services. Spend your design effort on failure behavior and on the choices you cannot easily reverse. Revisit the answers when scale, regulation or the team changes.

How Kastling approaches this

System design at Kastling would begin with the business process and the people who run it, then work through questions like these before any technology is chosen. The team would connect to the systems a company already uses, agree targets and success measures, and keep a named person in control of consequential decisions such as data location and vendor lock-in. Designs would target AWS or Azure environments, with specialists brought in where needed. See Custom Solutions for how this applies to internal tools, portals and applications.