What separates a good developer from a bad one now that agents write the code

Hiring used to lean on a visible skill: can this person write correct code, quickly, from a blank file? A coding agent now writes a plausible first version of almost anything in seconds, so that test no longer separates people. The question for a CTO, an IT lead or a founder is what does.

What separates a good developer from a bad one now?

The separator is judgment about code the developer did not type. A good developer decides what to build, gives the agent enough context to build it, checks the result against reality and takes responsibility for it in production. A bad developer pastes a request, accepts the output and ships it.

This guide takes a clear position: code output has become cheap, and six things have become the differentiators. They are problem framing, system design, review, testing and debugging, security awareness, and owning production. The table shows what each looks like in practice.

None of the left column needs a particular tool. All of it needs a person who understands the system well enough to be surprised when the agent is wrong.

What does the research say about AI and developer productivity?

The evidence is mixed, and every study has limits worth knowing before you set a policy.

METR, 2025. METR ran a randomized trial with 16 experienced open-source developers working on 246 real issues in large repositories they knew well, each issue randomly assigned to allow or forbid AI tools. Developers took 19% longer with AI allowed. They had forecast a 24% speedup beforehand and believed afterward that AI had sped them up by 20%. The authors list limits: only 16 developers, a specific kind of project, tools from early 2025 (mainly Cursor Pro with Claude 3.5 and 3.7 Sonnet), and the result does not show that AI slows most developers.

METR, 2026 follow-up. In February 2026, METR reported on a larger late-2025 study with 57 developers across 143 repositories and more than 800 tasks. Its estimates pointed toward a speedup this time, but the ranges for both groups included zero, and METR called the data an unreliable signal of the current effect. Developers increasingly declined to take part because they did not want to work without AI, and METR says this likely understates any speedup. The fair reading is that the early-2025 slowdown should not be quoted as a current fact, and nobody has a clean measurement yet.

DORA, 2025. Google's DORA research surveyed nearly 5,000 technology professionals. It found that 90% use AI at work, that more than 80% believe it has increased their productivity, and that 30% report little or no trust in AI-generated code. It found a positive relationship between AI adoption and delivery throughput and product performance, and a continued negative relationship with delivery stability. Its summary is that AI amplifies what is already there: strong teams get better, and struggling teams see their problems intensify.

Stack Overflow. In its 2025 survey, about 46% of respondents distrusted the accuracy of AI tools and about 33% trusted it, and 66% named "AI solutions that are almost right, but not quite" as a top frustration. The 2026 survey reports that 48% say they trust AI when they can easily validate the answers. Trust follows verifiability, which is the habit this guide recommends.

Taken together, these sources support one practical conclusion. Agents raise output, and the quality of a team's practices decides whether that output helps.

Which skills matter most when agents write the code?

Six skills carry the weight, and each maps to a failure you can see in a post-incident review.

1. Problem framing. Turning a vague business request into a precise problem. An agent will build the wrong thing quickly and confidently if the question is wrong.
2. System design. Choosing boundaries, data models and failure behavior. Agents work well inside a design and poorly at inventing one that will survive growth.
3. Review. Reading a diff for correctness, scope and fit with the codebase. Review is now the main place where quality is added.
4. Testing and debugging. Writing checks that fail for the right reason, and tracing a defect to its cause. Debugging is also where the learning gap was largest when developers leaned on an agent in the study described later in this guide.
5. Security awareness. Knowing about access control, secrets and dependencies well enough to spot trouble. Our guide to vibe coding risks covers the five checks.
6. Owning production. Monitoring, rollback, backups and on-call. Agents do not carry the pager.

These are not new skills. What changed is the share of a developer's day spent on them, and how visible a gap becomes when output is no longer the bottleneck.

How important is asking the right question, and how do you build the skill?

It is the main skill, because the agent will answer whatever it is asked. A vague question returns a confident answer to a different problem. Anthropic's guidance for Claude Code says that if you name the file, describe the symptom and say what "fixed" looks like, you usually save yourself a round of corrections, and OpenAI's Codex guidance suggests four parts to a prompt: goal, context, constraints and what "done" means.

Question-asking is a habit, and these six practices build it.

1. State the goal and constraints before asking for code. Write what the change is for, what must not change, which libraries are allowed and who the users are. Writing it forces you to find out what you do not know.
2. Ask what could go wrong. Before accepting a design, ask for the ways it fails in production, ranked by business impact, and how you would notice each one.
3. Ask for alternatives and tradeoffs. Request two or three approaches with their costs in complexity, speed and maintenance, then choose. Agents tend to present the first approach as the only one.
4. Ask the agent to explain before you accept. If you cannot restate the change in your own words, you cannot own it. Ask for the reasoning, then ask what it assumed.
5. Write the test first. Anthropic's guidance says that Claude stops when work looks done, and that without a pass-or-fail check you become the test suite. Give it a failing test to turn green, and ask for evidence such as the command it ran and its output.
6. Read the diff. All of it. OpenAI's guide tells Codex users to have it write tests, run the checks and review the diff, and the same applies to the human.

To grow the habit on a team, review the prompts as well as the code. Ask developers to attach the request they gave the agent to the pull request, and discuss weak ones the way you would discuss weak code. OpenAI also advises that when Codex makes the same mistake twice, you ask it for a retrospective and update the repository's instruction file, which turns each mistake into a standing rule.

What do good and bad prompts look like?

A good prompt supplies what the agent cannot see. These are plain-text examples written for illustration.

Illustrative example: Consider a 40-person company that gives two contractors the same task: let customers download past invoices as PDFs. The first types the request, runs the result and submits a pull request an hour later. The second writes a short brief first, asks the agent for two designs, picks one, and adds a failing test for a customer who requests another customer's invoice. The second contractor's pull request catches an access-control hole that the first one shipped. The company never sees the difference in the demo, and sees it clearly in an audit.

How do you evaluate a developer who works with agents?

Test the thing you care about: how they decide. A work sample beats a trivia round. Give the candidate an agent and a small task with a flaw hidden in it, then watch them.

Questions that separate the two groups:

• "Tell me about a time an agent gave you code that looked right and was wrong. How did you find out?" Strong answers name a specific failure and the check that caught it.
• "Here is a 200-line diff an agent produced. What do you check first?" Strong answers start with scope, access rules and tests, not formatting.
• "What do you not let an agent do unsupervised?" Strong answers name production data, migrations, secrets and deployments.
• "A page is slow after your release and the agent proposes caching. What do you ask?" Strong answers ask what is slow and how they know before agreeing to a fix.
• "Show me a brief or prompt you wrote for a recent task." Look for constraints, a finish line and the things the agent should not touch.
• "How do you know a change is safe to deploy, and how do you undo it?" Strong answers include tests, monitoring and a rollback path.

When you hire or contract for an AI project, staffing an AI project covers the in-house, agency and partner options. For existing teams, the cost of weak review shows up later as technical debt in business systems.

What about junior developers learning with agents?

The early evidence says the way they use the agent matters more than whether they use it. Anthropic ran a randomized trial with 52 software engineers, mostly junior, learning a Python library they had not used before. The group using AI scored 50% on a follow-up quiz and the hand-coding group scored 67%, with the largest gap on debugging questions. The AI group finished about two minutes faster, which was not statistically significant.

Participants who asked conceptual questions or requested explanations alongside code kept more of what they learned than those who delegated the whole task. The authors note limits: a small sample, a chat assistant instead of an agentic tool, and a quiz taken soon after the task, so long-term effects are unknown.

For a manager, this means juniors should write the test first, explain every change in their own words, and debug with the agent as a tutor, not as a replacement.

How Kastling approaches this

Kastling would begin with the work the software supports and the people who will rely on it, then decide how much engineering review a change needs. For custom software, that means a named person who owns each consequential decision, tests agreed before development starts, and agents used inside a reviewed process, not outside it. The team would connect to the systems a business already runs and agree how success will be measured. This work sits within Custom Solutions, with AWS or Azure environments built by specialists where the application needs them.