Shift Left: From Quality Assurance to Quality Engineering
By Darryl Brown
For most of our history, quality was something we checked for at the end.
We had skilled people doing that checking, and they were good at it — which is a significant part of why our production bug rate was so low. Customers almost never found bugs. That was a genuine point of pride.
However, checking for quality at the end of a process is the most expensive place to find it. We started asking whether we could move that discovery earlier. Much earlier.
In Post 2 of this series I wrote about QA being our bottleneck by design. Leadership had recognized that the constraint had to live somewhere in the process, and had made a deliberate decision that QA was the right place for it — a safety net the organization treated as a feature, not a flaw. It produced a near-zero production bug rate. But it came with a cost to throughput that we eventually decided to address differently.
The deeper question, once we’d addressed the throughput problem, was: why were work items arriving at QA with problems in the first place?
When we traced our quality issues upstream, we found that many of them started long before a developer wrote a line of code. Work items that lacked clarity, specificity, or a clear definition of success were entering our process and generating churn — features that didn’t match what customers actually needed, rework, rejected work items cycling back through the system. The problem wasn’t in QA. It was in requirements.
The further a problem travels through the process before it’s caught, the more expensive it becomes to fix. A requirements misunderstanding that reaches QA means rebuilding work already done. One that reaches production means all of that, plus the cost of a live issue. The goal of shifting left is to catch problems as early as possible — before significant work has been built on top of a flawed foundation.
This realization led to a shift in how we think about work items, and it connects directly to the “Ask why: discipline that runs throughout this series.
Before a work item describes a solution, it needs to answer a more fundamental question: what problem are we actually solving, and for whom? We moved away from work items that prescribed how something should be built toward ones that defined the problem clearly and stated the result required. That shift gave developers the freedom to figure out the best path, while ensuring everyone agreed on the destination before work began.
The difference in practice is easier to show than to describe. Here is a real work item that recently failed our Definition of Ready check:
Title: Research: Can the iPad block file uploads that are encrypted or not PDFs
Description: Steps to test:
- In the customer environment, in the Examples cabinet, add a Test Form.
- In the Media Viewer section, try to upload a locked PDF and a PDF that isn’t really a PDF (e.g. a renamed JPEG). Try them individually and try them both at the same time.
- Observe that the problem is correctly reported and the file(s) is/are not uploaded.
- In the Library, try to upload a locked PDF and a PDF that isn’t really a PDF.
- Observe that the problem is correctly reported and the file is not uploaded.
Implement blocking users from uploading encrypted/protected PDFs via the iPad, similar to what was done for web.
When our automated Definition of Ready check evaluated this work item, it flagged it as not ready and added the following comment directly on it:
“This ticket lacks the critical elements required for a Feature ticket’s Definition of Ready:
1. Context is missing: Why are we investigating blocking file uploads on the iPad? What specific problem are we solving? Who requested this research/feature?
2. Requirements need clarification: Define what constitutes an ‘encrypted’ PDF. Specify what makes a PDF ‘not real’. Describe the exact criteria for blocking file uploads.
3. Acceptance Criteria are not defined: What are the specific, measurable outcomes? How will we verify the implementation?
Consider restructuring this as a Research ticket to first explore the technical feasibility and requirements before converting it to a Feature ticket.”
The work item was revised. Here is what it looked like after passing the check:
Title: Research: Can the iPad block file uploads that are encrypted or not PDFs
Research Question: Can we block encrypted/protected PDFs and non-PDF files from being uploaded via iPad, similar to the web implementation?
Scope:
- In Scope: Testing file upload restrictions for encrypted/protected PDFs and non-PDF files in both Media Viewer and Library upload flows. Comparison with existing web implementation. Timebox: 1 day.
- Out of Scope: Implementing the actual blocking feature. File types other than PDFs and JPEGs. Platforms other than iPad.
Expected Deliverable: Written summary of findings: Can the iPad reliably detect and block encrypted/protected PDFs and renamed files? What technical approaches are available? Any platform limitations or user experience considerations. Recommendation on next steps. List of open questions or risks.
Context: Customers have reported multiple issues per week related to problematic file uploads on iPad. The web app already blocks these uploads; parity is needed on iPad. The goal is to reduce support burden and improve reliability for all customers.
The difference is not subtle. The first work item tells a developer what steps to follow without explaining why any of it matters. The second defines the problem, the scope, the expected output, and the context that makes the work meaningful. A developer picking up the second work item knows exactly what success looks like before they write a single line of code. And critically — that evaluation was done by an automated AI check, not a human reviewer. As described in Post 4, one of our quality engineers built that workflow in a single day. It runs on every work item. It either clears it as ready or comments directly with a specific explanation of what needs to be addressed — and references the creator so the right person knows to act. No human has to triage, interpret, or forward anything.
To codify this thinking across the process, we introduced two concepts: a Definition of Ready and a Definition of Done.
The Definition of Ready is the set of criteria a work item must satisfy before it can be assigned to a developer for the week. It covers clarity of the problem, the customer it affects, the acceptance criteria, and any identified risk. No work item gets assigned until it passes. The automated check is the first line of enforcement.
The Definition of Done works at the other end — a set of quality gates a solution must pass before it advances to the next stage of the process. This is where our automated code reviews live, including an automated pass that runs on every set of proposed changes before any human reviews them. We are actively adding additional AI expert reviewers — for security, performance, and best practices — as part of this same gate. The goal in either case is the same: quality criteria built into the process rather than applied at the end of it.
Together these two definitions create quality gates at the beginning and end of each work item’s journey. The work between them has guardrails it didn’t have before.
The name for this shift is Quality Engineering — as distinct from Quality Assurance. The difference is more than semantic. Assurance is reactive: it checks whether something meets a standard after the fact. Engineering is proactive: it designs the standard into the process so that fewer things fail to meet it in the first place.
Simon Sinek tells a story in Start with Why about Japanese engineers who designed quality so thoroughly into their manufacturing process that they could eliminate the need to test at the end. They knew it was right before it was finished. That story became a touchstone for our Director of Quality Engineering as he shaped this work — and it’s the right frame for what we’re building. We are not trying to catch more bugs at the end. We are trying to engineer a process where fewer bugs reach the end at all.
My Director of Quality Engineering is writing his own more detailed account of this transformation in his own blog. I’d encourage you to follow that work if this topic interests you — he is closer to it than I am and has more to say about the specifics.
This work is recent. We began in earnest in Q4 2025 and we are still measuring results. The early signal is encouraging — fewer work items bouncing back late in the process, and when they do bounce, they don’t travel as far to be fixed. But we haven’t declared victory. We’re watching the data and adjusting as we go, which is consistent with how we’ve approached everything else in this series.
That is the right note to close on.
Every post in this series has described something we tried, measured, learned from, and adjusted. Nothing arrived fully formed.
It might sound, reading these posts in order, as if we had a plan from the beginning — as if each change followed logically from the last. That’s not quite right. There are really two layers to what’s described here. The first is the foundation: the culture of safety, the habit of asking why, the willingness to try things and occasionally get them wrong. That foundation didn’t come from any single decision. It grew slowly, through trust built over time. The second layer is everything built on top of it — the pipeline changes, the automations, the sprint structure, the quality engineering work. Those experiments were only possible because the foundation was there. If you’re reading this series looking for somewhere to start, start there. The experiments will follow.
The pipeline simplification surprised us with how fast it moved. The sprint structure worked better than expected once we committed to defining it honestly. The AI adoption was harder than anticipated and easier than feared. The quality engineering work is the newest and least proven of everything described here.
What has been consistent throughout is the practice: step back, ask why, design an experiment, measure what happens, and keep going. Not because we had a master plan, but because we had a team willing to try things and a culture where trying things — and occasionally getting them wrong — was safe.
That culture didn’t come from a process or a tool. It came from the people on this team. I’ve tried to create the conditions for it. They did the rest.
If you’re checking for quality at the end, you’re already too late.
Thank you for reading this series. I hope something in it was useful, whether you’re building a development team, leading one, or just trying to figure out what the job actually is.
