I am writing this while our industry is undergoing a significant amount of change. While software engineering work will continue to be done by people who seek to understand and get better at the craft, it has never been easier to create code.
In an effort to understand the different types of capabilities of the tools that we have available to us, I came up with a new way to think about, and categorize, the capabilities of the tools.
Let me introduce you to the Agentic Maturity Model.
In software when we count things we start at zero so let's start there.
This is the basic prompt loop that we should all be familiar with. The degree of specificity or the type of prompt varies greatly from person to person, but the general structure remains the same.
We enter a prompt. The tool does some work. Did it achieve the result we were looking for? No. Well, that results in another prompt. If it did achieve the result, then we're finished.
Basically all the tools we use are capable of this. Whether it's web-based or CLI-based or IDE-based, they all achieve Level 0. Everyone from technical to non-technical understands how a Level 0 tool works.
Level 1 is a multi-chat parallel workflow. Typically this is done via git worktrees. However, we can make separate copies, completely separate copies of the code.
In Level 1 you have multiple copies of the code that you work on independently of each other. Then you merge all the results back into your main or primary working copy.
A lot of the tools support this and it's not clear the type of user who primarily takes advantage of this approach to agentic work. It may be the case where Level 1 is best suited to certain types of work or certain types of users prefer to work in a Level 1 based workflow.
We currently do not have data or we are not aware of research being done into agentic-based workflows to make a definitive answer on if it's the types of work or types of users that prefer this approach to agentic work. If this is your PhD thesis we would love to have your contributions on this document.
This is where we begin to see actual agentic work taking place. The single agent orchestration layer at Level 2 requires an understanding of how we can communicate and the types of communication that is possible.
Instead of using your primary discussion to do work, we can use the primary chat window as an orchestrator responsible for delegating tasks. The human instructs the orchestrator which then results in an agent being spun up and a separate discussion occurring between the orchestrator and the agent. The agent is the one responsible for the actual output.
The main distinction between this level and the next one is that it depends on the amount of work being done. The agent is only doing one thing, whether that is working on refactoring, or working on implementing new features, or adding tests.
There are no workflows at Level 2.
Certainly, Level 2 and 3 have a lot of similarities, but we'll get into that. We would be interested to hear your thoughts on how to separate these two categories.
In this category we see workers being built to do specific tasks, different types of workers to implement features, depending on the type of feature.
We need a different agent to review or check the output coming from the worker before it is pushed and committed, before it's reviewed by the human.
We still have the orchestration chat. And there are, or should be, a human in the loop to decide if the work is correct.
Once you realize that you can build workers, or agents, to do specific tasks, you come to realize that you can build an agent to handle committing the code. It is my recommendation, for work that matters, you do not let the commit agent have the ability to run git add or git commit without a human decision point.
So at Level 3 we have the ability to run entire workflows where we can guide the work being done at the beginning and we can review the work while it's being done and check the work before it's saved into version control. The human is still very much in the loop.
At the current final level, we seek to automate as much as possible. The tooling is able to take tasks from a ticketing system and automatically trigger a full complete workflow.
At Level 4, we do not guide the agent while work is being done. The human is responsible for writing tasks and reviewing the final output before it's merged but not before it's committed.
There is certainly more inherent risk to quality at Level 4, so we need to have plenty of work and testing going into understanding the agent capabilities to ensure quality of output is close to, or equivalent to, the lower categories of work.
- Contributing — How to contribute to the Agentic Maturity Model
- Supporters — Show your support by adding your name