The Starting Point
I wanted mathematical source code to look like mathematics
Writing engineering algorithms is surprisingly low level. I wanted to be able
to put something close to
(solve '(- (* x x) 42) 'x)
into normal Clojure code, solve or simplify the mathematics ahead of runtime,
and turn the result into efficient source code that runs in production.
I also wanted to explore the same expressions interactively in the REPL. No language switch, no external Mathematica process, no awkward binding to another system, and no separate "prototype mathematics" that later has to be rewritten as production code.
The result is Gauss: a Clojure computer algebra system covering symbolic algebra, equation solving, symbolic integration, summation, arbitrary-precision arithmetic, geometric algebra, plotting and other mathematical domains. It runs primarily on the JVM, integrates with Clojure directly, and includes a notebook frontend with pluggable evaluation.
It also became the project I used to test a second question: how far can I push agentic software development before the process starts to drift, rot or collapse under its own scale?
Scale of the Experiment
Large enough that one agent session cannot understand the project
I use those numbers as indicators of project scale, not as a productivity benchmark. I did not run a conventional-development control group and I did not track my own active engineering time well enough to defend claims like "10× faster."
What I can say is that the implementation throughput was high enough that reviewing every line manually stopped being realistic. That changed the engineering problem. The main question was no longer how quickly code could be produced. It was how to keep a system coherent when code could be produced faster than I could personally inspect it.
The Experiment
Could I avoid long-horizon drift without giving one agent the whole project?
Going in, I already assumed that coding agents were good at small, well-defined tasks. I was much less confident about their ability to preserve an architecture over many sessions, review their own work reliably, or remain sane as their context filled up.
So I deliberately did not try to solve the problem with one enormous context, one long-running coding conversation or a swarm that all knew everything. I went in the other direction.
The project used a hierarchical, version-controlled specification as the source of truth. Work was divided into small pieces. Agents normally started with an almost fresh context and received only the specification, task and handoff information they actually needed.
The thesis was simple: if the global architecture and specification stay stable, individual agents should not need to carry the entire project in context.
The Orchestrator
Specify → divide → implement → review → verify
I built my own agent orchestrator for the project. A normal feature did not go straight from prompt to implementation.
A specification agent described the feature. Another agent checked the result against the existing specification hierarchy: duplication, missing requirements, architectural drift and contradictions. The loop could run several times, with a hard stop so the agents could not review each other forever.
This was the important human checkpoint. Before implementation started I checked whether the feature was actually the feature I wanted, whether the architecture still made sense, and whether the scope was worth building.
A decomposition agent turned the approved specification into implementation tasks. A separate review agent checked that the tasks covered the complete specification. Complex features could be processed in sequential and parallel batches.
Each implementation agent received a narrow task, coding instructions and the relevant specification. It could modify files, use the Clojure REPL, run tests and commit code. If it encountered ambiguity it was expected to make a reasonable decision, continue, and log the decision for later human review.
A separate agent reviewed the implementation and tests for correctness, missing behavior, code smells and specification compliance. Findings could start another implementation round.
A final verification agent checked what was actually implemented against what had originally been approved. Only after that did the result come back to me for the final decision. Deployment stayed manual.
This worked better than I expected. The project is far beyond the amount of code or domain knowledge that fits into one normal agent session, yet I saw much less long-term drift than I expected. The agents did not need perfect memory of the project. They needed a stable source of truth and a task small enough to reason about.
Feedback Is Infrastructure
The faster and better the feedback, the better the agents worked
One of the clearest lessons was that agent autonomy depends heavily on the quality and speed of feedback. A task where an agent can immediately execute code, inspect the result and try again is a completely different problem from a task where feedback arrives an hour later.
- REPL first: implementation agents were told to execute Clojure expressions and explore behavior instead of assuming that generated code should work.
- Fast test feedback: I built test tooling that could run relevant subsets for Clojure and ClojureScript and return condensed failures rather than dumping thousands of successful assertions back into the model context.
- Mathematical verification: where possible the library verifies its own results — numerically checking solutions, differentiating an antiderivative back to the original expression, or running sweeps against textbook examples.
- External comparison: agents also generated golden-record suites and compared results against systems such as SymPy where that gave an independent reference.
Waiting more than an hour for the complete test suite is not a useful feedback loop for an implementation agent. Neither is feeding a model pages of everything passed. Agentic development needed supporting tooling, not just better prompts.
What Worked Surprisingly Well
Some work was better suited to agents than I expected
Giving agents access to the Clojure REPL changed the quality of implementation work. They could construct small expressions, inspect behavior, modify the approach and only then commit code. That removed a surprising amount of speculative coding.
Given reference material and explicit instructions to check correctness, agents could build large sets of textbook and external-reference examples. That is tedious work manually and exactly the sort of breadth I wanted for a CAS.
For one breaking call-signature change, the agent did not try to edit thousands of occurrences one by one. It recognized that Clojure source is data, wrote a transformation script and changed roughly 5,000 call sites. One exceptional case remained and was fixed separately. I probably would not have chosen to write that transformation tooling by hand for a one-off refactor.
Where It Broke
The useful failures were not syntax errors
Agents occasionally produced unbalanced parentheses, and large expressions sometimes made an agent fight the formatter. Those are annoying, but not very interesting. The failures that mattered were cases where the implementation looked plausible, tests passed, and the system was still conceptually wrong.
ClojureScript was one of the hardest parts of the project. JavaScript gives you floating-point numbers and BigInt, while a symbolic algebra system needs exact ratios, arbitrary precision and generic numerical behavior.
I introduced wrappers and supporting macros, but agents repeatedly ignored them and wrote code in the style that is normal for JavaScript. Some programming patterns seem deeply learned enough that telling the model "this project does it differently" does not always win.
The architectural solution had to come from me. Eventually I made the product decision to suspend ClojureScript support, move faster on the JVM, and reintroduce CLJS later. That created debt, but it was the right trade-off at that point in the project.
A recurring pattern was tests equivalent to
(is (some? result)).
They proved that something came back. They did not prove that the answer was
mathematically correct.
This was probably the most persistent quality problem in the project. I changed both implementation and review instructions to explicitly require behavioral assertions, built golden-record suites, added textbook examples and made review agents look harder for tests that merely exercised structure. It improved the situation, but it never disappeared completely.
In one solver extension the agent implemented a numerical solution where I expected a symbolic one. The code could be internally correct and still violate the actual purpose of the feature.
This is where "write working code" and "build the right system" become two separate jobs. More implementation agents do not solve an incomplete or ambiguous product decision.
When feedback was wrong, agents could become impressively productive in the wrong direction. A stale REPL once made behavior look inexplicable. Instead of questioning the feedback environment, an implementation agent descended into low-level Java and JVM details and started producing increasingly complicated fixes for a problem that was not actually there.
Agent reasoning does not rescue a broken feedback loop. It can make the detour more elaborate.
Architecture Became More Important, Not Less
High development speed lets you hit bad architecture faster
I started the project with the thesis that architecture would become more important with agentic coding. That was one of the clearest things I saw during development.
An agent can duplicate a weak abstraction very quickly. It can add another special case for ratios instead of using the generic numeric tower. It can create a slightly modified copy of an existing function instead of extracting the common behavior. It can introduce a new dependency because it did not notice that the repository already contains the required machinery.
None of this is unique to AI. Humans do exactly the same things. The difference is throughput. When implementation is cheap, architectural debt can spread through the system before one person has time to notice it.
The answer was not to review every line. I could not. It was to make architecture more explicit, maintain a source-of-truth specification, keep interfaces clear, give agents narrow contexts, and use independent review agents to look for duplication, code smells and violations.
What My Job Became
Less programmer, more architect, product owner and engineering manager
Saying "the agents did the coding" hides the more interesting part. The work did not disappear. It moved.
- Architecture: defining the numerical tower, module boundaries, interfaces and the framework in which agents could make local decisions safely.
- Product management: deciding which feature mattered next, what not to build, which compromises were acceptable and when deliberately accumulated debt had to be repaid.
- Specification approval: checking that a formally complete specification was also the thing I actually intended to build.
- Exception handling: stepping in when a problem needed a genuinely new architectural idea, cutting-edge mathematical research or a change of direction.
- Quality governance: deciding what evidence was sufficient to trust an implementation when line-by-line manual review was no longer realistic.
- Tooling: building the orchestrator, test runner and context/feedback mechanisms the agents needed to operate efficiently.
Development skill was still required. I needed enough implementation knowledge to know when something smelled wrong, and strong architecture skill became more important rather than less. But I spent much less of my time doing the mechanical implementation work myself.
When Coding Agents Hit a Real Knowledge Boundary
More attempts were not always the answer
Some CAS problems are simply hard. They require knowledge from numerical analysis, symbolic integration or fairly obscure mathematical literature. There were cases where a coding agent burned a large amount of context trying implementation after implementation and none of them were feasible.
For those problems I separated research from implementation. I used ChatGPT Pro for long-form mathematical and technical research, sometimes spending close to an hour on a single frontier problem, then fed the resulting constraints or approach back into the implementation workflow.
That became another useful boundary: when the missing ingredient is knowledge, repeatedly asking the coding agent to try another implementation is not orchestration. It is just burning tokens.
Use Agents Where Agents Are Needed
Do not spend reasoning on work a deterministic tool can do better
Agentic does not mean "make the model do everything." The more I worked on the project, the more I separated creative or judgment-heavy work from mechanical work.
- Formatting: use a formatter or hook, not an agent.
- Routine validation: use tests, linters and deterministic checks.
- Large repetitive transformations: let the agent design a script, then let the script do the mechanical work.
- Implementation strategy: use an agent.
- Bug hunting and code-smell review: use a fresh agent that did not write the code.
- Architecture, risk and product trade-offs: keep a human responsible.
What I Took Away From It
The useful lessons are about engineering systems, not prompting tricks
- Architecture matters more when implementation gets faster. A bad abstraction does not merely slow you down later. High-throughput agents can propagate it through the codebase before you notice.
- Do not solve long-term memory by putting everything into context. A stable specification plus small, nearly fresh task contexts worked surprisingly well for keeping the project coherent.
- Fast feedback is one of the main determinants of useful autonomy. REPLs, focused tests and compact failure output let agents operate for long periods. Slow or misleading feedback sent them into the weeds.
- Passing tests are not enough when the agents also wrote the tests. Review has to ask whether the tests establish the intended behavior, not just whether CI is green.
- Separate implementation from review. Fresh review agents were much more useful when explicitly asked to try hard to find bugs, code smells and missed requirements.
- Specification quality becomes a scaling mechanism. Ambiguous features, hidden non-goals and logical holes turn directly into agent churn. A well-defined feature can be decomposed. An unclear one just distributes the ambiguity.
- Agentic coding creates coordination problems before it eliminates them. Once several features or task batches move concurrently, prioritization, dependency management and integration start looking a lot like managing a team of humans.
- The human moves up the abstraction stack. Product decisions, architecture, risk acceptance and deciding what "correct enough" means did not become less important. They became the bottleneck.
What This Does Not Prove
No 10× claim, no autonomous-software fantasy
I did not run a controlled benchmark against a conventional engineering team. I do not have reliable human-hour tracking or per-task model-cost accounting. So I cannot tell you that this process was exactly 8×, 10× or 20× faster.
I also would not auto-approve architecture, auto-approve a feature specification or let this workflow auto-deploy into a production system where a wrong result could create real damage.
What the project does demonstrate is narrower and, to me, more useful: a large software system can be developed through many bounded agent contexts without relying on one giant persistent conversation, and implementation speed can remain high deep into the project if architecture, decomposition and feedback are treated as first-class engineering problems.
Where I Would Use This Approach
And where I would be much more cautious
The best tasks have clear interfaces, limited ambiguity and a fast way for the agent to find out whether it is wrong. The worse the feedback loop gets, the less autonomy I would give the agent.
- Good fit: languages and environments with REPLs, fast tests or instant scripting; well-defined features; explicit interfaces; automated validation; large amounts of implementation or refactoring work.
- Poor fit: long compile/test cycles, weak observability, requirements where correctness is subjective or difficult to measure — and high-stakes environments used blindly. Humans produce bugs; agents do too. QA, judgement and risk-appropriate mitigations still apply.
Technical Snapshot
The system behind the experiment
Clojure computer algebra system with symbolic algebra, calculus, equation solving, exact and arbitrary-precision arithmetic, geometric algebra, plotting and other mathematical modules.
JVM is the primary target. ClojureScript support was temporarily suspended during development because exact numerical semantics made it a substantially harder target, then reintroduced later.
Designed for interactive exploration from normal Clojure development, with a notebook frontend and pluggable browser, local and remote evaluation.
Specification, verification, decomposition, implementation, review and final verification run as separate roles with bounded contexts and version-controlled handoffs.
Primarily Claude/Claude Code for implementation, review, task breakdown and orchestration. ChatGPT Pro was used mainly for extended research on difficult mathematical and technical problems.
Focused test runner, golden records, textbook sweeps, independent reference comparisons and mathematical checks such as numerical solution verification and differentiating symbolic antiderivatives.
Next Step
Agentic Engineering Is Mostly an Engineering-System Problem
If your team already uses coding agents, the interesting question is probably no longer whether they can write a function. It is whether your architecture, specifications, tests, feedback loops and review process let them operate at useful autonomy without quietly increasing risk and technical debt.
That is the part I am interested in: designing the environment around the agents so they can spend their effort on the work that actually needs reasoning.