AI Engineering Is GANs All the Way Down

By: on Apr 23, 2026
A spiral concrete staircase with red railings viewed from above, looking down through nested loops

The Pattern Nobody Talks About

In 2014, Ian Goodfellow published a paper called "Generative Adversarial Nets" that changed everything. The core idea was almost stupidly simple: take two neural networks. One generates fake data. The other tries to tell the fakes from the real thing. Make them compete. The generator gets better at faking. The discriminator gets better at catching fakes. Both improve because of the tension between them.

Goodfellow described it as a counterfeiter versus the police. The counterfeiter learns to make better fakes. The police learn to spot subtler forgeries. Neither can improve without the other pushing them.

That paper was about generating images. But the pattern — one thing creates, another thing judges, and the loop between them is what produces quality — that pattern is everywhere now. It's the hidden architecture behind the best AI engineering happening today. And most people building with AI don't realize they're running a GAN.

The Original Insight

What made GANs revolutionary wasn't just that they generated good images. It was the discovery that adversarial feedback loops produce better results than either component alone.

Before GANs, generative models tried to learn the data distribution directly. It was hard. The loss functions were fuzzy. You couldn't easily tell if the output was good. Goodfellow's insight was: don't define "good" with a formula. Train another network to learn what "good" means, and let the two networks figure it out together.

The generator doesn't need a perfect loss function. It just needs a critic that's trying to catch it. The critic doesn't need to generate anything. It just needs to judge. Separate the creation from the evaluation, make them compete, and quality emerges from the tension.

That's not just a machine learning technique. That's a design principle.

Your Coding Agent Is Already a GAN

Think about what happens when you use Claude Code or Cursor to write code with a test suite running.

The agent writes code. That's the generator. It's producing output — functions, classes, configurations — trying to satisfy a specification it understands imperfectly.

The test suite judges that code. That's the discriminator. It doesn't generate anything. It just says pass or fail. Real or fake. Correct or wrong.

When a test fails, the agent reads the error, adjusts its approach, and generates again. The test suite evaluates again. This loop continues until the tests pass. Each round, the agent's output gets closer to correct — not because someone told it the exact answer, but because it had a critic pushing back on its attempts.

Generator. Discriminator. Adversarial feedback loop. You're running a GAN in your terminal and calling it "agentic coding."

It Gets More Interesting When You Stack the Loops

Here's something I actually do. I keep my agent prompts in my NixOS configuration — the same place I define my packages, services, and system state. The prompt is just another declarative artifact:

# In your NixOS config
environment.etc."basecamp/AGENT_INSTRUCTIONS.md".text = ''
  You are a code review agent. Evaluate the following diff for:
  - Security vulnerabilities (injection, auth bypass)
  - Performance regressions (N+1 queries, missing indexes)
  - API contract violations
  Respond with PASS or FAIL and a one-line reason.
'';

Now the prompt lives in version control, is reproducible, and renders to a file on disk. But here's where the GAN pattern kicks in. I can pull that prompt out at any time and pipe it straight to a local model:

# Render the prompt from your Nix config and pipe it to Ollama
PROMPT=$(nix eval --raw '.#nixosConfigurations.nixai.config.environment.etc."basecamp/AGENT_INSTRUCTIONS.md".text')
echo "$PROMPT\n\nHere is the diff:\n$(git diff)" | ollama run qwen3

That's one half of the loop — the generator. Qwen produces a response. Now the discriminator: I have Claude Code read Qwen's output, judge whether it actually caught the bugs I seeded into a test diff, and tell me where the prompt fell short.

If the prompt missed something? I tweak the Nix config, nix eval again, pipe to Ollama again, judge again. The whole loop takes seconds because everything is local and declarative. No deploying. No waiting. Change the source of truth, render, test, judge, repeat.

That's a GAN with different models playing each role. Claude is the discriminator — judging Qwen's responses against my criteria. Qwen is the generator — producing output from the prompt I'm iterating on. The Nix config is the thing I'm actually training. And the adversarial tension between "did the local model catch the bug?" and "the prompt wasn't specific enough" is what drives the prompt toward quality.

One creates. One judges. The loop produces quality that neither could achieve alone.

Constitutional AI: Anthropic's Version of the Same Idea

Anthropic's Constitutional AI paper (2022) took this pattern and applied it to alignment. Instead of a human labeling every harmful output, they had one AI model generate responses, then another AI model critique and revise those responses against a set of principles.

Generator: the model producing responses. Discriminator: the model judging whether those responses violate the constitution. The feedback loop: revise until the critic is satisfied.

They called it RLAIF — Reinforcement Learning from AI Feedback. But structurally? It's a GAN. One model creates, another evaluates, and the adversarial pressure between them drives improvement. The insight from 2014 keeps showing up because it keeps being right.

The Pattern in the Wild

Once you see it, you can't unsee it. The generator/discriminator dynamic is the skeleton of every AI system that actually works well:

  • Code agents + test suites: Agent generates code, tests discriminate between correct and incorrect implementations. The agent iterates until tests pass.
  • Prompt engineering with a local model: You (or your agent) generate prompts, the local model's output gets judged against criteria. Iterate until the output matches expectations. (Full walkthrough here — Claude Code judging a Qwen model through Ollama.)
  • Code review agents: One model writes the PR, another reviews it for bugs, style, or security issues. The author revises based on the reviewer's critique.
  • Self-play in game AI: AlphaGo plays against itself. Each version is simultaneously the generator (making moves) and the discriminator (exposing the opponent's weaknesses). Same loop, different domain.
  • Red-teaming LLMs: One model tries to produce harmful outputs, another tries to catch them. The adversarial pressure makes both the attack model and the safety model stronger.
  • CI/CD pipelines with AI: The agent pushes code, the pipeline (linters, type checkers, tests, security scanners) acts as a multi-headed discriminator. Fail any check, iterate.

Every one of these is the same 2014 paper wearing different clothes.

And it's getting formalized. The multi-agent patterns I wrote up from the Vibe-Ops talk at Upper Bound 2026 are this same skeleton with names attached: a "Review and Critique" loop — a generator produces, a critic evaluates, and the two iterate until the work crosses a quality bar — is a GAN in everything but the training gradients. Research like Self-Refine and Reflexion turns the same loop into a documented technique. The more agentic the tooling gets, the more these generator/critic loops stop being something you stumble into and start being deliberate architecture.

Why Feedback Loops Beat Single-Shot Generation

Here's the thing most people get wrong about working with AI: they treat it as a single-shot generator. You give the model a prompt, it gives you output, you use the output. One pass. Done.

That's like training a GAN with one gradient step. You get garbage.

The magic is in the loop. The iteration. The back-and-forth between creation and judgment. Every round of "that's not right, try again" is a training step. Every test failure that gets fixed is the discriminator pushing the generator toward better output.

Single-shot prompting: "Write me a function that parses dates."
GAN-style engineering: "Write a function that parses dates. Here are 15 test cases. Run them. Fix what fails. Repeat until they all pass."

The second approach isn't slower. It's faster, because you converge on correct instead of hoping for correct.

How to Build Your Own Adversarial Loops

You don't need a PhD in machine learning to apply this. You just need to stop thinking of AI as a generator and start thinking of it as half of a feedback loop. Here's how:

1. Always have a discriminator. Before you ask an agent to generate anything, define how you'll judge the output. Tests, acceptance criteria, a checklist, a second model reviewing the first — something that can say "not good enough" and force another round.

2. Separate the roles. The thing generating the output should not be the thing judging the output. Self-evaluation produces blind spots. Use a different model, a test suite, a linter, a human reviewer — anything that brings an independent perspective to the judgment.

3. Make the loop tight. The faster you can go from generation to judgment to revision, the more iterations you get, the better the output. This is why coding agents with inline test execution are so powerful — the loop is seconds, not hours.

4. Let the discriminator be harsh. In GAN training, a weak discriminator produces a weak generator. Same here. Strict tests, comprehensive criteria, and honest evaluation all produce better output than lenient checks that let mediocre work through.

The Bigger Picture

Goodfellow's 2014 paper wasn't just a technique for generating images. It was a discovery about how quality emerges from tension. Put two systems in opposition — one that creates, one that judges — and the adversarial pressure between them produces results neither could achieve alone.

That principle is now the backbone of AI engineering, whether people recognize it or not. The best coding agent workflows, the best prompt engineering techniques, the best AI safety approaches — they all have a generator and a discriminator locked in a productive struggle.

Next time you set up an AI workflow, ask yourself: where's my discriminator? If you don't have one, you're running half a GAN. And half a GAN is just a random generator hoping for the best.

The title is a wink at "turtles all the way down" — the old infinite-regress fable where the world rests on a turtle, which rests on another turtle, and so on forever. Terry Pratchett built a universe on the gag: his Discworld sails through space on the shell of Great A'Tuin, a literal world-bearing turtle. AI engineering has the same recursive joke at its core — a generator inside a discriminator inside a bigger loop, quality emerging from the tension at every level. Loops all the way down.

Publish over perfect. But test over hope. The feedback loop is the whole game.

Header image by Tine Ivanič on Unsplash

Content on this blog was created using human and AI-assisted workflows described here. Original ideas and editorial decisions by Justin Quaintance.