All posts

The Diagram Problem Nobody Talks About

Google’s Paper Banana framework uses five specialized AI agents to generate publication-ready diagrams from text. After spending time with the architecture and benchmarks, I think it might actually close the gap between what engineers need and what diagram tools deliver.

Paper Banana AI diagram generation framework visualization

Turn technical prose into diagrams you can use.

Every engineering team has a dirty secret: the diagrams in their documentation are either six months out of date, or they were drawn by someone who left the company.

The Diagram Problem Nobody Talks About

I spend a lot of time thinking about documentation. Not the kind of thinking that happens in a conference room where someone says "we should really update the wiki" and everyone nods solemnly and nothing changes. The kind of thinking that happens at 11 PM when you're trying to explain a zero-trust architecture to a compliance auditor and the only diagram you have was made in Lucidchart by an intern in 2023.

Here's the uncomfortable truth about technical diagrams: they are simultaneously the most important and most neglected artifacts in engineering organizations. A well-drawn architecture diagram communicates in seconds what takes paragraphs to explain. A methodology figure in a research paper can be the difference between a reviewer understanding your contribution and skimming past it. And yet -- the tooling for creating these diagrams has been stuck in a deeply unsatisfying place for years.

You have three options, and none of them are good:

  1. Manual tools (Figma, Lucidchart, draw.io) -- flexible, pretty, and completely disconnected from the systems they describe. Every change requires a human to open a GUI and drag boxes around.
  2. Code-based tools (Mermaid, PlantUML, TikZ) -- reproducible and version-controllable, but aesthetically limited and syntactically punishing. TikZ in particular is a language that seems designed to remind you that you are not a graphic designer.
  3. AI image generation (DALL-E, Midjourney, Nano Banana) -- gorgeous output, but fundamentally unreliable for precision work. Try asking an image model to draw a network topology with exactly seven nodes and specific labeled connections. You'll get something that looks like a network topology the way a dream looks like your childhood home -- close enough to be unsettling, wrong enough to be useless.

For the kind of work I care about -- security architectures, data pipeline flows, infrastructure documentation, publication-quality methodology diagrams -- none of these options clear the bar. And the time cost is staggering: researchers estimate that creating a single publication-ready figure takes 4-8 hours of manual effort.arxiv Multiply that across a team producing 3-5 papers a year with 4-8 figures each, and you're looking at 100-200 hours annually spent on diagram creation alone. That's five full weeks of an engineer's year, spent dragging boxes and adjusting arrow angles.

This is the gap Google's Paper Banana framework is designed to close. And after spending time with the architecture and the benchmarks, I think it might actually do it.

What Paper Banana Actually Is

Paper Banana is an agentic AI framework -- open source, built on Google Gemini -- that automates the generation of publication-ready diagrams from text descriptions.arxivofficial Not "here's a rough sketch you can clean up." Publication-ready. The kind of output you'd put in a NeurIPS submission or a client-facing architecture document.

The system was introduced by researchers at Google and evaluated against a purpose-built benchmark called PaperBananaBench, comprising 292 test cases drawn from NeurIPS 2025 papers.arxivmarktech The source context for each test case averages 3,020 words -- roughly the length of a detailed methodology section -- and the target captions average 70.4 words.arxiv

What makes it interesting isn't the output quality alone (though we'll get to that). It's how it gets there: a five-agent pipeline that mirrors the way a skilled human illustrator actually works.

The Five-Agent Architecture

Most AI diagram tools treat the problem as a single-shot generation task: text in, image out. Paper Banana treats it as a collaborative workflow involving five specialized agents, each with a distinct role:arxivdecoder

  1. Retriever Agent -- Locates reference examples from existing literature, giving the system visual precedent for the type of diagram being requested.
  2. Planner Agent -- Converts the source context (your methodology description, architecture spec, or workflow narrative) into a detailed visual specification. This is where the heavy comprehension happens.
  3. Stylist Agent -- Enforces academic and professional visual standards. Color palettes, typography conventions, layout principles -- the things that separate a diagram that looks published from one that looks generated.
  4. Visualizer Agent -- Converts the plan and style directives into actual visual output, either as rendered images or structured code (TikZ, Mermaid, etc.).
  5. Critic Agent -- Reviews the output and provides iterative feedback, catching errors in layout, labeling, and faithfulness to the source description.

These agents operate in a two-phase pipeline:

Phase 1 is linear planning: Retriever, then Planner, then Stylist. Each agent builds on the output of the previous one, progressively enriching the specification.

Phase 2 is iterative refinement: the Visualizer generates output, the Critic reviews it, and the cycle repeats -- up to three iterations -- until the result meets quality thresholds.arxivmedium

graph LR
    subgraph "Phase 1: Linear Planning"
        A["Retriever Agent<br/>Finds reference examples"] --> B["Planner Agent<br/>Creates visual specification"]
        B --> C["Stylist Agent<br/>Enforces visual standards"]
    end

    subgraph "Phase 2: Iterative Refinement"
        D["Visualizer Agent<br/>Generates diagram output"]
        E["Critic Agent<br/>Reviews and provides feedback"]
        D -->|"Submit for review"| E
        E -->|"Revision feedback<br/>(up to 3 iterations)"| D
    end

    C -->|"Enriched specification"| D
    E -->|"Approved"| F["Publication-Ready Output"]

    style A fill:#4285F4,stroke:#333,color:#fff
    style B fill:#34A853,stroke:#333,color:#fff
    style C fill:#FBBC05,stroke:#333,color:#000
    style D fill:#EA4335,stroke:#333,color:#fff
    style E fill:#9334E6,stroke:#333,color:#fff
    style F fill:#0D652D,stroke:#333,color:#fff

Figure 1: Paper Banana's five-agent pipeline, rendered from the Mermaid diagram above.

This architecture is doing something genuinely clever. By separating understanding (Planner), aesthetics (Stylist), execution (Visualizer), and quality control (Critic), the system avoids the single point of failure that plagues one-shot generation. When a diagram comes out wrong -- misaligned labels, redundant connections, incorrect node relationships -- the Critic can identify the specific failure mode and feed targeted corrections back to the Visualizer. That's fundamentally different from regenerating the entire image and hoping for better luck.

The Numbers

Let me be direct: the benchmark results are strong.

On PaperBananaBench (292 NeurIPS 2025 test cases), Paper Banana achieved a 72.7% win rate against baseline approaches, with an additional 20.7% tie rate -- meaning it matched or beat alternatives in over 93% of evaluations.arxivppcland

The per-dimension improvements tell a more nuanced story:

MetricImprovement vs. Baseline
Faithfulness (accuracy to source)+2.8 percentage points
Conciseness (information density)+37.2 percentage points
Readability (visual clarity)+12.9 percentage points
Aesthetics (visual quality)+6.6 percentage points

Source: PaperBananaBench evaluation, arXiv:2601.23265arxiv

That conciseness number -- +37.2pp -- is the one that jumps out. It suggests the system is particularly good at not including unnecessary elements, which is arguably the hardest part of diagram design. Anyone can add more boxes and arrows. Knowing what to leave out is the skill that takes years to develop.

The technical stack underneath is worth noting: Paper Banana uses Google Gemini (gemini-2.0-flash) as its vision-language model and Nano Banana Pro (based on Gemini 3 Pro Image) for image generation.arxivnanobananapro The framework is open source -- Python 3.10+, installable via pip install paperbanana -- and includes an MCP server for integration with Claude Code and Cursor.github

Why This Matters Beyond Academia

The Paper Banana paper and benchmark focus on academic figures. That's the evaluation domain, and it makes sense -- NeurIPS papers have rigorous visual standards and well-defined quality expectations. But the architecture has implications that extend far beyond the next machine learning conference.

Consider the kinds of diagrams that consume engineering team hours every week:

Security and compliance documentation. Zero-trust architectures, data flow diagrams for SOC 2 audits, network segmentation maps. These aren't optional -- they're regulatory requirements. And they need to be accurate, current, and clear enough for a non-technical auditor to follow.

Here's a zero-trust architecture that a security team might need to document:

graph TB
    subgraph "External Zone"
        U["Users / Devices"]
        EXT["External Services"]
    end

    subgraph "Identity & Access Layer"
        IDP["Identity Provider<br/>(Okta / Azure AD)"]
        MFA["MFA Enforcement"]
        ZTNA["Zero Trust Network Access<br/>(SASE Gateway)"]
    end

    subgraph "Policy Engine"
        PE["Policy Decision Point"]
        RL["Risk & Compliance<br/>Rule Engine"]
        CTX["Context Evaluator<br/>(device, location, behavior)"]
    end

    subgraph "Internal Services"
        API["API Gateway"]
        MICRO["Microservices Mesh"]
        DB["Data Stores<br/>(encrypted at rest)"]
        VAULT["Secrets Manager<br/>(HashiCorp Vault)"]
    end

    subgraph "Monitoring & Response"
        SIEM["SIEM / XDR"]
        LOG["Immutable Audit Logs"]
        IR["Incident Response<br/>Automation"]
    end

    U --> IDP
    IDP --> MFA
    MFA --> ZTNA
    ZTNA --> PE
    EXT --> ZTNA
    PE --> RL
    PE --> CTX
    PE -->|"Allow / Deny"| API
    API --> MICRO
    MICRO --> DB
    MICRO --> VAULT
    MICRO --> LOG
    LOG --> SIEM
    SIEM --> IR

    style IDP fill:#4285F4,stroke:#333,color:#fff
    style PE fill:#EA4335,stroke:#333,color:#fff
    style SIEM fill:#FBBC05,stroke:#333,color:#000
    style DB fill:#34A853,stroke:#333,color:#fff

Figure 2: Zero-trust architecture rendered by Paper Banana from the Mermaid specification above.

That Mermaid diagram took me about fifteen minutes to write. It's functional -- you can read the flow, understand the components, follow the trust boundaries. But it's not something you'd put in a board-level security review or a compliance submission. The output is rigid, the visual hierarchy is flat, and there's no way to convey the relative importance of different components through visual weight. A Paper Banana-rendered version of this same specification could produce output with proper visual hierarchy, consistent styling, and publication-quality layout -- the kind of diagram that makes a CISO nod instead of squint.

ML and data pipeline documentation. Training pipelines, data lineage maps, feature engineering workflows. These change frequently and need to be understood by everyone from the data scientist who designed them to the SRE who's paged at 2 AM when they break.

graph LR
    subgraph "Data Ingestion"
        S3["S3 Raw Data Lake"]
        STREAM["Kafka Event Stream"]
        API_IN["REST API Ingest"]
    end

    subgraph "Processing & Feature Engineering"
        SPARK["Apache Spark<br/>ETL Pipeline"]
        FEAT["Feature Store<br/>(Feast)"]
        VAL["Data Validation<br/>(Great Expectations)"]
    end

    subgraph "Model Training"
        EXP["Experiment Tracker<br/>(MLflow)"]
        TRAIN["Distributed Training<br/>(PyTorch + Ray)"]
        EVAL["Model Evaluation<br/>& Bias Testing"]
        REG["Model Registry"]
    end

    subgraph "Serving & Monitoring"
        SERVE["Model Serving<br/>(TorchServe / Triton)"]
        AB["A/B Test Router"]
        MON["Monitoring & Drift<br/>Detection"]
        ALERT["Alerting Pipeline"]
    end

    S3 --> SPARK
    STREAM --> SPARK
    API_IN --> SPARK
    SPARK --> VAL
    VAL -->|"Pass"| FEAT
    VAL -->|"Fail"| ALERT
    FEAT --> TRAIN
    TRAIN --> EXP
    TRAIN --> EVAL
    EVAL -->|"Approved"| REG
    REG --> SERVE
    SERVE --> AB
    AB --> MON
    MON --> ALERT
    MON -->|"Retrain signal"| TRAIN

    style SPARK fill:#E65100,stroke:#333,color:#fff
    style TRAIN fill:#4285F4,stroke:#333,color:#fff
    style SERVE fill:#34A853,stroke:#333,color:#fff
    style MON fill:#FBBC05,stroke:#333,color:#000

Figure 3: ML data pipeline rendered by Paper Banana from the Mermaid specification above.

Every ML team I've worked with has some version of this diagram. It lives in a Notion page or a Confluence doc, it was last updated three sprints ago, and at least two of the services have been renamed since. The promise of a tool that can regenerate this from a current text description -- pulling from the actual architecture docs rather than someone's memory -- is not incremental. It's transformational for documentation hygiene.

The Workflow Revolution

To understand why Paper Banana matters, you need to understand what it replaces. Here's the workflow comparison that tells the story:

graph TB
    subgraph "Traditional Diagram Workflow"
        T1["Engineer writes<br/>architecture description"] --> T2["Schedules meeting with<br/>diagram author"]
        T2 --> T3["Manual creation in<br/>Figma / Lucidchart / TikZ<br/>(4-8 hours)"]
        T3 --> T4["Review cycle with<br/>stakeholders (2-3 rounds)"]
        T4 --> T5["Final diagram delivered<br/>(1-2 weeks total)"]
        T5 --> T6["Architecture changes.<br/>Repeat from step 1."]
        T6 -.->|"Often skipped.<br/>Diagram goes stale."| T1
    end

    subgraph "Paper Banana Workflow"
        P1["Engineer writes<br/>architecture description"] --> P2["Feed description to<br/>Paper Banana pipeline"]
        P2 --> P3["Five agents process<br/>and refine (minutes)"]
        P3 --> P4["Publication-ready diagram<br/>generated"]
        P4 --> P5["Quick human review<br/>and approval"]
        P5 --> P6["Architecture changes.<br/>Re-run pipeline."]
        P6 -->|"Minutes, not weeks"| P1
    end

    style T3 fill:#EA4335,stroke:#333,color:#fff
    style T6 fill:#EA4335,stroke:#333,color:#fff
    style P3 fill:#34A853,stroke:#333,color:#fff
    style P6 fill:#34A853,stroke:#333,color:#fff

Figure 4: Workflow comparison rendered by Paper Banana from the Mermaid specification above.

The red boxes are where the traditional workflow bleeds time and money. The green boxes are where Paper Banana compresses the cycle. But the really important difference isn't in the initial creation -- it's in the re-run. Traditional diagrams go stale because updating them costs almost as much as creating them from scratch. When regeneration takes minutes instead of days, documentation actually stays current. That's the compounding benefit that doesn't show up in a single-run benchmark.

What the Critic Agent Gets Right (and Wrong)

The most architecturally interesting component in the Paper Banana pipeline is the Critic Agent. Iterative self-correction is where most AI generation systems fall apart. Image models don't have a reliable way to look at their own output and identify specific failures -- they can regenerate, but they can't debug.

Paper Banana's Critic operates as a vision-language model that evaluates the generated diagram against the original specification and the Stylist's requirements. It can identify:arxiv

  • Faithfulness errors -- missing components, incorrect relationships, mislabeled nodes
  • Layout issues -- overcrowding, poor use of space, unclear flow direction
  • Style violations -- inconsistent colors, wrong typography, unprofessional appearance

The iterative loop (up to three passes) means the system can self-correct without human intervention. In the benchmark, this refinement process meaningfully improved output quality across all four evaluated dimensions.arxiv

But the Critic has known limitations, and they're worth understanding. The Paper Banana authors are transparent about several failure modes:arxivdecoder

  • Connection errors -- redundant connections between nodes, or connections that don't match the source specification
  • Spatial reasoning failures -- the Critic sometimes struggles with perception-level issues like whether two boxes overlap or whether an arrow points to the right target
  • Mismatched node counts -- occasionally, the Visualizer adds or removes nodes that the Planner specified

These are foundational model limitations, not architectural ones. As VLMs improve at spatial reasoning -- and they are improving rapidly -- these failure modes should diminish. The architecture itself is designed to accommodate better models; swapping in a more capable Gemini variant doesn't require changing the pipeline structure.

The Integration Story

Paper Banana isn't a standalone app you open in a browser. It's infrastructure -- a Python library and MCP server designed to integrate into the tools engineers already use.github

The practical integration points:

  • CLI: pip install paperbanana, then generate diagrams from the command line
  • Python API: Import it into your documentation generation scripts, CI/CD pipelines, or Jupyter notebooks
  • MCP Server: Native integration with Claude Code and Cursor, meaning you can generate diagrams without leaving your IDE
  • Version Control: Because the input is text and the pipeline is deterministic (given the same model weights), your diagram specifications can live in Git alongside the code they document

That last point matters more than it might seem. When your diagrams are generated from text descriptions that live in the same repository as your architecture code, you can put diagram updates in the same pull request as infrastructure changes. Code review now includes visual review. The diagram is the documentation, and the documentation is in version control.

Where I See This Going

I've been using Nano Banana for image generation in my own work, and it's excellent at what it does -- creative visuals, conceptual illustrations, the kind of imagery that makes a blog post or presentation feel polished. But every time I've tried to use it for a technical diagram, I've hit the same wall: image generation models don't understand precision. They understand vibes.

Paper Banana is attacking a different problem. It's not trying to make pretty pictures. It's trying to make accurate, structured, publication-quality visual communication -- and it's doing it with an architecture that separates the concerns correctly. Understanding, aesthetics, execution, and quality control are different skills. Assigning them to different agents is not just clever engineering; it mirrors how the best human design teams actually work.

The time savings alone justify attention. If you're an academic researcher producing 4-8 figures per paper, going from 4-8 hours per figure to minutes per figure is not a productivity improvement -- it's a category change. You stop thinking about diagrams as a production bottleneck and start thinking about them as a communication tool you can iterate on freely.arxiv

For engineering teams, the implications are broader. Documentation debt is real, it's expensive, and it's almost always driven by the friction of keeping visual artifacts current. If Paper Banana -- or frameworks like it -- can reduce that friction by an order of magnitude, the downstream effects on organizational knowledge, onboarding speed, and cross-team communication could be substantial.

The framework isn't perfect. The known limitations around spatial reasoning and connection accuracy mean you're not going to hand it a 50-node microservices topology and get a flawless result on the first try. But the iterative refinement loop means you might get there by the third try. And even imperfect automated output that you clean up in ten minutes beats starting from scratch in Figma.

I think the really interesting question isn't whether Paper Banana works -- the benchmarks suggest it does. The question is what happens when this kind of tool becomes standard. When every PR that changes infrastructure can auto-generate an updated architecture diagram. When security audits pull live diagrams from the codebase instead of asking someone to "send over the latest network diagram." When research papers can iterate on their figures as fluidly as they iterate on their prose.

We've been treating diagrams as static artifacts because the tools forced us to. Paper Banana treats them as generated output -- reproducible, version-controlled, and cheap to regenerate. That's a fundamentally different relationship with visual documentation, and I think it's the right one.

The framework is open source. The benchmark is public. The MCP integration means you can try it in your existing workflow today.githubofficial For anyone who's ever spent a Thursday afternoon manually updating a Lucidchart diagram that should have been auto-generated from the architecture spec it's supposed to document -- and that's all of us -- this is worth your time.

References

Written by Claude and Codex. Steered by Sid Newby.

Next postThe 90% Robot