Shuichiro Ogawa
日本語

Notes · updated 2026-06-30

The Genealogy of Generative AI Engineering and Its Assessment from a Design Perspective

In the four years since commercial LLM use began in 2022, the vocabulary describing engineering practices that leverage generative AI has rapidly differentiated. Prompt engineering, context engineering, harness engineering, and loop engineering each denote a different design object and a different scope of responsibility. This note defines each concept from primary sources and assesses the areas where design should engage along three axes.

For individual UI-generation agent tools, see vibe-coding-design-production. For the industry-academia contrast on AX (Agentic Experience), see agentic-experience-design-synthesis. For trends in design agent tools, see design-agent-tools-landscape-2026; for the integration of MCP and design systems, see mcp-design-agent-integration.

Genealogy: From Prompts to Loops

The four stages are nested. Later stages do not completely replace earlier ones; rather, the scope of the design object expands stepwise. Prompt engineering targets the design of instruction text, context engineering the design of the information environment across the entire context window, harness engineering the design of the whole execution infrastructure built around the model, and loop engineering the design of the autonomous execution loops within that infrastructure.

Prompt Engineering: Designing Instruction Text (2022 Onward)

This is the technique of optimizing the input text (prompts) given to an LLM. Methods for guiding reasoning, such as Chain-of-Thought (Wei et al. 2022) and Tree-of-Thought (Yao et al. 2023), were systematized between 2022 and 2024.

From 2025 onward, reasoning guidance became built into the models themselves. With GPT-5’s reasoning tokens, Claude Opus’s extended thinking, and DeepSeek R1’s native CoT, the focus of design shifted from “teaching how to reason” to “designing when to use reasoning tokens”1.

The term “prompt engineering” is not dead. OpenAI still maintains the term in its official documentation as of 20262. In practice, however, the center of gravity has shifted from one-off optimization of instruction text to designing systems that dynamically assemble context, and it is the next term, “context engineering,” that gave a name to this destination.

Context Engineering: Designing the Information Environment (2025)

Lutke (Shopify CEO) proposed the term in June 20253.

“I really like the term ‘context engineering’ over prompt engineering. It describes the core skill better: the art of providing all the context for the task to be plausibly solvable by the LLM.”

Karpathy immediately endorsed the term4.

“context engineering is the delicate art and science of filling the context window with just the right information for the next step.”

Anthropic formalized a technical definition in September 20255. It is “the set of strategies for curating and maintaining the optimal set of tokens for each inference call,” with components including system prompts, user messages, tool definitions and results, conversation history, retrieved knowledge (RAG), and structured output guidelines.

Chase (LangChain CEO) summarized the concept succinctly6. “In most cases, when agents fail to perform reliably, the cause is that the appropriate context, instructions, and tools have not been passed to the model.”

Anthropic further introduced the concept of context rot. Context decays over time, and stale information distorts the agent’s judgment. Freshness management is a problem of information architecture, making it an area where design involvement is structurally required.

Academic systematization has also progressed. Mei et al. (2025), in a 166-page survey paper, classified the foundational components of context retrieval / generation, processing, and management7.

Willison, from a position of defending prompt engineering, acknowledged that context engineering is the term closer to actual practice8. Gartner declared that “context engineering is in, prompt engineering is out,” predicting it will appear in 80% of AI tools by 2028.

Harness Engineering: Designing the Execution Infrastructure (Late 2025 to 2026)

This is a concept that Anthropic systematized in November 20259 and for which OpenAI published its own version in February 202610. It takes “everything built around the model” as the object of engineering, forming a broader framework that subsumes “context engineering.”

Trivedy (LangChain) formalized the concept in March 202611. Agent = Model + Harness. There is a case in which changing only the harness improved a coding agent from Top 30 to Top 5. The harness is organized into five layers. These are tool orchestration, verification loops, context and memory, guardrails, and observability.

Anthropic’s design principle is distilled into the following sentence12.

“Every component of an agentic harness encodes an assumption about what the model can’t do on its own.”

Each component of the harness embodies an assumption about “what the model cannot do on its own.” As model capabilities improve, those assumptions become outdated and the corresponding components can be removed13.

Osmani stated this structure succinctly14. “A decent model combined with an excellent harness beats an excellent model combined with a poor harness.”

OpenAI presented a case in which roughly one million lines of code were built in five months using Codex agents alone (roughly 1,500 PRs, zero hand-written lines of code)10. It held that AGENTS.md should be treated as “a table of contents, not an encyclopedia.” This is because a massive instruction file crowds out the context available for tasks and code.

Claude Code is itself an implementation example of “harness engineering.” An official article in June 2026 systematized seven instruction delivery methods15. These are CLAUDE.md (project conventions), Rules (constraints, path-scope support), Skills (procedural workflows, saving tokens through progressive disclosure), Hooks (deterministic control, running linters, blocking destructive commands), Subagents (task isolation in separate contexts), Settings (tool permissions and prohibitions), and MCP servers (external tool integration).

The costs are high. By Anthropic’s measurements, multi-agent configurations consume 15 times the tokens, and harness orchestration raises total cost 20-fold16.

Loop Engineering: Designing Autonomous Execution Loops (June 2026)

This is the newest of the four stages. Osmani gave the term its name in the blog post “Loop Engineering” on June 7, 202617. The next day, an X post by Steinberger (OpenClaw / OpenAI) recorded 6.5 million views, and the concept spread rapidly18.

“You shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents.”

Cherny (Anthropic, head of Claude Code) corroborated this in a talk on June 21, 202619.

“I don’t prompt Claude anymore… My job is to write loops.”

Loop engineering refers to the practice of designing not the prompts written directly to an agent but the loops that prompt the agent (the cycle of execution, evaluation, and re-execution) themselves.

Its relationship to preceding concepts is as follows. Productized loops — from ReAct-style reasoning loops (2022) through AutoGPT (2023) and bash one-liner loops (2025) to Claude Code’s /loop and /goal commands — became the technical foundation. “Loop engineering” is not a superordinate concept to “harness engineering” but a concept focused on the “execution loop design” within the harness.

Criticism spread at the same time. Warnings against “loopmaxxing” (the misconception that looping indefinitely will solve everything) have come from multiple sources20. Cost inflation, mode collapse (homogenization of outputs), and reduced observability have been flagged as risks.

Cross-Cutting Practices

Vibe Coding and Agentic Engineering

“Vibe coding,” which Karpathy tossed out in February 2025, refers to a production style that accepts AI suggestions without review21. “Vibe engineering,” proposed by Willison in October 2025, is the practice of accelerating with AI while maintaining quality and reliability, and stands as the antonym of vibe coding22. Agentic engineering, defined by Karpathy in April 2026, made explicit the structure in which responsibility for taste, engineering, and design lies with humans, and agents are fallible entities23. For details, see vibe-coding-design-production.

Eval Engineering: Designing Evaluation Pipelines

This is the practice of designing and operating evaluation pipelines for LLM applications (especially agents). It has no single coiner, but the role of “AI eval engineer” has been taking hold from 2025 into 202624. Husain and Shankar have trained more than 2,000 people (including teams at OpenAI and Anthropic), and a development methodology called Evaluation-Driven Development (EDD) also emerged in 2026.

Elman of NNg argued that designers’ core skill is shifting to “defining rigorous evaluation criteria”25. He proposed the Judge-Evaluate-Iterate loop, redefining design critique for the AI era in three steps: (1) defining judgment criteria grounded in user research, (2) applying them to actual outputs (scaled with LLM-as-a-judge; see the LLM-as-a-Judge literature map), and (3) iterating based on the results. This is where “eval engineering” and design intersect.

Comparison Table

StageCoined by (Year)Design ObjectBottleneckDesign Involvement
Prompt engineeringMultiple (2022 onward)Instruction textHow to phrase it so it worksLimited
Context engineeringLutke (2025-06)Entire information environmentWhat to showStructural (contextualizing design systems, intent modeling, context rot)
Harness engineeringAnthropic (2025-11) / OpenAI (2026-02)Entire execution infrastructureWhat to surround it withNew territory (monitoring UX, delegation interfaces, design under cost constraints)
Loop engineeringOsmani (2026-06-07)Autonomous execution loopsHow to run the loopDesigning evaluation loops, curbing loopmaxxing
Cross-Cutting PracticeCoined by (Year)PositioningRelation to Design
Vibe codingKarpathy (2025-02)Production without responsibilityAbsence of quality judgment
Agentic engineeringKarpathy (2026-04)The profession of piloting agentsTaste and design named explicitly
Eval engineeringMultiple (2025 onward)Designing evaluation pipelinesDefining evaluation criteria as design’s core

Assessment from Design: Three Axes

As of mid-2026, three structural axes are visible in the area where design and AI engineering intersect.

Axis 1: Context Architecture (Passing Design Knowledge to Agents)

Perez of NNg proposed context architecture in June 202626. It is a concept that applies the principles of information architecture (IA) to AI systems, positioned above both “prompt engineering” and “context engineering.” It defines seven context layers (system instructions, RAG, skills, tools / MCP, long-term memory, conversation memory, and user prompts), governed by four IA principles (structuring, discoverability, mental-model alignment, and memory design). The stance is “structure over volume,” holding that naming, relationships, and constraints are not neutral choices.

A concrete implementation of this concept is DESIGN.md. It is a specification that Google Labs published in April 202627, combining YAML front matter (machine-readable design tokens) with Markdown (human-readable design intent). It is a mechanism by which Claude Code, Cursor, Copilot, and others consume design decisions as context, and as of v0.3.0 (2026-06-15) it had earned more than 11,000 GitHub stars.

Atlassian tested this specification and reported that on-demand delivery via an MCP server, compared with loading the full “DESIGN.md,” improved accuracy by 4.9%, reduced errors by 11%, increased speed by 34%, and saved 16% of tokens28. When a design system is large, dynamic delivery is more effective than a static file.

Figma announced MCP integration at Config 2026, enabling third-party agents to read and write the tokens, components, and variables of Figma files as structured data29. The direction is to supply the structure of the design system itself as context, rather than reasoning over screenshots.

The maturity of the design system sets the ceiling on agent output quality (corroborated by academic literature and five vendors). This finding is consistent with the Vercel experiment organized in mcp-design-agent-integration (reducing 15 tools to 1 raised the success rate from 80% to 100% with a 3.5x speedup).

Axis 2: Evaluation as Design (The Shift to Defining Evaluation Criteria)

Maeda discussed the transition “from UX to AX (Agentic Experience)” at SXSW in March 202630. It is a shift from Norman’s gulf of execution to the gulf of evaluation: in a world where agents perform tasks on our behalf, the design problem changes from “how to assist execution” to “how to enable judgment of whether execution results are good or bad.”

The taste that Karpathy named in “agentic engineering” is closer to judgment in system design than to design’s aesthetic sensibility31. A concrete example from Sequoia AI Ascent 2026 shows this. An agent writes code that links a Stripe purchase to a Google account by email. It works technically, but taste, he held, is the ability to judge that a persistent user ID should be used instead.

Zhu et al. (2026) released the TASTE dataset, building a benchmark in which professional designers evaluate AI-generated graphics along nine dimensions (typography, aesthetic sensibility, spatial design, tone, etc.)32. Current VLM judges fall short of designers’ judgment, but the study demonstrated that small fine-tuned models can narrow the gap.

The Judge-Evaluate-Iterate loop proposed by NNg’s Elman shows a way to connect design critique to the engineering process25. The act of defining “what to measure” in “eval engineering” is becoming a core function of the design profession.

Axis 3: Tensions at Professional Boundaries

Gibbons of NNg classified “the four design jobs AI created” in June 202633. They are (1) design that uses AI as a tool, (2) interface design for AI products, (3) design of data structures and instructions that agents can interpret (close to IA), and (4) definition of model behavior, evaluation criteria, and principles (in collaboration with engineering). (3) and (4) sit at the boundary between design and engineering.

Smashing Magazine reported in April 2026 that “production-ready is becoming the design deliverable”34. Businesses increasingly prioritize output speed over experience quality, and the pressure to integrate design and engineering is growing.

NNg’s “State of UX 2026” counters this35. It held that what AI is replacing is “the parts that were never design to begin with,” and that curated taste, research-backed contextual understanding, and critical thinking are difficult to automate.

Figma announced Code Layers at Config 2026, demonstrating a feature that clones a repository and converts flows into editable design layers29. The direction is to technically fuse the boundary between design and code, and this tension will intensify further.

Zhuo argued that “in a world of AI mass production, true quality lies in narrative depth, deliberate craft, and cultural resonance”36. AI raises the floor on the quality of mass-producible deliverables, but designers’ value shifts to the judgment and choices above that floor.

References

Footnotes

  1. ibm.com/think/prompt-engineering ↩

  2. developers.openai.com prompt engineering guide ↩

  3. x.com/tobi/status/1935533422589399127 (2025-06-19) ↩

  4. x.com/karpathy/status/1937902205765607626 (2025-06-25) ↩

  5. anthropic.com/engineering/effective-context-engineering-for-ai-agents (2025-09-29) ↩

  6. langchain.com/blog/the-rise-of-context-engineering (2025-06-23) ↩

  7. arxiv.org/abs/2507.13334 (2025-07-17, 166 pages) ↩

  8. simonwillison.net/2025/jun/27/context-engineering/ ↩

  9. anthropic.com/engineering/effective-harnesses-for-long-running-agents (2025-11-26) ↩

  10. openai.com/index/harness-engineering/ (2026-02-11) [requires primary verification] ↩ ↩2

  11. langchain.com/blog/the-anatomy-of-an-agent-harness (2026-03) ↩

  12. anthropic.com/engineering/harness-design-long-running-apps (2026-03-24) ↩

  13. anthropic.com/engineering/managed-agents (2026-04-08) ↩

  14. addyosmani.com/blog/agent-harness-engineering/ (2026-04-19) ↩

  15. claude.com/blog/steering-claude-code-skills-hooks-rules-subagents-and-more (2026-06-18) ↩

  16. Measured by Anthropic. Multi-agent 15x tokens, harness 20x cost ↩

  17. addyosmani.com/blog/loop-engineering/ (2026-06-07) ↩

  18. Peter Steinberger X post (2026-06-08). 6.5 million views [requires primary verification: specific URL] ↩

  19. Boris Cherny (Anthropic) Meta talk (2026-06-21) [requires primary verification: recording/transcript] ↩

  20. bdtechtalks.com/2026/06/22/ai-loop-engineering/ (2026-06-22) ↩

  21. x.com/karpathy/status/1886192184808149383 (2025-02-02) ↩

  22. simonwillison.net/2025/Oct/7/vibe-engineering/ ↩

  23. Karpathy X (2025-12 / 2026-04) [requires primary verification: specific URL] ↩

  24. siliconangle.com eval engineering (2026-05-17) ↩

  25. nngroup.com/articles/ai-era-critique/ (2026-06-12) ↩ ↩2

  26. nngroup.com/articles/context-architecture/ (2026-06-12) ↩

  27. github.com/google-labs-code/design.md (2026-04-21, v0.3.0 2026-06-15) ↩

  28. atlassian.com DESIGN.md test results (2026) ↩

  29. figma.com/blog/config-2026-recap/ (2026-06-25) ↩ ↩2

  30. johnmaeda.medium.com/design-in-tech-report-2026-from-ux-to-ax-f9d83164f4d2 (2026-03) ↩

  31. karpathy.bearblog.dev/sequoia-ascent-2026/ ↩

  32. arxiv.org/abs/2605.20731 (2026-05) ↩

  33. nngroup.com/articles/design-jobs-ai-created/ (2026-06-05) ↩

  34. smashingmagazine.com/2026/04/production-ready-becomes-design-deliverable-ux/ (2026-04) ↩

  35. nngroup.com/articles/state-of-ux-2026/ ↩

  36. lg.substack.com/p/the-ai-quality-coup (2025-04) ↩

← All Notes · Home