Building the Kitchen: Frameworks & Runtime Environments for Agentic AI

Imagine that we have already designed a kitchen.

We know what ingredients we need, which equipment is required, how different stations coordinate, and how the chef decides what to prepare next.

But there is still one important question.

How do we actually build this kitchen and make it operational?

We could build everything from scratch. We could use existing equipment and systems. Or we could combine both approaches.

Agentic AI development presents a similar challenge.

We understand the role of foundation models, how tools extend their capabilities, how execution works, and how orchestration coordinates the overall process.

But connecting these components into a working application requires more than simply writing a prompt and calling a model API.

We need ways to define agents, connect tools, manage state, coordinate execution, handle failures, and observe what happens while the system runs.

This is where frameworks and runtime environments enter the picture.

In this article, we'll explore what agentic AI frameworks actually provide, how runtimes fit into the architecture, what different frameworks bring to the table, and how to decide whether a framework is even necessary.

🍳 From Architecture to Implementation

Let's briefly revisit our kitchen.

So far, we've explored several important layers:

• Foundation models: The intelligence behind the agent.
• Tools: The capabilities that allow an agent to interact with external systems.
• Execution: The process through which those capabilities are used.
• Orchestration: The coordination of actions toward a goal.
• State and observations: The information needed to understand progress and make subsequent decisions.

We have the architectural blueprint.

But a blueprint doesn't operate a kitchen.

We need the infrastructure and equipment that bring the design to life.

In software, developers can implement these components independently using model APIs, application code, databases, tool integrations, and custom orchestration logic.

However, as the system grows, managing every interaction manually can become difficult.

Consider what happens when an agent needs to:

1. Receive a user request.
2. Send the request and context to a model.
3. Interpret the model's response.
4. Execute a requested tool.
5. Return the tool result to the model.
6. Continue until a stopping condition is reached.
7. Preserve relevant state for future interactions.
8. Record what happened for debugging and evaluation.

Even this relatively simple interaction requires several moving parts.

A framework can provide reusable abstractions for many of these responsibilities.

But before exploring frameworks, we need to understand what they actually are.

🧰 What Exactly Is an Agentic AI Framework?

Think of a framework as a collection of prebuilt kitchen equipment, standard procedures, and reusable components.

Instead of manufacturing every utensil and designing every process ourselves, we use existing components to assemble the kitchen.

An agentic AI framework provides abstractions and utilities that help developers build applications involving models, tools, state, and orchestration.

Depending on the framework, it may provide:

• Agent definitions and configuration.
• Model-provider integrations.
• Tool registration and execution.
• Agent loops and control-flow primitives.
• Sequential, parallel, or graph-based workflows.
• State and conversation management.
• Human approval and intervention.
• Error handling and execution controls.
• Tracing, monitoring, and debugging integrations.

The important detail is that not every framework provides all these capabilities, and the same capability may be implemented differently across frameworks.

Some frameworks focus on lightweight agent loops. Others emphasize explicit workflow graphs, multi-agent coordination, or long-running execution.

The framework you choose influences how much of the architecture you implement yourself and how much is provided through its abstractions.

Frameworks are not the agents themselves

This distinction is worth understanding.

An agent is the system that uses a model, instructions, tools, and execution logic to pursue a goal.

A framework is a development layer that helps us build and manage that system.

A framework doesn't automatically make an agent intelligent, reliable, or production-ready.

It provides the building blocks. The developer is still responsible for deciding how those blocks should be used.

⚙️ Framework vs. SDK vs. Runtime vs. Platform

These terms are often used together, sometimes even interchangeably.

But they describe different responsibilities.

Let's separate them.

Framework

A framework provides abstractions and reusable components for building an application.

For example, it might provide agent definitions, tool interfaces, workflow primitives, and state-management utilities.

Think of it as the collection of equipment and construction patterns used to assemble the kitchen.

SDK

A Software Development Kit provides libraries, APIs, and utilities that help developers interact with a particular service or system.

An SDK may be part of a framework, or it may be a standalone way to interact with an API.

For example, a model provider might offer an SDK for sending requests, handling responses, and integrating model capabilities into an application.

An SDK doesn't necessarily provide a complete agent architecture.

Runtime

A runtime provides the execution machinery responsible for running an application.

For an agentic system, runtime responsibilities may include:

• Running the agent loop.
• Invoking tools.
• Managing execution state.
• Handling continuation and interruption.
• Streaming intermediate output.
• Enforcing timeouts and execution limits.
• Coordinating handoffs.
• Integrating with the environment where the agent operates.

A framework may include a runtime, provide an interface to one, or depend on an external runtime.

Platform

A platform provides a broader environment for building, deploying, operating, or managing applications.

Depending on the platform, this may include hosted model access, managed agent execution, storage, monitoring, authentication, and deployment infrastructure.

A platform may offer its own SDKs and frameworks, but these are not necessarily the same thing.

Putting the differences together

One important clarification: These terms describe different responsibilities, not necessarily separate software products.

A single framework can provide both development abstractions and runtime capabilities. LangGraph, for example, explicitly describes itself as both a framework and runtime.

The distinction is architectural rather than strictly organizational.

🔄 What Actually Happens Inside an Agent Runtime?

Let's return to the kitchen.

The chef receives an order, decides what to prepare, uses the equipment, checks the results, and continues until the meal is ready.

The runtime is responsible for making this execution process happen.

A simplified agent runtime might follow this sequence:

1. Receive input

The application receives a user request and prepares the relevant context.

2. Invoke the model

The runtime sends the model the instructions, available tools, and necessary information.

3. Interpret the response

The model may return a final answer, request a tool call, or indicate that control should move elsewhere.

4. Execute the requested operation

If a tool call is requested, the runtime or application executes it according to the configured tool contract and permissions.

5. Return the observation

The tool result is added to the ongoing execution context.

6. Continue or stop

The runtime determines whether another model interaction is needed or whether a valid stopping condition has been reached.

This loop may repeat several times.

A model response is not necessarily the end of an agent run.

The runtime needs to understand the difference between an intermediate decision and a completed task.

Why this matters

Imagine an agent investigating a cloud incident.

It queries application logs, receives a partial result, and decides to retrieve the next page.

The runtime or application must support continuation, preserve the relevant execution state, and return the additional evidence to the agent.

If the next request fails, the system must handle that failure without silently treating the investigation as complete.

This is where runtime behavior becomes important.

The quality of the model's reasoning matters, but so does the machinery responsible for executing and coordinating its decisions.

🏗️ Exploring the Framework Landscape

There isn't one universal framework for building agentic AI applications.

Different frameworks make different architectural choices.

Let's look at three examples and understand what they bring to the table.

These are examples of different design approaches, not a ranking of which framework is best.

1. LangGraph — Explicit Control Over Workflows

Documentation: https://docs.langchain.com/oss/python/langgraph/overview

Imagine designing a kitchen where every station is represented as a node, and the connections between stations define how work moves through the kitchen.

This is the mental model behind LangGraph.

LangGraph is a low-level orchestration framework and runtime for building long-running, stateful agents.

It represents execution through:

• Nodes: Individual operations, such as model calls or tool execution.
• Edges: Connections that determine where execution moves next.
• State: Information shared and updated throughout the workflow.

A graph can contain deterministic steps as well as model-driven decisions.

For example, a cloud investigation workflow might look like this:

Receive incident → Retrieve logs → Inspect metrics → Evaluate evidence

The evaluation step could determine whether the system has enough information to continue, investigate another source, or return a result.

One important distinction is that LangGraph doesn't force developers into a particular agent architecture.

It gives developers control over the execution graph and state. They can combine ordinary program logic with LLM-driven decisions in the same workflow.

Its capabilities include:

• Durable execution through checkpointing and recovery.
• Streaming of intermediate results and state updates.
• Human-in-the-loop interruption and continuation.
• Short-term and longer-term memory patterns.
• Explicit control over workflow transitions.

For example, an investigation can pause while waiting for human approval and resume from a saved checkpoint rather than starting the entire process again.

LangGraph is particularly useful when developers need explicit control over execution flow, shared state, durable execution, and human intervention.

It is especially valuable when the workflow itself is an important part of the application's architecture.

However, graph-based orchestration also introduces design complexity.

A graph with too many nodes, transitions, and shared-state dependencies can become difficult to maintain.

The goal is not to represent every small operation as a separate node. It is to make important control-flow decisions explicit.

2. OpenAI Agents SDK — A Lightweight Agent-Oriented Approach

Documentation: https://developers.openai.com/api/docs/guides/agents/sdk

Imagine having a collection of standardized kitchen stations that already know how to receive instructions, use equipment, and coordinate handoffs.

You configure the chef and the available capabilities rather than implementing every interaction from scratch.

The OpenAI Agents SDK provides a code-first approach to building agentic applications.

Its core concepts include:

• Agents: Models configured with instructions and tools.
• Tools: Functions or integrations that agents can invoke.
• Handoffs: Mechanisms for transferring control to another agent.
• Guardrails: Validation and safety mechanisms.
• Sessions and state: Ways to manage information across interactions.
• Tracing: Visibility into what happened during an execution.

The SDK provides a runner that manages the agent loop.

At a high level, the runner:

1. Invokes the current agent.
2. Inspects the model's response.
3. Executes requested tools when necessary.
4. Transfers control when a handoff occurs.
5. Continues until a final result or another stopping condition is reached.

For example, imagine a support agent that receives a request and determines that a billing specialist should handle it.

The main agent can hand off control to the specialist rather than manually implementing every interaction between the two.

An important distinction: SDK vs. hosted execution

The OpenAI Agents SDK runs inside your application.

Your application remains responsible for important decisions involving deployment, infrastructure, storage, approval policies, and tool implementations.

This is different from a managed agent service where the provider operates more of the underlying execution environment.

OpenAI's current agent offerings distinguish between the Agents SDK, the hosted Agents API, and the lower-level Responses API.

They serve different development and execution needs.

The SDK is useful when developers want reusable agent primitives while retaining control over their application and infrastructure.

It doesn't mean that deployment, persistence, or operational concerns disappear.

Developers still need to design the surrounding application.

3. Microsoft Agent Framework — Agents and Workflows Together

Documentation: https://learn.microsoft.com/en-us/agent-framework/overview

Imagine a kitchen where individual chefs can work independently, but the entire kitchen also has a defined system for coordinating their work.

Microsoft Agent Framework brings together agent development and workflow orchestration.

Its current architecture covers four major areas:

• Agents: Individual agents that use models, tools, and integrations to process requests.
• Harness Agents: An opinionated, more complete setup for long-running, multi-step work.
• Workflows: Functional and graph-based workflows that connect agents and operations through explicit execution paths.
• Integrations: Connections to model providers, tools, MCP servers, middleware, and other services.

One particularly interesting capability is the Agent Harness.

Think of a harness as the additional structure surrounding an agent to help it carry out longer-running work.

Depending on its configuration, it can provide capabilities such as:

• Planning and task tracking.
• Context management and compaction.
• File access and memory.
• Tool approval policies.
• Execution and observability support.

This moves beyond simply defining an agent and connecting tools. It provides additional structure for managing the work an agent performs over multiple steps.

The framework also supports sessions, context providers, middleware, and explicit workflow orchestration.

Its relationship with AutoGen and Semantic Kernel

Microsoft Agent Framework represents the next generation of Microsoft's agent development ecosystem.

Microsoft describes it as the direct successor to AutoGen and Semantic Kernel, bringing together concepts from both projects.

It combines agent abstractions, enterprise-oriented development capabilities, and explicit workflow orchestration.

This makes it particularly interesting for applications that combine autonomous agent behavior with structured business workflows.

One important consideration is language and feature availability.

The framework supports Python and .NET, but feature parity should not be assumed across every language and integration. Some capabilities and newer features may have different availability or maturity depending on the environment.

For example, Microsoft's current documentation identifies its Go implementation as being in public preview, with some capabilities not yet available.

Always check the documentation for the language and version you intend to use.

🧭 Comparing the Architectural Approaches

We've now explored three frameworks.

But comparing them only by their feature lists doesn't tell us how they would approach the same problem.

Let's use our cloud incident investigation scenario.

A user asks:

"Investigate why the application started returning errors after the latest deployment."

Each framework can help developers build a system that investigates this problem, but the architectural approach can differ.

With LangGraph

The developer explicitly models the investigation as a graph.

Start → Retrieve logs → Inspect deployment → Analyze metrics → Evaluate evidence

Each operation updates shared state.

The evaluation step determines which node should run next. If evidence is missing, the workflow can branch into another investigation. If approval is needed, execution can pause and resume later.

The emphasis is on explicit control over the workflow and its state.

With OpenAI Agents SDK

The developer defines an investigation agent with instructions and tools for retrieving logs, deployment history, and metrics.

The runner manages the agent loop.

The model can decide which tool to call next, interpret the results, and continue the investigation. If the application uses specialist agents, it can also configure handoffs.

The emphasis is on agent primitives, tool use, and application-controlled execution.

With Microsoft Agent Framework

The developer can combine an investigation agent with a structured workflow.

For example, the workflow might define required investigation stages, while the agent handles open-ended analysis within a particular stage.

The system can also use sessions, middleware, and harness capabilities where appropriate.

The emphasis is on combining agents, structured workflows, and broader application capabilities within Microsoft's agent ecosystem.

A practical comparison

These are starting points, not exclusive capabilities.

All three overlap in areas such as tool execution, state management, and orchestration.

The choice should depend on the required control flow, integrations, execution model, language, and operational constraints.

🧩 Frameworks Hide Complexity, But Don't Remove It

One of the biggest advantages of frameworks is that they reduce repetitive implementation work.

But abstraction comes with a trade-off.

Imagine using a modern kitchen appliance.

You don't need to understand every internal electrical component to operate it. The appliance provides a simpler interface.

However, when something goes wrong, understanding what happens underneath becomes important.

Agent frameworks work similarly.

Depending on the framework, they may provide abstractions for:

• Model request handling.
• Tool-call parsing.
• Execution loops.
• State transitions.
• Agent handoffs.
• Retry policies.
• Streaming and tracing.

But the underlying operations still exist.

A framework cannot automatically guarantee that:

• A tool returns complete information.
• An agent chooses the correct action.
• A retry is safe.
• A workflow terminates at the right time.
• A multi-agent system reaches a correct conclusion.
• Sensitive operations are properly authorized.

Even retries need deliberate design.

A reliable retry policy must consider:

• Which failures are actually retryable.
• How many attempts are allowed.
• How long the system should wait between attempts.
• Whether the operation is safe to repeat.
• How duplicate side effects are prevented.

For example, retrying a read-only log query may be reasonable after a temporary network failure.

Retrying a payment or infrastructure modification without considering duplicate execution could produce a very different outcome.

An abstraction makes complexity easier to work with. It does not make the complexity disappear.

This is why understanding the architecture before selecting a framework matters.

🧠 The Framework Is Not the Runtime Environment

Let's take the distinction a step further.

Suppose we have built an agent using a framework.

Where does it actually run?

It might run:

• Inside a local development process.
• As part of a backend application.
• Inside a container.
• In a serverless environment.
• On a dedicated worker.
• In a managed cloud environment.

The framework helps define and coordinate the agent.

The execution environment determines where and under what conditions that application operates.

This distinction becomes particularly important when agents perform long-running tasks or interact with external systems.

Consider a cloud operations agent that takes several minutes to investigate an incident.

What happens if the application process restarts?

What if a tool request times out?

What if the agent pauses while waiting for human approval?

What if the task needs to resume later?

These are not simply model or prompt questions.

They are runtime and infrastructure questions.

A production architecture may need:

• Persistent execution state.
• Durable task records.
• Background workers.
• Timeout and retry policies.
• Secure tool execution environments.
• Authentication and authorization.
• Concurrency controls.
• Logging and tracing.
• Recovery and continuation mechanisms.

A framework may provide some of these capabilities, but others may belong to the application or infrastructure layer.

The important thing is to know where each responsibility lives.

Three kinds of state worth distinguishing

The word state appears frequently in agentic AI documentation, but it can refer to different things.

1. Conversation state

This includes previous messages and the relevant context needed to continue an interaction.

For example, an assistant may need to remember which application the user was discussing earlier in the conversation.

2. Execution state

This represents the progress of the current agent run or workflow.

For example:

• Which investigation steps have completed?
• Which tasks are still pending?
• Which tool calls failed?
• What evidence has been collected?
• Is human approval required?

3. Durable state

This refers to information persisted so that the system can recover or resume after an interruption.

For example, if an incident investigation pauses while waiting for approval, the system should be able to recover its progress rather than restarting from the beginning.

These are different architectural concerns, not necessarily three separate databases or objects.

An application may store them together or use different storage mechanisms depending on its requirements.

The important part is that the system knows what information must survive beyond a single model call or application process.

🛡️ Production Readiness: What Should We Evaluate Before Choosing a Framework?

A framework may make a prototype look impressive.

But a working demonstration and a dependable production application are two different things.

Before selecting a framework, consider the following areas.

1. Control over execution

Can you control the agent loop, tool execution, and workflow transitions?

Can you decide when the agent should stop, retry, or hand off control?

This matters when the application has strict business rules or operational requirements.

2. State and persistence

Can the system preserve conversation and execution state across interactions?

Can a workflow resume after an interruption?

Can you distinguish between completed, failed, pending, and partially completed tasks?

3. Tool integration

Can the framework connect to the tools and services your application needs?

Does it support the integration approach you intend to use?

Can you define clear contracts for tool inputs, outputs, errors, and completeness?

4. Security and permissions

Can you control which tools an agent can access?

Can sensitive operations require human approval?

Can the application enforce authorization outside the model's instructions?

Remember: a framework exposing a tool does not automatically make that tool safe.

5. Observability and debugging

Can you inspect model calls, tool invocations, handoffs, and workflow transitions?

Can you understand why the agent selected a particular action?

Can you identify where an execution failed?

Without observability, debugging a multi-step agent can become extremely difficult.

6. Evaluation and testing

Can you test individual components and complete workflows?

Can you simulate unavailable tools, incomplete results, and unexpected model decisions?

Can you compare behavior across different runs?

A framework that makes development easy but provides limited visibility into execution may become difficult to operate as the application grows.

7. Portability and integration

Does the framework work with your preferred model providers, infrastructure, programming language, and existing services?

How tightly does your application depend on framework-specific abstractions?

Can you replace a component without rebuilding the entire system?

These questions are often more valuable than comparing the number of features advertised by different frameworks.

⚖️ Should We Always Use an Agent Framework?

No.

This is one of the most important decisions in agent engineering.

Sometimes developers introduce an agent framework simply because they are building something involving an LLM.

But not every LLM-powered application needs an agent, and not every agent needs a heavy framework.

Consider three scenarios.

Scenario A: A simple question-answering application

A user asks a question, the application sends it to a model, and the model returns an answer.

There may be no need for an agent framework.

A direct model API integration could be sufficient.

Scenario B: A tool-using assistant

An assistant receives a request, selects a tool, retrieves information, and produces a response.

A lightweight agent SDK or a small custom loop may provide useful abstractions without requiring a complex workflow engine.

Scenario C: A long-running operational agent

An agent investigates incidents, coordinates several tools, tracks progress, waits for approvals, and may need to resume after interruptions.

This kind of system may benefit from a framework with explicit workflow management, persistent state, and durable execution capabilities.

A simple framework selection guide

Use this as a starting point rather than a strict technical rule.

Start with your application requirements
                 |
                 v
       Is it just a model call
          with a response?
           /          \
         Yes           No
          |             |
          v             v
      Direct API    More complex behavior
          |             |
      Keep it simple    v
                 Does it require tool
                 calling and a basic
                    agent loop?
                    /       \
                  Yes        No
                   |          |
                   v          v
            Lightweight SDK   Complex workflows,
            or custom loop    persistence, branching,
                              or multiple agents?
                                  |
                                  v
                         Consider an orchestration
                               framework

The decision should follow the complexity of the problem.

A useful principle is:

Start with the simplest architecture that satisfies the requirements. Introduce abstractions when they reduce real complexity, not simply because they are available.

🔍 A Practical Way to Think About the Entire Stack

Let's bring everything together using our cloud incident investigation example.

A user asks:

"Investigate why the application started returning errors after the latest deployment."

The system may contain the following components:

Foundation model

Interprets the request, reasons over observations, and proposes appropriate next steps.

Tools

Retrieve logs, inspect deployment history, and query infrastructure metrics.

Orchestration

Coordinates the investigation, decides what happens next, and handles branching or delegation.

Framework

Provides reusable components for defining agents, connecting tools, managing state, and implementing orchestration.

Runtime

Executes the agent loop, manages tool interactions, handles continuation, and controls the running workflow.

Infrastructure

Provides the environment in which the application runs, including compute, networking, storage, identity, and monitoring.

Evaluation and observability

Help developers understand whether the investigation behaved correctly and whether the final conclusion was supported by sufficient evidence.

Agentic AI application architecture

flowchart TD
    A["Application / User Request"]

    subgraph F["Agentic AI Framework"]
        B["Agent Definitions"]
        C["Orchestration"]
        D["Tool Registration"]
        E["State Management"]
    end

    G["Agent Runtime / Runner"]

    H["Foundation Model Provider"]
    I["External Tools and Services"]
    J["Persistent State Store"]
    K["Observability and Monitoring"]

    L["Infrastructure<br/>Compute · Networking · Identity"]

    A --> B
    B --> C
    C --> G
    D --> G
    E --> G

    G <--> H
    G <--> I
    G <--> J
    G --> K

    G --> L

Conceptual architecture. Actual framework and runtime boundaries can overlap, and some responsibilities may be implemented by the application or an external platform.

Notice something important.

The framework is only one part of this architecture.

It doesn't replace the model, tools, orchestration design, runtime environment, or infrastructure.

It helps bring these components together.

🧠 The Mental Model to Take Away

1. Frameworks provide building blocks. They reduce repetitive implementation work and offer reusable abstractions.
2. Frameworks and runtimes describe different responsibilities. A single product can provide both.
3. Different frameworks make different architectural choices. Understand those choices instead of selecting a framework based only on popularity.
4. State is more than conversation history. Execution progress and durable recovery are important concerns too.
5. Abstraction doesn't remove engineering responsibility. Reliability, permissions, validation, and failure handling still matter.
6. Not every application needs a framework. Choose the simplest architecture that solves the actual problem.

The model provides intelligence. Tools provide capabilities. Orchestration coordinates actions. Frameworks help assemble the system. Runtimes make it execute.

Each layer contributes something different.

🔪 From Building the Kitchen to Connecting the Kitchens

We've now moved from understanding the architecture of an agentic system to understanding how developers can implement and operate one.

But modern agentic systems don't always operate in isolation.

They need to interact with databases, external services, internal applications, other agents, and sometimes entirely different systems built by different teams.

How do these components communicate?

How can an agent discover and use external capabilities without requiring a custom integration for every individual system?

And how can multiple agents exchange information and coordinate across boundaries?

That takes us to another important layer of the architecture: communication and integration protocols.

Next: Connecting the Kitchen — MCP, A2A & Agent Communication

Further reading

LangGraph

• LangGraph — Overview
• LangGraph — Durable execution

OpenAI

• OpenAI Agents SDK
• Running agents
• Agents — Choosing a runtime

Microsoft

• Microsoft Agent Framework — Overview
• Microsoft Agent Framework — Agent Harness
• Microsoft Agent Framework — Documentation

This article is Part 5 of The Culinary Architecture of Agentic AI, a series exploring the architecture, components, and engineering principles behind agentic systems.

Last reviewed: November 2026. Framework capabilities and language support evolve; consult the official documentation for current version-specific details.