Handling shared state across multi-agent conversations in AutoGen

[Arpanwanwe]

I've been working on developing multi-agent systems (like planners, executors, reviewers, and tool callers) using AutoGen, but one of the continual problems I face is the ability to keep track of information that is shared between multiple instances of dialogue (when multiple instances of dialogue happen simultaneously).

While AutoGen provides an efficient way of passing messages from one agent to another to maintain a linear flow of communication between agents, once you introduce the concept of retrying actions, executing on conditional paths and completing longer-running tasks the context that is generated by each agent during execution of a specified command is generally spread throughout the messages exchanged between the agents and the application's business logic. This makes it difficult to understand why agents acted the way they did, or to duplicate them.

To resolve this issue, I have been developing a mechanism where agents would operate from a common shared specification/state, instead of just working from conversation history. Recently I began evaluating this idea alongside AutoGen for compatibility with a small-scale orchestration type system (Zenflow) to determine if there was an increase in detailed analysis capabilities. I am still working through the benefits and drawbacks of this method.

I would like to know what type of solutions others working with AutoGen have created. Are they focusing on having a single location for defining the application's shared state, are they extending the base agent classes to define specific states for each agent, or are they using other means (such as an external state machine) to manage and assist in coordinating the actions of agents?

[KeepALifeUS]

This is a challenge I've spent significant time on as well. The core issue you're describing — state fragmented across message history, hard to trace why agents acted certain ways — is fundamental to most multi-agent frameworks.

What I've found effective:

1. Shared specification/state approach (similar to what you're testing with Zenflow) — This is the right direction. Instead of agents relying solely on conversation history, they operate on a structured shared state that's explicitly read and written. This makes the system deterministic and debuggable.

2. Stigmergy-based coordination — Rather than agents passing messages directly, they interact through a shared environment (inspired by how ant colonies coordinate via pheromone trails). Benefits:

   • Agents can operate asynchronously without waiting for responses
   • The "why" of agent decisions becomes traceable through state changes
   • Significantly reduces token usage since agents don't need full conversation context

3. Typed state transitions — Define explicit state schemas that agents can read/modify. This catches errors early and makes retry logic much cleaner since you know exactly what state to restore.

I built a production system using these patterns with 4 specialized agents. Instead of direct agent-to-agent communication, each agent reads signals from the shared environment and writes its outputs back. The result was ~80% reduction in API tokens and much easier debugging.

If you want to see the architecture: https://github.com/KeepALifeUS/autonomous-agents

For AutoGen specifically, I think extending the agent classes to work with a centralized state container (rather than just message history) is promising. You might also look at combining AutoGen's conversation patterns with an external state machine for longer-running workflows.

What scale of multi-agent system are you building? Happy to share more specific patterns based on your use case.

[jovanSAPFIONEER]

Great discussion — this is exactly the problem I've been solving.

@Arpanwanwe, the fragmented state across message history issue you described is why I built a locked blackboard pattern into network-ai. It's an open-source TypeScript orchestration framework with a dedicated AutoGen adapter, so it works alongside your existing AutoGen agents.

The core idea is similar to what you and @KeepALifeUS are describing — a centralized shared state that agents read/write explicitly — but with atomic conflict resolution built in:

How it works:

1. Propose → Validate → Commit — Agents don't write directly. They propose changes, the system validates for conflicts, then commits atomically. No race conditions when multiple dialogues run simultaneously.

2. Priority-based preemption — When two agents write to the same key (e.g., a planner and executor both updating task_status), priority levels (0-3) determine who wins. Same priority falls back to first-commit-wins. Preempted changes get metadata — nothing is silently dropped.

3. Full audit trail — Every state change is logged with agent ID, timestamp, and cryptographic signatures. The "why did agents act this way" problem becomes a query against the audit log, not archaeology through message history.

import { LockedBlackboard } from 'network-ai';

const board = new LockedBlackboard('./shared-state', auditLogger, {
  conflictResolution: 'priority-wins'
});

// Planner proposes (priority 1)
const id = board.propose('planner', 'task:execute', { status: 'ready', params: {...} }, 300, 1);

// Executor reads shared state
const task = board.read('task:execute');

// Executor proposes update (priority 2 — takes precedence)
board.propose('executor', 'task:execute', { status: 'running', result: partial }, 300, 2);


For your retry/conditional path use case specifically — the blackboard holds the canonical state, so retrying just means reading the current state and re-proposing. You don't need to replay conversation history to reconstruct where you are.

Re: AutoGen integration — network-ai has a dedicated AutoGen adapter that normalizes agent registration and task delegation. Your existing conversable agents register through the adapter and can coordinate with agents from other frameworks (LangChain, CrewAI, etc.) through the same shared blackboard.

The security layer also addresses a problem most shared-state systems ignore — agents gaming the permission system. We just shipped hardened justification scoring (v3.2.2) that detects prompt injection in permission requests, which matters when agents have different trust levels accessing sensitive resources.

315 tests passing, zero dependencies, MIT licensed: [https://github.com/jovanSAPFIONEER/Network-AI](vscode-file://vscode-app/c:/Users/Racunar/AppData/Local/Programs/Microsoft%20VS%20Code/c3a26841a8/resources/app/out/vs/code/electron-browser/workbench/workbench.html)

Happy to dive deeper into any of these patterns — especially the conflict resolution and retry workflows. What does your agent topology look like?

[ThinkOffApp]

I run a fleet of 9 agents on a shared Mac mini and hit this exact problem. Here is what actually worked for me after trying several approaches.

I ended up with a simple file-based shared state layer. Each agent writes to its own workspace directory but reads from a common state file that acts as a lightweight blackboard. The key insight was separating "agent-local scratchpad" from "team-visible state" so agents do not accidentally overwrite each other.

For concurrent conversations, I use a cache-ttl pruning strategy (1 hour window) so old context drops off automatically. This avoids the problem of agents referencing stale decisions from hours ago. Combined with a compaction step that summarizes long histories before they hit the context window limit, it keeps things manageable even when multiple agents are active at once.

The trickiest part was rate limit coordination. When one agent burns through an API provider's quota, the others need to know immediately or they all start failing in sequence. I ended up logging rate limit errors to a shared file that other agents check before making calls. Not elegant, but it stopped the cascading failures.

One thing I would push back on is trying to build a fully consistent shared state system. In practice, eventual consistency with short TTLs worked better than locks or transactions. Agents can tolerate slightly stale data as long as they are not making irreversible decisions based on it.