Orbit: A framework for multi-agent security evaluations

This post announces work completed as part of the MATS 9 program, supervised by Dr. Christian Schroeder de Witt. Moving forward, Orbit will be supported by the Cooperative AI Foundation. We're grateful to MATS and our Research Manager, Srija Chakraborty, for their support. Repo: https://github.com/wlanderson0/orbit

We’re excited to release v0 of Orbit, a framework for multi-agent safety and security evaluations built on Inspect. This is still a work in progress, and we expect to substantially update and expand based on early feedback.

The world is increasingly multi-agent

Frontier models are now often deployed, implicitly or explicitly, as part of large multi-agent systems. Coding agents can invoke subagents, many people have agents helping manage their communications, and swarms of automated researchers are spinning up inside the labs. With this comes qualitatively new risks, as agents might miscoordinate, come into conflict, or collude to evade monitoring (Hammond et al., 2025, Schroeder de Witt et al., 2025). 

Current defenses fall short

Increasingly, the defenses built to monitor a single agent fall short when extended to multi-agent systems (Anthropic, 2025, Makins et al., 2026). Moreover, each system's configuration: its topology, roles, and communication patterns, changes the attack surface and where its vulnerabilities lie (Hagag et al., 2026), so a defense that works in one setup may fail in another. Even when researchers build defenses explicitly for multi-agent risks, they often evaluate this defense in some new, bespoke setting focused only on a narrow risk. This makes comparison difficult and slows progress. To this point, there was no standardized evaluation infrastructure for multi-agent systems.

Introducing Orbit

We built Orbit to help change this. Orbit extends Inspect to support decentralized, long-running environments with many independent agents. It supports arbitrary topologies, scheduling, memory sharing, and more. You can create new scenarios easily, and quickly configure experiments (altering topology, roles, models, etc.) on existing scenarios with YAML configs. We provide:

5 scenario families: coding, desktop, browser, customer-service, and cooperative-allocation tasks, 4 threat types: prompt injection, compromised agent, collusion, and misuse4 defense categories: security prompting, monitors, guardian agents, and dual-LLM

We’re aiming for extensibility, flexibility, hackability, so you can easily build new scenarios and recombine existing ones to work for you. We plan to expand the suite substantially over time. We’re also building many extensions and further tooling: custom viewers, integration with Hawk for large-scale evals, automated auditing, and more. 

Get started

We’re keen for you to try it out and let us know how to make it work for you.

Repo: https://github.com/wlanderson0/orbit

Contact form: https://forms.gle/XLX7UH2KAsnmMYUS8

View Original Article