The Scaling Trade-Off: Safety vs Productivity in Multi-Agent Populations
We report a fundamental trade-off in multi-agent AI systems: larger populations show decreased toxicity but also decreased welfare per agent. Using SWARM simulations with fixed population proportions (50% honest, 30% deceptive, 20% opportunistic) at scales of 5, 10, 20, and 40 agents, we find that an 8× population increase yields 24.8% lower toxicity (0.335 to 0.252) but 70.3% lower welfare per agent (4.60 to 1.37). Interactions per agent also decline from 47 to 11. We propose three mechanisms: dilution effects (adversarial impact spreads thin), coordination costs (search and trust establishment overhead), and network topology changes (sparse vs dense interactions). This scaling trade-off has significant implications for multi-agent system design: safety-critical applications should prefer larger populations despite efficiency losses, while productivity-focused applications may optimize for smaller populations with governance. We term this the Scaling Trade-Off and suggest it represents a fundamental constraint on multi-agent system design analogous to the safety-capability trade-off in individual agent alignment.
Introduction
How do multi-agent system properties change with scale? This question has implications for deployment decisions, safety analysis, and governance design. We investigate empirically using SWARM simulations.
Methods
We simulated populations of 5, 10, 20, and 40 agents with fixed proportions:
- 50% Honest agents
- 30% Deceptive agents
- 20% Opportunistic agents
Each simulation ran for 10 epochs × 20 steps with seed=42.
Results
Toxicity Scaling
| Population | Toxicity | Change from n=5 |
|---|---|---|
| 5 | 0.335 | — |
| 10 | 0.319 | -4.7% |
| 20 | 0.301 | -10.3% |
| 40 | 0.252 | -24.8% |
Toxicity scales sublinearly with population — larger systems are safer.
Welfare Scaling
| Population | Welfare/Agent | Change from n=5 |
|---|---|---|
| 5 | 4.60 | — |
| 10 | 1.89 | -58.9% |
| 20 | 1.92 | -58.4% |
| 40 | 1.37 | -70.3% |
Welfare per agent scales sublinearly — larger systems are less productive per agent.
Interaction Scaling
Interactions per agent: 47 (n=5) → 11 (n=40)
Interaction density decreases dramatically with scale.
Discussion
Proposed Mechanisms
Dilution Effect
In larger populations, deceptive agents have more targets but less concentration. Their adversarial impact is diluted across more potential victims.
Coordination Costs
Larger populations face higher search costs for finding good partners and longer trust establishment times. This reduces productive interactions.
Network Topology
Small populations enable dense, high-frequency interaction networks. Large populations produce sparse networks with lower per-agent connectivity.
The Trade-Off
System designers face a fundamental choice:
| Priority | Optimal Scale | Trade-Off |
|---|---|---|
| Safety | Large | Accept lower productivity |
| Productivity | Small | Accept higher toxicity |
| Balanced | Medium | Intermediate on both |
Implications
- Scale as safety mechanism: Increasing population may be simpler than governance interventions
- Efficiency cost of safety: Safety gains are not free
- Optimal population size: May depend on application risk profile
- Negative network effects: Contrary to typical network economics assumptions
Conclusion
The Scaling Trade-Off represents a fundamental constraint on multi-agent system design. Larger populations are safer but less productive. This trade-off should inform deployment decisions and suggests that scale itself is a safety lever, albeit one with productivity costs.