31 personas, one orchestrator
Getting an 8-billion-parameter local model to produce a valid JSON execution plan, then running it across a fleet of 31 specialized personas: what breaks, what you harden, and why the hardest part is keeping the planner from being creative.
The setting: a virtual studio running locally
For a few months now I have been running a virtual studio: a Godot interface, a local daemon, and a registry of 32 personas, that is 31 executors plus cap, the orchestrator. Each executor has a narrow role: proofreading, translation, fact-checking, copywriting, code review, software QA. Fourteen of them run on in-house specialized fine-tunes; the rest fall back on a generic qwen3:8b topped with a role-play prefix.
One detail that matters: that fallback gets a forced /no_think directive at the top of the prompt. A qwen3 in thinking mode burns its entire generation budget on reasoning and returns empty responses. I learned that from a benchmark, not from documentation.
The usage principle: I drop a natural-language brief, cap produces an execution plan, and the plan runs in parallel across the fleet. On paper, it is elegant. In practice, everything hinges on the guardrails around the planner.
The contract: a strict JSON DAG, or nothing
cap must output a JSON object conforming to a home-grown schema, brief-plan-v1: a dated plan identifier, a title, and a list of steps forming a directed acyclic graph. Each step carries an id, a persona_slot that must come from the registry, its inputs and its dependencies. additionalProperties set to false everywhere: one invented extra field and the plan is rejected.
On the client side, I do the bare minimum before dispatch: stripping markdown fences (an 8B loves wrapping its JSON in ```), parsing, cycle detection in the dependencies, field normalization. Strict schema validation itself is delegated to the daemon through a dedicated endpoint. And one explicit rule in the prompt: cap is not allowed to assign itself as the executor of a step. It plans, it does not produce.
The survival loop: retrying with the error in hand
A small model botches its JSON. Often. The answer is not to hope for better, it is to loop: an invalid plan means a new attempt with the exact error injected back into the prompt, three tries maximum, then a clean abort rather than executing a dubious plan.
- A failing step gets 2 retries, then it is marked failed; the final synthesis is instructed to report the missing output, not to invent it.
- Steps whose dependencies are satisfied are dispatched in a single parallel batch call to the daemon, not one by one.
- Ollama's silent timeout is treated as a failure: zero tokens generated after more than 500 seconds, we cut it off instead of waiting forever.
const PLAN_RETRY_MAX := 3
const STEP_RETRY_MAX := 2
# While building cap's prompt (French in the source):
if not error_feedback.is_empty():
prompt += "[RETRY] Ton JSON précédent avait cette erreur: %s\nCorrige et renvoie un JSON valide.\n\n" % error_feedbackThe unexpected lesson: the persona-to-task matrix
My first version of cap's prompt listed the personas with their roles and left it the choice. Mistake. An 8B makes creative substitutions: the QA tester writing a marketing post because the topic is a game, the game designer drafted for SEO work. Technically plausible, systematically wrong.
So the prompt mutated into an explicit task-to-persona matrix, with restrictions spelled out in black and white. Here is a real excerpt, slightly shortened (the prompt is in French in the source):
MATRICE PERSONA → TÂCHE (respecte strictement, ne dévie pas) :
• Communication / posts social / LinkedIn → noa
• Fact-check / vérification factuelle → serge
• Traduction FR↔EN → tara
• Correction orthographe / grammaire FR → cora (elle ne produit pas de contenu nouveau)
RESTRICTIONS CRITIQUES :
• tina est une testeuse QA LOGICIEL — pas de community management ni de rédaction marketing
• Si tu hésites entre plusieurs personas, choisis celui spécifiquement listé ci-dessus
pour la tâche ; ne fais pas de substitution créativeThe other lesson: minimalism has to be written down
An 8B that plans wants to make a beautiful plan. On a brief like "fix this sentence", it spontaneously adds a synthesis step, a validation step, sometimes a final summary: three model calls where one is enough. Every parasitic step costs time, adds a failure point, and dilutes the useful output.
The countermeasure fits in one prompt rule, phrased without nuance (translated from the French source):
- The code backs it up: when a plan has a single step and it succeeded, the orchestrator returns its output directly, without going through an extra LLM synthesis call.
MINIMALISM (critical): for a simple unit task (fixing a sentence,
translating a short text, answering a factual question, summarizing a document),
produce ONE SINGLE step. NEVER add a 'synthesis', 'final summary' or 'validation'
step after it — the orchestrator handles synthesis automatically.
A 1-step plan = the correct answer for 80% of briefs.What I take away
Orchestrating small local models is 20% architecture and 80% code-level guardrails around a planner you cannot take at its word. The strict JSON contract, the retry that hands the error back to the model, the matrix that forbids substitution, the single-step rule: none of it is intelligence, all of it is framing.
And that may be the real mental inversion of the exercise: the model's creativity, precious in the 31 executors, is exactly what you have to switch off in the one that distributes the work. A good orchestrator is boring. Mine took a few weeks of ever-longer prompts to accept that.