“Zero cross-contamination across 42+ projects after isolation fix.”
Client Context
A multi-project AI engine managed chains across 42+ projects from a single orchestration point. The engine had a RESUME-COMPLETE feature that detected when a chain had already been run and skipped execution. But the resume check was blind: it only verified that some previous run existed — not whether it belonged to the same project.
EBookr’s PRD was silently written to Pi’s Project-Docs directory because both runs used --project Pi. The engine host was always Pi. The real target was EBookr. The summary storedproject (the engine host) but nevertarget (the actual project being operated on). Every resume appeared to be for the same project.
Meanwhile, a subshell isolation bug in Phase 6.5 caused the loop to terminate after 2 chains instead of 6. The operator saw “5/6 chains complete” in the banner. Only one had actually run. The other five skipped silently.
The Illegibility Audit
We audited before we architected. Four diagnostic areas revealed the full picture of what was broken — and why a quick fix would fail without understanding the system dynamics.
Data Provenance
summary.json stored only aproject field — the engine host, always Pi. No target field for the actual project being operated on. Resume logic checked path existence only. No project-origin verification. The summary was lying — it said Pi for everything, masking which project the run was for.
Legacy Constraints
Agent templates had 61 hardcoded references to/Users/amar/Desktop/Projects/Pi. Templates could not be copied to another project without manual path editing. Every new project required duplicating and modifying template files. Portability was impossible by design.
Workflow Reality
source $PI_ENV was called inside a while loop body. Combined with set -e re-enabled after each iteration, the loop terminated after 2 out of 6 chains. The banner showed “5/6 chains complete” — a workflow reality mismatch that masked the throughput loss.
Incentive Alignment
RESUME-COMPLETE was designed for speed: skip completed work, run only what’s new. But speed was prioritized over correctness. The resume shortcut saved time at the cost of data integrity. No guard existed to verify that the previous run was actually for the same project. The incentive structure rewarded fast execution over accurate execution.
What We Built
We built rough first — a tactical three-commit fix that became a production-grade isolation layer. Each commit added depth to the guard. The result is a three-layer defense that prevents any chain from operating on the wrong project.
L1Governor: Project-Aware Orchestrator
chain-runner.sh now accepts a --project /-p flag. The orchestrator resolves the absolute target path, falls back to the engine root if the target is not found, and sets PI_PROJECT_ROOTas an environment variable for all downstream chains. The banner displays the target project for operator awareness. Manual resolution replaced implicit defaulting.
L2Executor: Target-Aware Runtime
run-chain-direct.py reads the target from --project and writes it as the target field in summary.json. Before resuming, the runtime compares the resolved prev_summary.targetagainst the current target_path. Mismatch means fresh start. One field — one field — eliminated the cross-contamination path.
L3Guard: Three-Way Resume Validation
A deterministic guard with three checks. First: targetfield match (new summaries). Second: legacy projectfield + task path regex fallback (old summaries). Third: spec-ID overlap via regex (pik-\d+|spec-[a-z])for task-match verification. Zero LLM calls in the guard logic — pure deterministic regex path comparison. This guard handles both new and legacy summaries without breaking the past.
L4Harness: Template Portability
61 hardcoded /Users/amar/Desktop/Projects/Pipaths replaced with the $PROJECTtemplate variable in run-chain-direct.py. Agent templates became portable. pik upgrade --fixnow works on any project, not just Pi. A new project can be initialized and run without editing a single template file.
Results
EBookr PRDs written to Pi’s Project-Docs. Silent data corruption across projects. No operator alert.
Zero cross-contamination. Every chain targets the correct project directory. Resume skips on target mismatch. Operator alerted.
source $PI_ENV moved outside loop. Subs hell replaced with explicit error handling. 5/6 chains now execute reliably per run.
What was corrected
- Resume guard:
summary.jsonnow storestarget— the actual project being operated on, not the engine host. Three-way guard prevents any cross-project resume match. - Chain runner:
--projectflag accepted and propagated to all downstream chains. Target displayed in banner.PI_PROJECT_ROOTset as environment variable. - Template paths: 61 hardcoded Pi paths replaced with
$PROJECTvariable. Templates work on any project without modification. - Subshell isolation:
source $PI_ENVmoved outside thewhileloop. Python run in explicit subshell(set +e; ...; exit)to isolate exit code from outerset -econtext. All 6 chains now execute reliably.
System Architecture
The isolation layer operates across four stages. Each stage is deterministic — zero LLM calls in the guard logic. The system prevents cross-project contamination at every level of the execution pipeline, from orchestration to path resolution.
Governor
chain-runner.sh resolves --project flag to absolute path. Falls back to engine root if target not found. Sets PI_PROJECT_ROOT for downstream chains. Displays target in banner.
Executor
run-chain-direct.py reads --project, writes target field to summary.json. Before resume, compares prev_summary.target with current target_path. Mismatch → fresh start.
Guard
Three-way resume validation: target field match (new), project field + task regex (legacy), spec-ID overlap via regex. Pure deterministic path comparison.
Harness
61 hardcoded Pi paths replaced with $PROJECT template variable. pik upgrade --fix now works on any project without template editing.
Business Impact
Projects protected
Zero cross-contamination after isolation fix
Throughput recovered
Subshell bug fix restored 5/6 chain execution per run
Template paths ported
Hardcoded Pi paths replaced with $PROJECT variable
“The resume feature checked whether the previous run existed — not whether it belonged to the same project. The fix added one field:target. One field. Three guard layers. Zero cross-contamination.”This engagement produced the Project Isolation Guard pattern — now reusable for any multi-project chain execution system. The pattern includes target-aware resume with three-way validation, legacy summary fallback, spec-ID overlap verification, and template portability via $PROJECT. Available in our Systemized Toolbox.