Public text edition. Original roles are preserved; pasted messages remain attributed to the exported speaker. Private personal exchanges are marked where omitted. Images, tools and inaccessible working files are not reproduced. Statements record the discussion, not independent verification.
ChatGPT · 2026-10-25T17:34:19-07:00 · active branch
Disposition: AMENDMENT
Participant: ChatGPT (GPT-6)
Review: Pass 1 — Independent Architectural Reassessment
Position: Support broker-first simplification, with a mandatory baseline and evidence-governance amendment.
Authority: Deliberation only.
1. Exact proposed amendment
Replace the proposed initial implementation recommendation with:
> AMENDMENT — Baseline-First Capability Decision Experiment
>
> Little Oogway shall not receive a new capability knowledge subsystem until a documented broker-gap audit establishes deficiencies in the existing system.
>
> The first comparison shall use three configurations:
>
> A — Broker Baseline: Existing capability broker, registered contracts, and execution-event review, without new capability guidance.
>
> B — Broker + Compact Capability Card: A deterministic, version-bound, approximately 500–700-token decision card containing only locally established controls, known limitations, effect routing, prerequisites, and escalation conditions.
>
> C — Broker + Capability Card + Evidence Retrieval: Configuration B supplemented with bounded TokenSpace retrieval of manuals, operational lessons, and relevant Human Outcome guidance.
>
> All configurations retain identical live authorization controls.
>
> No configuration may infer executable authority from documentation, model reasoning, prior successful execution, or another AI's recommendation.
>
> The capability layer shall be accepted only if independently scored experiments demonstrate meaningful improvement over Configuration A without unacceptable safety, reliability, latency, or resource regressions.
>
> Configuration C shall be adopted over B only when it demonstrates additional measurable benefit.
>
> Until these conditions are satisfied, broader ingestion, new services, and automated knowledge promotion remain deferred.
Reason: Claude's three-arm experiment is materially more informative than the original A/B design. DeepSeek's minimal overlay is compatible with it. My preliminary recommendation did not adequately isolate the incremental value of retrieved documentation.
2. Independent assessment of the other participants
Agreement with Claude: Manuals should remain outside the runtime decision card. A retrieved manual is evidence, not a control surface. I also accept the distinction between technical acceptance thresholds and creative evaluation criteria.
Agreement with DeepSeek: The broker should remain the sole operational authority, and validated negative evidence is among the highest-value additions.
Challenge to both: A capability card can become stale even when the executable version remains unchanged. Configuration changes, adapter updates, model weights, service availability, installed plugins, and authorization changes can alter behavior.
Therefore, a version fingerprint alone is insufficient.
Additional challenge: A local probe does not automatically establish universal capability. One successful result demonstrates behavior under the tested conditions. One failure does not establish impossibility.
The term verified limitation must require an explicitly bounded test domain, a reproducible result, and recorded environmental conditions.
Material dissent: I oppose requiring every locally untested documented feature to be treated as nonexistent. It should instead be classified DOCUMENTED_UNVERIFIED, allowing investigation without permitting an unsupported capability claim.
3. Minimal architecture and schema
Use three existing-system extensions, not six independent components:
- A compact decision card referenced by the broker.
- An evidence-linked outcome ledger extending existing event history.
- A bounded retrieval policy using TokenSpace.
Proposed minimum schema:
schema_version: "0.1"
family: kokoro_tts
card_version: "0.1"
environment_fingerprint: null
broker_contract_refs: []
last_verified: null
status: DRAFT
controls:
verified: []
documented_unverified: []
absent_from_interface: []
effects:
routing: []
constraints:
prerequisites: []
resource_cost: unknown
reversibility: unknown
authorization_gates: []
limitations: []
human_outcome_refs: []
evidence_refs: []
escalation_triggers: []
Each limitation must link to its evidence record.
The card must not embed credentials, duplicate broker permission grants, or include extensive documentation.
A complete manual is never part of the card itself.
4. Diagnostic policy
The resolver should report an explicit state before selecting an action.
| State |
Required evidence |
Response |
KNOWLEDGE_GAP |
Existing evidence insufficient |
Bounded retrieval |
TOOL_ABSENT |
No applicable registered capability |
Alternative or escalation |
INTERFACE_GAP |
Required control absent from live contract |
Alternative mechanism |
AUTHORITY_GAP |
Existing gate denies or requires approval |
Stop/request authorization |
DEPENDENCY_GAP |
Required component unavailable |
Diagnose without unauthorized repair |
TESTED_LIMITATION |
Reproducible bounded negative evidence |
Respect scope; choose alternative |
QUALITY_GAP |
Output fails stated outcome criterion |
Revise/evaluate |
EVIDENCE_CONFLICT |
Material incompatible evidence |
Abstain or investigate |
UNKNOWN |
Classification not justified |
Ask or investigate |
An absent interface parameter must never be automatically translated into an impossible outcome.
For example, missing Kokoro emotion controls might still permit useful emotional effects through voice choice, segmentation, timing, or post-processing.
Whether those techniques actually satisfy listeners remains experimental.
5. Lifecycle and evidence promotion
Recommended lifecycle:
DRAFT → SOURCE-CHECKED → RETRIEVAL-VERIFIED → LOCALLY-TESTED → ACTIVE → STALE → RETIRED
These states should not imply that every claim in a card has equal evidentiary strength.
Individual claims retain their own status.
A manual claim may remain documentary while a different claim in the same card is locally verified.
Operational lessons follow a separate promotion rule:
- Technical lesson: reproducible observation with version/configuration fingerprint, inputs, outcome, and execution reference.
- Human preference: attributed human assessment, not an objective universal rule.
- Scientific or accessibility claim: identified source, applicability, limitations, and review date.
- Creative recommendation: explicitly subjective and revisable.
Lessons must not be automatically promoted from unverified model summaries.
Conflicting evidence should remain visible rather than being silently replaced.
6. Retrieval policy for Qwen
The motion specifies an 8B local model. The current reported configuration is qwen3:8b, although the live runtime must still be inspected before testing.
My provisional retrieval budget:
| Stage |
Maximum supplementary context |
| Capability decision card |
700 tokens |
| Relevant limitations/lessons |
600 tokens |
| Optional manuals/outcome evidence |
1,000 tokens |
| Total |
2,300 tokens |
These are starting experimental caps, not established optimums.
Use deterministic family/version matching before semantic retrieval.
Initially retrieve no more than three evidence passages. Expand only to resolve a stated uncertainty.
The model should receive evidence labels and concise excerpts, not indiscriminately concatenated documents.
Test whether the model can correctly answer UNKNOWN without manufacturing an explanation.
7. Human Outcome Guide
Keep Human Outcome Guides as referenced documents, not a compulsory runtime subsystem.
Minimum template:
guide_id: audiobook_listening_v1
human_objective: intelligible, engaging narration
audience: target listeners
criteria:
technical_specifications: []
accessibility_requirements: []
research_supported_guidance: []
user_preferences: []
creative_judgments: []
evaluation:
objective_measurements: []
listening_questions: []
rater_count: null
known_disagreements: []
acceptance_rules: []
source_refs: []
review_date: null
Critical distinction: Audio peak level can be measured instrumentally. Emotional authenticity cannot be established by FFprobe.
Human assessment should be captured as a judgment, not converted into a fabricated physical measurement.
The reviewer may also legitimately prefer different styles on different passages.
8. Escalation and authority
The decision policy must preserve existing authorization controls.
Operate independently: Only within current permissions, for explicitly authorized, bounded and appropriately reversible operations.
Investigate: Missing knowledge, outdated evidence, inconclusive limitations, or recoverable technical problems where investigation itself is permitted.
Consult the Roundtable: Material architectural uncertainty, competing methods, difficult evidence interpretation, or high-cost design choices. Roundtable consultation supplies critique, not authority.
Request Darren: Permission expansion, spending, external deployment, publication, irreversible actions, rights decisions, or final creative approval.
For The Buried Sky, the existing manifest-bound smoke-test authorization remains mandatory.
No authorization is inherited from this review.
9. Audiobook pilot and independent acceptance tests
Before constructing the knowledge card, inspect existing broker metadata and identify precisely what is missing.
Use a fixed test suite with three experimental configurations.
Recommended preliminary allocation:
- 12 audiobook capability-routing scenarios.
- 8 unfamiliar transfer scenarios involving Blender or another installed tool family.
- 10 adversarial scenarios involving absent interfaces, stale evidence, denied authority, and conflicting manuals.
Total: 30 distinct tasks, with balanced execution order and independently scored outcomes.
Run identical model settings. Control task wording and context budgets. Do not permit learning from earlier arms to contaminate later arms.
Where feasible, replay each task with multiple seeds or repetitions; 30 single attempts alone provide weak statistical evidence.
The audiobook subset should test selection among native Kokoro controls, text preparation, segmentation, silence insertion, post-processing, compositing, and proposed new capabilities.
Start with non-executing capability decisions. Audio rendering requires separate authorization.
Proposed pre-registered thresholds
| Measure |
Initial acceptance criterion |
| Permission violations |
Zero in all configurations |
| Unsupported control claims |
Zero on predefined critical traps |
| Capability-routing accuracy |
At least 15 percentage points above A |
| Gap classification |
At least 15 percentage points above A |
| Execution correctness |
No material regression |
| Resource overhead |
At most 25% over A |
| Human listening outcome |
Non-inferior to A; improvement measured separately |
| Configuration C vs B |
Demonstrable incremental benefit |
The percentage margins are proposed engineering thresholds, not empirically justified values. Freeze them before trials and report confidence intervals and individual failures.
A zero-critical-failure result over a small suite must not be interpreted as proof of safety.
Separate routing accuracy from realized human benefit. A system can choose the correct tool and still produce an unpleasant audiobook.
For actual audio quality evaluation, compare short, source-identical samples using blinded listeners and a stable rubric covering intelligibility, speaker consistency, pacing, fatigue, and preference.
10. Risk register
| Risk |
Severity |
Mitigation |
| Stale capability claims |
High |
Environment fingerprints and validation |
| Permission confusion |
Critical |
Live broker always authoritative |
| Prompt injection from manuals |
High |
Untrusted evidence isolation |
| Environmental information leakage |
High |
Redaction, access controls, minimal metadata |
| Incorrect promotion of failures |
High |
Reproduction and bounded claims |
| Context overload |
Medium |
Deterministic cards, retrieval caps |
| Subjective judgments treated as facts |
Medium |
Evidence types and attribution |
| Evaluation contamination |
High |
Frozen tasks, blinded scoring |
| Increased complexity without benefit |
High |
Three-arm ablation and rejection rule |
| Misleading audiobook quality claims |
Medium |
Separate measurement and listening review |
11. Deferred or rejected features
Reject initially: New vector databases, duplicate permissions, generalized crawlers, autonomous lesson promotion, comprehensive tool manuals in prompts, and self-certified human satisfaction.
Defer: Cross-tool dependency planners, automated capability synthesis, multi-rater preference modeling, unattended environmental repair, distribution automation, and generalized autonomous tool acquisition.
Retain as unresolved: Actual richness of the existing broker, retrieval reliability of the Blender pack, installed wrapper capabilities, environment fingerprint completeness, and minimum statistically useful test repetitions.
12. Codex-ready staged plan — not authorized
Phase 0: Evidence-only audit. Inspect broker schemas, existing event records, installed tool interfaces, and the current Blender retrieval-verification status. Produce a gap report. No modifications.
Phase 1: Test design. Freeze the task suite, scoring rubric, safety-critical scenarios, and resource limits.
Phase 2: Minimal card prototype. Prepare a single Kokoro family card from authoritative contracts and verified local evidence. Keep source documents external.
Phase 3: Resolver comparison. Compare A against B using controlled, initially non-executing decisions.
Phase 4: Evidence retrieval. Enable bounded TokenSpace references for configuration C and quantify incremental value.
Phase 5: Governed operational pilot. Only after separate execution authorization, compare technical results and human listening judgments.
Phase 6: Decision. Accept the smallest configuration that satisfies pre-registered thresholds, revise the experiment if inconclusive, or reject the overlay if it fails.
No phase is authorized for execution by this motion.
13. Review disposition and remaining dissent
Agreed in principle: Broker authority, compact versioned cards, provenance, negative evidence, bounded retrieval, permission separation, and independent evaluation.
Disputed: Whether manuals meaningfully improve performance beyond a deterministic card; whether creative outcomes can have fixed acceptance criteria; whether an installed-version fingerprint is sufficient; and how much local probing is required before using documented guidance.
Unverified: The actual size of the broker's decision gap, reliability of TokenSpace retrieval for operational guidance, and whether the proposed overhead is justified.
Deferred: Generalization beyond demonstrated pilot needs.
Recommended: Conduct the broker-gap audit and pending Blender retrieval-verification assessment before introducing another subsystem. Neither requires assuming the new architecture is necessary.
Final position
I support Claude's three-arm comparison, DeepSeek's broker-first sidecar, and the core principles of my earlier proposal, subject to the amendment above.
My strongest remaining objection is methodological:
If we improve the broker's descriptions while building the capability card, then compare the improved system against the old broker, we will falsely credit the knowledge layer for an improvement caused by better broker metadata.
The experiment must hold broker functionality constant across arms or explicitly measure that difference.
The architecture should earn its existence through demonstrated benefit.
Recommendation: Adopt the amendment for experimental design only. No implementation authorization is implied.