Deep-Dive DD-13 — OpenHarness (HKUDS): Academic Baseline

         ┌──────────────────────────────────────────┐
         │              The Loop                     │
         │  (calls model; dispatches tools;          │
         │   observes stop conditions)               │
         └──────────────┬───────────────────────────┘
                        │
        ┌───────────────┼───────────────┬───────────────┐
        ▼               ▼               ▼               ▼
   Tool Registry   Context Mgr      Memory         Decision Log
   (small,         (auto-           (explicit,      (transparent,
    stable set)     compaction)      tiered)         first-class)

# Deep-Dive DD-13 — OpenHarness (HKUDS): Academic Baseline

**Course**: Master Course · **Deep-Dive**: DD-13 · **Duration**: 45 min · **Prerequisites**: Modules 0–12, DD-01–12

> *Built for research reproducibility and inspectability. Auto-compaction design. The clean-room academic comparison baseline.*

---

## The Subject

OpenHarness (from the HKUDS lab, April 2026) is the **academic harness** — built not for production use but for research. The question it answers is not "can I ship this?" but "can a paper make a defensible claim against this harness?" Every component is modular, every decision is traceable, every mechanism is examinable in isolation. It is the clean-room baseline against which production harnesses are compared.

| Metric | Value |
| --- | --- |
| Origin | HKUDS lab (academic), April 2026 |
| Purpose | Research reproducibility and inspectability |
| Tool count | Small and stable — a research-minimal set, not a production surface |
| Defining mechanism | Auto-compaction (research-grade, fully visible) |
| Memory | Designed and explicit; not an afterthought |
| Sandbox / permissions | Minimal — not the research question |
| Observability | Inspectability is the *product*, not a feature |

What makes OpenHarness "academic" is not that it is weak — it is that it optimizes for a different objective function. A production harness optimizes for capability under constraints (cost, latency, safety). OpenHarness optimizes for **inspectability under reproducibility**: can a researcher point at exactly the mechanism that produced this decision, vary it, and measure the effect? If the answer is ever "it's buried in the framework," the harness has failed its purpose.

## What "Academic" Means Here

Three properties define the academic posture, and each is a deliberate architectural choice rather than a missing feature:

1. **Inspectability** — every component (loop, tools, context, memory) is modular and individually examinable. You can study the compaction mechanism without the tool layer, the memory model without the loop, the loop without the memory. Module 10 (observability) frames inspectability as a property of *what you can see*; OpenHarness treats it as a property of *what you can isolate*. The two are related but distinct — a production harness may emit rich telemetry about a black box; OpenHarness makes the box itself open.
2. **Auto-compaction visibility** — the compaction mechanism is implemented as a first-class, named, parameterized component rather than an emergent behavior of the loop. You can see *when* it triggers, *what* it summarizes, *what* it discards, and *what* it preserves. This is the reference implementation Module 3 points at when it teaches compaction — because in a production harness the compaction is usually entangled with the session store, the provider's token accounting, and the framework's retry logic.
3. **Transparent decisions** — every harness decision (tool selection, stop condition, context pruning) is logged at the point it is made, with the inputs that produced it. Not as an after-the-fact trace reconstruction, but as a first-class record. This is the difference between "we can reproduce the run" and "we can defend why the run went the way it did."

## Architecture

OpenHarness is structured so that each Module 0.1 job (loop, tools, safety, context, memory) is a separate, swappable component with a documented interface:

```
         ┌──────────────────────────────────────────┐
         │              The Loop                     │
         │  (calls model; dispatches tools;          │
         │   observes stop conditions)               │
         └──────────────┬───────────────────────────┘
                        │
        ┌───────────────┼───────────────┬───────────────┐
        ▼               ▼               ▼               ▼
   Tool Registry   Context Mgr      Memory         Decision Log
   (small,         (auto-           (explicit,      (transparent,
    stable set)     compaction)      tiered)         first-class)
```

The built-in personal agent, **ohmo**, is the reference demonstration — a working agent that exercises every component so a researcher can run it, vary a parameter (compaction threshold, memory tier, tool set), and measure the effect on a benchmark without rebuilding the harness. ohmo is to OpenHarness what Pi (DD-01) is to the minimal-harness category: the readable instance that proves the architecture.

### The auto-compaction implementation (Module 3 reference)

This is OpenHarness's headline contribution and the reason Module 3 cites it. Compaction in most production harnesses is a recovery behavior — something the harness does *because it has to* when the context window fills. In OpenHarness, compaction is a *designed mechanism* with four visible parts:

1. **A trigger predicate** — a declared condition (token threshold, message count, or a custom predicate) that decides *when* compaction fires. Not magic, not buried — a function you can read.
2. **A selection policy** — which messages/tool results get summarized, which get preserved verbatim, which get dropped. The policy is named and parameterized, not implicit.
3. **A summarization step** — the actual summarization call (model-based) with its own logged inputs and outputs, so you can audit *what the summarizer was shown* and *what it produced*.
4. **A non-destructive record** — the original context is retained for analysis even after the compacted version is what the model sees on the next turn. This mirrors DD-21 (Tau)'s `CompactionEntry` with `replaces_entry_ids`: the on-disk truth is never rewritten, only the in-context view is swapped.

Because all four parts are individually examinable, a researcher can ask questions a production harness cannot answer without instrumentation: "did compaction change the outcome?", "would a different selection policy have preserved the critical instruction?", "is the summarizer introducing drift?" Module 3's compaction unit points here precisely because the mechanism is studiable rather than merely present.

### Memory model

OpenHarness treats memory as an explicit, tiered abstraction rather than an emergent property of the context window. The tiers map onto Module 4's vocabulary:

- **Working memory** — the current context window, post-compaction.
- **Episodic memory** — prior turns/sessions, retrievable but not in-context by default.
- **Semantic memory** — distilled facts the agent has learned, keyed for retrieval.

The architectural choice that distinguishes OpenHarness from production harnesses is that these tiers are *separate stores with declared interfaces*, not layers of the same buffer. This makes the read path and the write path individually auditable — a property Module 4.3 (write-gating) wants but that production harnesses usually blur. The cost is integration friction: a production harness conflates these tiers precisely because doing so is faster and cheaper. OpenHarness pays the friction to buy the inspectability.

### Tool count

The tool set is research-minimal: small, stable, and chosen so that capability claims can be attributed to the model rather than to a rich tool surface. This is the opposite of a production harness's incentive (more tools = more capability = better benchmarks). For research, a large tool set is noise — it confounds the question "what did the model do?" with "what did the tools enable?" OpenHarness keeps the surface small so the model's decisions are the signal.

## Phase 3 — Design Decision Audit (selected)

| Module | Pattern | Tradeoff accepted |
| --- | --- | --- |
| 1 Execution Loop | Standard, but fully logged at decision points | Slightly more overhead; far more auditable |
| 3 Context Mgmt | Designed auto-compaction, non-destructive | More code than an emergent compaction; the point |
| 4 Memory | Explicit tiered stores with declared interfaces | Integration friction vs inspectability |
| 5 Sandboxing | Minimal — not the research question | Not safe to ship |
| 6 Permission | Minimal | Same |
| 10 Observability | Inspectability-as-product; decisions logged at source | The defining strength |

**Three decisions I agree with:**
1. Compaction as a designed, named, four-part mechanism — this is the reference the course points at, and deservedly so.
2. Non-destructive compaction record — the original context is never overwritten, only the in-context view. Matches DD-21's design and Module 8's event-sourcing principle.
3. Memory as separate declared stores — buys the inspectability that justifies the friction.

**Three decisions I would make differently — but with the caveat that OpenHarness is not trying to be production:**
1. The sandbox gap is real for anyone who misuses this as a production harness; document the boundary loudly.
2. The permission model is below floor — fine for research, fatal if shipped.
3. Structured per-turn telemetry would make the inspectability machine-readable, not just human-readable.

## Score: 30/60

| Module | Score | Notes |
|---|---|---|
| M1 Loop | 3/5 | Standard, decision-logged; not the contribution |
| M2 Tools | 2/5 | Deliberately small; correct for research, low capability |
| M3 Context | 4/5 | Auto-compaction is research-grade and inspectable — the reference |
| M4 Memory | 4/5 | Explicit tiered stores; clean read/write separation |
| M5 Sandbox | 1/5 | Minimal — not the research question |
| M6 Permission | 2/5 | Minimal |
| M7 Errors | 2/5 | Basic; not the focus |
| M8 State | 3/5 | Non-destructive record present; not as complete as DD-21's session tree |
| M9 Verification | 1/5 | None |
| M10 Observability | 4/5 | Inspectability is the product; below 5 only because it's human-readable over machine-readable |
| M11 Security | 1/5 | Not the research question |

Note: as with DD-01 (Pi), the absolute score is low because the rubric scores *production-readiness across all dimensions*. OpenHarness is not trying to be production-grade across all dimensions — it is deliberately inspectable. **The score reflects what OpenHarness is not (production-ready), not a failure of what it is (the cleanest mechanism reference in the roster).** Its value is pedagogical and methodological: it is the harness that makes every other harness's claims falsifiable.

### Architect's Verdict
> *OpenHarness optimizes for research reproducibility and inspectability — the cleanest implementation to study, compare against, and make academic claims about, with auto-compaction as a designed four-part mechanism rather than an emergent recovery behavior. It is not production-ready and does not try to be: sandboxing, permissions, and the production observability stack are absent because they are not the research question. Build on it for research and for studying mechanisms in isolation; do not ship it. Its role in the roster is methodological — it is the academic baseline that gives every production harness a defensible comparison reference, and the harness Module 3 points at when it teaches compaction.*

### MLSecOps Relevance
> *OpenHarness's inspectability is its security value: every component can be audited in isolation, making it the best harness for studying HOW a specific mechanism (compaction, memory, context, decision) works without production noise obscuring the signal. For MLSecOps specifically, it is the lab target for reasoning about compaction-induced drift (does the summarizer drop a security-critical instruction?) and memory-tier poisoning (can a write to semantic memory survive compaction?) — questions you can only answer against a harness where the mechanism is visible.*

### 3 things OpenHarness does better
1. **Compaction as a designed mechanism** — the four-part trigger/select/summarize/record design is the reference implementation Module 3 teaches against. No other harness in the roster makes compaction this inspectable.
2. **Inspectability-as-product** — every decision logged at source with its inputs. The closest peer is DD-21 (Tau)'s event union, but Tau optimizes for legibility-of-code while OpenHarness optimizes for legibility-of-behavior.
3. **Non-destructive context record** — the original context survives compaction for analysis, mirroring DD-21's `replaces_entry_ids` design and Module 8's event-sourcing principle.

### 3 things to fix (if you misuse it as production)
1. **Add a sandbox** — bash/exec run without containment; correct for research, fatal if shipped.
2. **Add a permission model** — below floor; fine in the lab, not elsewhere.
3. **Emit machine-readable decision logs** — the inspectability is currently human-readable; a structured schema would make it analyzable at scale and bring Module 10 observability up to the DD-21/DD-14 event standard.

---

## References
1. **OpenHarness (HKUDS)** — the academic baseline; the clean-room comparison reference.
2. **Module 3** — context management and compaction; OpenHarness's auto-compaction is the reference implementation.
3. **Module 4 / 4.3** — memory tiers and write-gating; OpenHarness's explicit tiered stores with declared interfaces.
4. **Module 8** — state/checkpointing; the non-destructive compaction record echoes event-sourcing.
5. **Module 10** — observability and inspectability; OpenHarness treats inspectability as the product, not a feature.
6. **Module 0.3** — the 6-phase methodology; OpenHarness is the easiest harness to apply it to (fully inspectable).
7. **DD-01 (Pi)** — shared pattern: a low absolute score that reflects "not trying to be production," not a failure of the design.
8. **DD-21 (Tau)** — the closest peer on inspectability; Tau optimizes for legibility-of-code, OpenHarness for legibility-of-behavior. The non-destructive compaction record is a shared design.
9. **DD-14 (Mastra)** — the machine-readable observability standard OpenHarness's human-readable logs approach but don't meet.