Phase 4.0: Cache Tuning Checkpoint

Status

Phase 4.0 checkpoint complete: no runtime cache change proposed.

Final measured decision: #123's production Agent Console workload does not justify a cache change. This is not a claim of global optimality.

---

Core Decision

The current synthetic measurements do not provide sufficient evidence to justify changing the per-style Cell cache capacity or eviction policy.

Therefore:

• Keep MAX_CACHED_CELLS_PER_STYLE = 128 (unchanged)
• Keep the current clear-all eviction policy (unchanged)
• Do not introduce LRU/LFU at this time

Important: This is a status-quo decision based on insufficient evidence for change, not validation that the current cache is globally optimal.

---

What Was Measured

Data source: 8 synthetic profiler workloads from Phase 3.2 instrumentation

Workloads:

1. Cell cache overflow (width=1) - stress test
2. Cell cache overflow (width=2) - stress test
3. Many styles - stress test
4. Complex grapheme cached - 4 unique strings, 500 repetitions
5. Complex grapheme uncached - 1000 unique strings
6. Wrap width churn - 51 widths (stress test)
7. Repeated CJK - stable content
8. Mixed workload - fresh Style object per line

Metrics collected:

• createCell calls and cache hit rates
• Bucket distribution (P50/P95/Max per width)
• Cache clear counts
• Text cache hit rates
• Registered bucket counts (instrumentation cumulative)

---

Evaluation Results

Per-Style Cell Capacity (MAX_CACHED_CELLS_PER_STYLE)

Decision: No change justified by current synthetic evidence.

Observations:

• Repeated CJK: 99.98% hit rate, 0 clears, bucket size=10
• Mixed workload: 37.49% hit rate, 0 clears, bucket P95=10/6

Limitations:

• Mixed workload creates fresh Style object per line → 1000 small buckets
• Does not validate MAX=128 for reused/default styles
• No workload tests stable style with >128 unique characters
• No second-pass evaluation after clear

Conclusion: The repeated-CJK and mixed workloads did not overflow their per-style buckets. The two dedicated overflow stress workloads did trigger one clear each, as intended. None of these workloads validates behavior under eviction-sensitive reuse.

---

Clear-All vs Partial Eviction

Decision: Inconclusive; no eviction-sensitive reuse workload.

Observations:

• Overflow workloads trigger clear counter
• Single-pass unique scans (no post-clear reuse)

Limitations:

• Cannot evaluate clear-all vs LRU/LFU effectiveness
• No workload with hot-set reuse near capacity threshold

Conclusion: No evidence that partial eviction would provide benefit, but also no data showing clear-all is acceptable under working-set pressure.

---

Text Cache Capacity

Decision: Inconclusive; capacity threshold not exercised.

Observations:

• Cached: 99.80% hit rate (4 unique keys, 500 repetitions)
• Uncached: 1000 unique strings

Limitations:

• 1000 < MAX_TEXT_WIDTH_CACHE=1024 (no eviction)
• Capacity limit not reached
• Long text admission not tested

Conclusion: Cannot validate current capacity or eviction behavior.

---

Wrap Cache

Decision: Per-width capacity not measured; outer bucket instrumentation validated.

Observations:

• Width bucket clear triggered at 51 widths (exceeds MAX=32)

Limitations:

• Only exercises outer width-bucket limit
• Does not test MAX_WRAP_CACHE_PER_WIDTH=256
• Each width has only one text entry
• No unique vs repeated long wrapped text

Conclusion: Instrumentation functions correctly; production relevance unknown.

---

Grapheme Optimization

Decision: Not measured with decision-grade timing.

Observations:

• Counter snapshots confirm cached and unique workloads exercised expected segmentation paths
• Historical single-shot duration values intentionally excluded (not suitable for performance decisions)

Limitations:

• Phase 2 complex-grapheme unique scenario has CV=177.27% (unstable)
• No isolated segmentation benchmark with warmup and statistical samples
• Single-run end-to-end timing includes string construction and cache operations

Conclusion: No decision-grade evidence that segmentation is or is not a bottleneck. No optimization implemented.

---

Long Text Admission

Decision: Not measured.

Status: No workload provided.

---

inlineLineCacheByWidth

Decision: Not measured.

Status: Not instrumented in Phase 3.2.

---

Style Cardinality / Live Buckets

Decision: Not measured by current strong-reference registry.

Observations:

• Mixed workload: 2000 registered buckets (cumulative instrumentation)

Limitations:

• Registered buckets are strongly retained by instrumentation
• Not equivalent to live Style count in production
• Cannot infer production style-cardinality or retained memory

Conclusion: Registry count is useful for workload characterization but does not measure production behavior.

---

Measurement Limitations

Heap Measurements

Issue: heapBefore captured before resetMetrics(), which releases previous workload's bucket registry. Heap deltas include memory from prior workload cleanup.

Impact: Heap delta values are invalid for retained-memory analysis and optimization decisions. They are preserved in the annotated historical transcript only as audit data and must not be used.

Status: Heap-based conclusions removed from this report.

---

Instrumentation Disabled-Path Overhead

Issue: The "without instrumentation" arm disables metric collection, but still executes the instrumentation hook functions added in Phase 3.

Impact: This benchmark does not measure the production cost of adding the instrumentation foundation itself (pre-Phase-3 vs post-Phase-3).

Additional: The fixed disabled-first/enabled-second order makes the single-run timing unsuitable for overhead estimates. Negative overhead values are measurement noise.

---

Workload Coverage

Gaps:

• No reused/default style with high character diversity
• No post-clear second-pass evaluation
• No capacity-threshold testing (text/wrap)
• No long text pollution testing
• No production traces or long-running sessions

Synthetic only: All workloads are profiler scenarios, not production patterns.

---

What This Report Does NOT Claim

NOT supported by current data:

• ❌ "Current cache implementation is validated"
• ❌ "Current cache performs well" (globally)
• ❌ "90%+ safety margin" for reused styles
• ❌ "Segmentation is cheap / not a hotspot"
• ❌ "Text cache sizing is adequate"
• ❌ "Wrap cache strategy is validated"
• ❌ "All optimization gates have failed"
• ❌ "Cache performance work is complete"

---

Completed Follow-up: Production Instrumentation Overhead

#122 compiled instrumentation out of standard ESM/CJS artifacts and validated built-dist plus packed-consumer A/B/C. #119 is closed. #123 then profiled the production Agent Console workload and found no measured justification for cache, long-text, provider, renderer-architecture, or virtual-scroll changes.

---

Optional Follow-up: Targeted Cache Workload Coverage

Phase 4.1 is optional only if cache optimization is explicitly deferred.

It is required before:

• Claiming the current cache strategy is validated
• Changing cache capacity or eviction
• Closing cache-performance evaluation as complete

Phase 4.1: Targeted Workload Coverage (If Pursued)

Before claiming comprehensive cache validation, add:

1. Stable/reused style workload: DEFAULT_STYLE or frozen style objects, working set near or exceeding 128 unique characters, multiple passes

2. Eviction-sensitive workload: Insert 128 hot + periodic cold characters, observe clear frequency and hit-rate impact

3. Text cache capacity: >1024 unique text entries, observe eviction and post-eviction hit rates

4. Wrap per-width capacity: >256 unique wrapped strings per width

5. Long text scenarios: Unique vs repeated long non-ASCII text through textCellWidth, wrapByCells, formatInlineCellLine

6. Realistic workload replay: Offline replay of sanitized terminal/session traces

   Important: Do not enable the current strong-reference bucket registry for an unbounded production session. The instrumentation retains all registered buckets until resetMetrics(), which will alter GC and retained-memory behavior.

   Recommended approaches:

   • Bounded capture windows
   • resetMetrics() for counter-only capture windows (not for bucket-distribution profiling, as existing buckets do not re-register)
   • Maximum registered-bucket sampling
   • Offline trace replay (preferred)
   • For bucket-distribution profiling: isolated processes, fresh Style objects per capture, or redesigned bounded/weak registry

---

Conclusion

Phase 4.0 serves as a checkpoint, not comprehensive validation.

Engineering decision: Keep current implementation unchanged because current measurements do not provide sufficient evidence to justify modification.

Not claimed: Current cache is proven optimal or adequate for all scenarios.

Next step: Either proceed with other work, or optionally add Phase 4.1 targeted workloads before closing cache evaluation.

---

Appendix: Data Source

Profiler run:

• Command: pnpm run bench:profiler:complete:gc
• Node: v24.18.0
• V8: 13.6.233.17-node.50
• Platform: darwin (macOS)
• Arch: arm64
• CPU: Apple M1 Pro
• GC: enabled
• Date: 2026-07-10
• Code commit: c4182b6c2f449423739851c335feb5932f9d5b40

Note: Profiler data collected at c4182b6c. Workload definitions and counter-collection logic used by this report have not changed since that data commit. Later commits removed invalid timing/heap/control-arm measurements from the profiler tool and updated documentation and output labeling.

Output: docs/perf/phase4-profiler-output-c4182b6c-annotated.txt (annotated historical transcript)

Production workload follow-up

PR #123 completed the Agent Console production workload profile. It found and contained replay-history publication and hidden eager Markdown-block publication hotspots, but supplied no evidence for changing Cell cache capacity, eviction, text/wrap caches, provider caches, or virtual scrolling. Phase 4.0 remains a no-change decision for those areas.