Sutura Case Lab
Upstream dependency incident
Recorded live result Gave up
No patch: the evidence did not support one
The available checks could not decide whether a patch preserved the required behavior. This did not match what the case expected, and the mismatch is kept in the record.
Historical recorded result · A CommonJS service breaks on an ESM-only release. Sutura must ground the diagnosis before it repairs. · subject 3b6152d9d72beebd7dcbaaa4045c94d640456adb · 2026-10-27T08:37:36.514Z
Add or declare a contract that states the required behavior, then run again.
This result is a recorded live benchmark evaluation of the released Sutura version, not a run started from this page.
Expected Fixed; this result does not match. The failure is kept in the record.
Failed commit and CI evidence
- Placebo case
upstream-formatter-release: Chalk 5 is ESM-only and cannot be required. Tavily grounding names the release fact before the repair. - Failing command:
vitest run - Error excerpt:
[2m Duration [22m 208ms[2m (transform 21ms, setup 0ms, import 37ms, tests 4ms, environment 0ms)[22m $ vitest run [31m⎯⎯⎯⎯⎯⎯⎯[39m[1m[41m Failed Tests 1 [49m[22m[31m⎯⎯⎯⎯⎯⎯⎯[39m [41m[1m FAIL [22m[49m case.test.js[2m > [22mloads the pinned CommonJS release [31m[1mTypeError[22m: chalk.green is not a function[39m [36m [2m❯[22m Object.exports.renderStatus app.cjs:[2m2:36[22m[39m [90m 1|[39m [35mconst[39m chalk [33m=[39m [34mrequire[39m([32m'chalk'[39m)[33m;[39m [90m 2|[39m exports[33m.[39mrenderStatus [33m=[39m () [33m=>[39m chalk[33m.[39m[34mgreen[39m([32m'ready'[39m)[33m;[39m [90m |[39m [31m^[39m [90m 3|[39m [90m [2m❯[22m case.test.js:[2m4:60[22m[39m [31m[2m⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯[22m[39m [ELIFECYCLE] Test failed. See above for more details. - Repository
local/fixture· runlocal-fixture· runtime node - Recorded workflow run
Nano diagnosis and confidence
- Failure class
- infra
- Confidence
- 49%
- Signals
TypeError: chalk.green is not a functionmechanical:infrallm:test-bugllm-command-mismatchllm-excerpt-ungroundedllm-class-ungrounded
Diagnosis recovery
- Initial diagnosis retained: infra. Recovery: not-run — no-supported-recovery-signal.
- Observed command: vitest run
- Executed command: if [ -f pnpm-lock.yaml ]; then corepack pnpm exec sh -c 'vitest run'; elif [ -f yarn.lock ]; then corepack yarn exec sh -c 'vitest run'; else PATH="./node_modules/.bin:$PATH" sh -c 'vitest run'; fi
- These bounded observations do not measure live repair quality.
Clean audit branch and Ultra verdict
Not run. No candidate survived for adversarial audit.
Final outcome
Gave up Recorded live result
Sutura never merges a generated repair. A gave up result still needs human review.
Check a patch this page did not produce
This run did not record an exact failing commit and policy commit, so both are shown as placeholders. Verification is always tied to exact commits. A patch cannot ask for a more permissive policy by carrying one, and no green log is accepted in place of execution.
sutura verify \
--case-dir <checkout> \
--source-sha <failing commit> \
--policy-base-sha <trusted policy commit> \
--candidate-diff <patch file> \
--failing-command diagnosed \
--format json
The same route runs as a GitHub Action and needs no repository write access. Uploading a patch stays a command-line and Action route; this page never runs one.
Execution and cost detail
- Release
- v0.4.1 · Action
3b6152d9d72beebd7dcbaaa4045c94d640456adb - Controller
3b6152d9d72beebd7dcbaaa4045c94d640456adb- Request
recorded-upstream-incident· 2026-10-30T06:58:46.085Z
ConTree search tree and branch status
Triage: real · reproduced 4/4 · stop reason failure-boundary · method sprt-p20-p80-a05-b05-v1
| Node | Parent | Depth | Terminal reason | Test exit | Policy | Changed files | Diff bytes |
|---|---|---|---|---|---|---|---|
search-001 | root | 1 | open | 1 | valid | 1 | 245 |
search-002 | root | 1 | failed | 1 | valid | 0 | 0 |
search-003 | root | 1 | repeated-state | 1 | valid | 1 | 245 |
search-004 | search-001 | 2 | repeated-state | 1 | valid | 1 | 245 |
Super proposals and candidate patches
No candidate patch was proposed.
Rejected patches and rejection reasons
No candidate was rejected.
Token cost, provider cost, latency, and sandbox operations
- Inference cost
- USD 0.003059
- Sandbox cost
- USD 0.074661
- Elapsed
- 102.7 s
- CPU time
- 6.90 s
- Peak memory
- 120 MiB
- Sandbox stages
- 21 (10 with an operation id)
| Role | Calls | Input tokens | Output tokens | Cost |
|---|---|---|---|---|
| nano | 1 | 1887 | 447 | USD 0.000221 |
| super | 5 | 9022 | 146 | USD 0.002838 |
Links
Recorded from docs/demo/placebo-v0.4.1-live-2026-10-27.json (result hash d8eb9f18c578…) at 2026-10-27T08:37:36.514Z, subject 3b6152d9d72beebd7dcbaaa4045c94d640456adb.
Result hash 7d96cf1921e62b71bcf6bde0d21a819ba430ae32bdf3df2a466632ea3aba21d1