Open Kioku

Home › See it work

See it work

A real commit, stopped halfway.

A commit to Black, the Python formatter, gave one function a new required parameter and updated every caller. We stop that commit halfway, with the function changed and the callers not yet, and ask Open Kioku what is broken. The finished commit is the answer key.

6 of 6call sites it named are lines the real commit went on to change
0.2 sto name them, on a repository of 504 files
0builds, test runs or type checkers involved
26 sfor one recorded Claude Code session to make the change and leave nothing broken
1 · The change

One function gets a required parameter.

Python imports this file without complaint. Each call that still passes one argument fails only when it runs.

src/black/cache.py
-def get_cache_file(mode: Mode) -> Path:
-    return CACHE_DIR / f"cache.{mode.get_cache_key()}.pickle"
+def get_cache_file(mode: Mode, cache_dir: Path) -> Path:
+    return cache_dir / f"cache.{mode.get_cache_key()}.pickle"

psf/black 1ed66a8b33, “Add --cache-dir command-line option” ↗

2 · Ask what it broke

Ripple names every call the change left behind.

It reads the working tree against the last commit and the index of the repository. Nothing is built and nothing is run.

psf/black · ok ripple · recorded output, 0.2 s
$ ok ripple
Ripple: 6 site(s) no longer fit 1 changed declaration(s).

src/black/cache.py:52  function `get_cache_file`: signature changed
  - def get_cache_file(mode: Mode)
  + def get_cache_file(mode: Mode, cache_dir: Path)
  broken: 6 site(s)
    tests/test_black.py:2663 (in TestCaching.test_cache_file_length)  cache_file = get_cache_file(case)
        does not pass `cache_dir`, which is required now [proven]
    tests/test_black.py:2673 (in TestCaching.test_cache_file_path_ignores_python_cell_magic_separators)  cache_file = get_cache_file(mode)
        does not pass `cache_dir`, which is required now [proven]
    tests/test_black.py:2682 (in TestCaching.test_cache_broken_file)  cache_file = get_cache_file(mode)
        does not pass `cache_dir`, which is required now [proven]
    tests/test_black.py:2694 (in TestCaching.test_cache_empty_file)  cache_file = get_cache_file(mode)
        does not pass `cache_dir`, which is required now [proven]
    tests/test_black.py:2743 (in TestCaching.test_no_cache_when_writeback_diff)  cache_file = get_cache_file(mode)
        does not pass `cache_dir`, which is required now [proven]
    tests/test_black.py:2773 (in TestCaching.test_no_cache_when_stdin)  cache_file = get_cache_file(mode)
        does not pass `cache_dir`, which is required now [proven]
  1 other call(s) fit the new signature

2 changed declaration(s) with nothing to act on:
  src/black/cache.py:31  `get_cache_dir`: only optional parameters added
  src/black/cache.py:63  `Cache.read`: only optional parameters added

proven means the index resolved the call to this declaration. How a site is attributed ↗

3 · The answer key

Then compare it with what the authors did.

The finished commit changed each of the 6 lines ripple named.

Site ripple namedIn the finished commit
tests/test_black.py:2663changed by the commit
tests/test_black.py:2673changed by the commit
tests/test_black.py:2682changed by the commit
tests/test_black.py:2694changed by the commit
tests/test_black.py:2743changed by the commit
tests/test_black.py:2773changed by the commit

The comparison is made by the replay script ↗ from git diff of the commit. The same replay on 60 repositories ↗

4 · In a pull request

The same finding, where the review happens.

ok gate ci judges any pull request, whatever wrote it, and writes a Change Dossier: what the task declared, what the change touched, and what it left broken.

In CI
ok index .
ok gate ci --base origin/main \
  --task "$PR_TITLE" --write dossier.md

The whole dossier for this change ↗ · The gate ↗

Change Dossier · Ripple

6 site(s) no longer fit what this change altered (3 changed declaration(s); 1 other call(s) fit).

  • tests/test_black.py:2663 does not pass cache_dir, which is required now — src/black/cache.py: get_cache_file [proven]
  • tests/test_black.py:2673 does not pass cache_dir, which is required now — src/black/cache.py: get_cache_file [proven]
  • tests/test_black.py:2682 does not pass cache_dir, which is required now — src/black/cache.py: get_cache_file [proven]
  • tests/test_black.py:2694 does not pass cache_dir, which is required now — src/black/cache.py: get_cache_file [proven]
  • tests/test_black.py:2743 does not pass cache_dir, which is required now — src/black/cache.py: get_cache_file [proven]
  • tests/test_black.py:2773 does not pass cache_dir, which is required now — src/black/cache.py: get_cache_file [proven]
5 · In Claude Code

With the hooks, the agent hears it right after the edit.

One recorded session on the same repository, before the commit. The prompt asks for the parameter and does not mention callers.

  1. The prompt

    In `src/black/cache.py`, give `get_cache_file` a second required parameter, `cache_dir: Path`, and build the cache file path under it instead of calling `get_cache_dir()` inside the function.
  2. First edit: the signature

    The hook answers with the 7 calls that no longer fit: the six in the tests and one in the same file.

    Claude Code · after the edit · recorded
    Open Kioku: after this edit, 7 call site(s) elsewhere look as if they no longer fit what it changed.
    src/black/cache.py:51  function `get_cache_file`: signature changed
      was: def get_cache_file(mode: Mode)
      now: def get_cache_file(mode: Mode, cache_dir: Path)
      src/black/cache.py:68 (in Cache.read)  cache_file = get_cache_file(mode)
            does not pass `cache_dir`, which is required now [proven]
      tests/test_black.py:2663 (in TestCaching.test_cache_file_length)  cache_file = get_cache_file(case)
            does not pass `cache_dir`, which is required now [proven]
      tests/test_black.py:2673 (in TestCaching.test_cache_file_path_ignores_python_cell_magic_separators)  cache_file = get_cache_file(mode)
            does not pass `cache_dir`, which is required now [proven]
      tests/test_black.py:2682 (in TestCaching.test_cache_broken_file)  cache_file = get_cache_file(mode)
            does not pass `cache_dir`, which is required now [proven]
      tests/test_black.py:2694 (in TestCaching.test_cache_empty_file)  cache_file = get_cache_file(mode)
            does not pass `cache_dir`, which is required now [proven]
      tests/test_black.py:2743 (in TestCaching.test_no_cache_when_writeback_diff)  cache_file = get_cache_file(mode)
            does not pass `cache_dir`, which is required now [proven]
      tests/test_black.py:2773 (in TestCaching.test_no_cache_when_stdin)  cache_file = get_cache_file(mode)
            does not pass `cache_dir`, which is required now [proven]
    Check each one before you finish: update it, or leave it if it is right as it stands.
  3. Second edit: the call in the same file

    6 are left, and the hook says so again.

  4. The tests

    The agent updates the six calls. The next 2 edits get no notice: nothing is left.

  5. At the stop: the Change Dossier

    • Boundary: every changed file is inside the declared boundary.
    • 1 changed declaration(s), 1 a call could break on; no broken site found (7 call(s) checked and fit).

11 turns, 26 seconds, recorded 2026-11-14. Every step, both notices and the dossier ↗

Run it yourself

One script, the same output.

It clones Black into a temporary directory, prints steps one to four, and removes the clone. It needs git, python3 and ok 5.0.0 or later.

npm install -g open-kioku
git clone https://github.com/shivyadavus/open-kioku
open-kioku/scripts/demo-replay.sh

This repository's CI runs the script on every pull request and fails if the output differs from what this page shows.

Or by hand

Five commands.

The same replay, typed out.

git clone https://github.com/psf/black && cd black
git checkout 1ed66a8b33~1
ok index
git checkout 1ed66a8b33 -- src/black/cache.py
ok ripple