# `Mutare.Runner.Baseline`
[🔗](https://github.com/foxbenjaminfox/mutare/blob/v0.2.1/lib/mutare/runner/baseline.ex#L1)

Run the complete test suite green-checked against the baseline mutant
(`MUTARE_ACTIVE_MUTANT=0`) — once, several times to catch a flaky suite, or
with retries to survive an occasionally red startup run.

This is the authoritative green check *and* the source of `baseline_ms`. Two
things hang on it being a run of the *whole* suite:

  * **Timing.** `baseline_ms` is the wall-clock of one process boot plus the
    suite, which the runner scales into the per-mutant timeout cap. Summing
    per-file probe runs (as the old conflated probe did) folded N process boots
    into the figure and inflated every mutant's cap. When the suite runs more
    than once (see flakiness, below) we take the **slowest** green run, so the
    cap stays conservative.
  * **Green-ness.** The suite is confirmed green *together*. A per-file probe
    never runs the suite as a whole, so a cross-file dependency could pass
    file-by-file (or fail in isolation) without the suite's real state ever
    being checked.

Mutation testing on a red suite is meaningless — every "kill" is suspect — so a
non-green baseline aborts the run: `{:error, :baseline_failed, output}`. When
`:baseline_retries` is non-zero, an all-red baseline attempt is retried before
aborting; a mixed pass/fail attempt still aborts as flaky immediately. No
`--cover` here: mutant runs don't use it, so an uninstrumented baseline times
the cap against like conditions (and the `--cover` instrumentation belongs to
`Mutare.Runner.CoverageProbe`, which runs separately afterwards).

## Flakiness (`:baseline_runs`)

A flaky test — one that passes/fails nondeterministically *regardless of the
mutant* — is corrosive here: when it goes red during a mutant's run it marks
that mutant killed, a **false kill** that hides a real survivor. Flakiness is a
property of the suite, not of any one mutant, so the cheapest place to catch it
is right here: run the baseline up to `:baseline_runs` times (default 1) and
classify the outcomes (`classify/1`):

  * **all green** → proceed (`{:ok, slowest_green_ms}`);
  * **all red** → `{:error, :baseline_failed, output}` (a deterministically
    broken suite — unchanged from a single run);
  * **mixed** → a test disagreed with itself → `{:error, :baseline_flaky,
    detail}`, aborting loudly with the flaky tests named, rather than scoring
    against a suite that manufactures false kills.

Collection short-circuits the moment outcomes disagree — flakiness is already
proven, so there's no point running the rest. With `:baseline_runs` at its
default of 1 this is exactly the old single-run behavior: one run, green or red.

## Retry survival (`:baseline_retries`)

`:baseline_retries` (default 0) wraps the whole `:baseline_runs` check: if an
attempt is consistently red (`:baseline_failed`), Mutare tries the baseline
again up to that many times. This is for startup/load/order flakes where the
project may be green on a later run. It deliberately does **not** retry
`:baseline_flaky`: when one `:baseline_runs` attempt observes both pass and
fail, the suite has already disagreed with itself and scoring against it would
hide false kills.

# `outcome`

```elixir
@type outcome() :: {:pass, non_neg_integer()} | {:fail, String.t()}
```

# `result`

```elixir
@type result() ::
  {:ok, non_neg_integer()}
  | {:error, :baseline_failed, String.t()}
  | {:error, :baseline_flaky, String.t()}
```

# `classify`

```elixir
@spec classify([outcome()]) :: result()
```

Classify a non-empty list of baseline run outcomes into a run result.

Pure: the I/O lives in `run/2`, the decision (and the flaky message) here, so it
is unit-testable without spawning `mix`. All green → `{:ok, slowest_ms}`; all red
→ `{:error, :baseline_failed, _}`; a mix of both → `{:error, :baseline_flaky, _}`.

# `run`

```elixir
@spec run(Path.t(), pos_integer(), Mutare.Sandbox.Command.Invocation.run_opts()) ::
  result()
```

Run the whole suite up to `runs` times at baseline, then classify (`classify/1`).
Returns `{:ok, baseline_ms}` when consistently green, `{:error, :baseline_failed,
output}` when consistently red, or `{:error, :baseline_flaky, detail}` when the
runs disagree.

`opts` are the run options each run is invoked with
(`t:Mutare.Sandbox.Command.Invocation.run_opts/0`): a fixed `:partition` entry
(e.g. `MIX_TEST_PARTITION=1`) when `:partition_env` is on, so a partitioned suite
finds a valid database, and the `:max_heap_mb` cap; `[]` (the default) sets none.
The baseline is sequential, so one fixed partition suffices
(`Mutare.Runner.Partitions`).

# `run`

```elixir
@spec run(
  Path.t(),
  pos_integer(),
  non_neg_integer(),
  Mutare.Sandbox.Command.Invocation.run_opts()
) ::
  result()
```

Run the baseline check with `retries` extra all-red attempts before returning
`:baseline_failed`.

A retry wraps the full `runs` sample. `:baseline_flaky` is never retried because
a mixed pass/fail sample has already proven suite nondeterminism.

---

*Consult [api-reference.md](api-reference.md) for complete listing*