Mutare.Report.Events (mutare v0.5.1)

Copy Markdown View Source

The events of mix mutare --events FILE: one JSON object per line, appended as the run goes, so a consumer can follow a run with tail -f and filter it a line at a time (jq -c 'select(.status == "survived")'). The JSON report is the complete record of a finished run; this is the record of one in progress.

Every event has an event name and elapsed_ms, the milliseconds since the run started. In order:

  • start — the run has started, and its scan with it. version is this format's version (1), mutare the Mutare version.
  • scanned — the scan is done: mutants will be tested, from files files.
  • phase — the run entered phase: compiling, baseline, coverage_probe, running (with mutants, the number it will test), confirming_timeouts (with mutants, the number of provisional timeouts to re-run one at a time), and, under --partition-env, checking_partitions (with partitions, the number whose environment it re-runs one kill's tests in with no mutant active).
  • mutant — one mutant's verdict, written once the run accepts it. The run tests several mutants at once but accepts their verdicts in source order, so a slow mutant holds back the lines of mutants after it that have already finished: a quiet file does not mean no mutant has finished. A provisional timeout is re-run alone after the others (confirming_timeouts), so its line comes at the end. Fields:
    • evaluated — how many mutant lines the file holds, this one included: against the scanned count, how far the run has got;
    • id — the report id, an integer: the JSON report's id, which that report writes as a string;
    • file, line, column — where the mutant is: the location the human report prints, --line selects by, and # mutare:ignore reads;
    • mutator, variant — the family, and the variant labels its # mutare:ignore[family:label] filters match (a list, empty when the family declares none);
    • status — one of killed, timeout, atom_exhausted, survived, no_coverage, ignored, poisoned, harness_error;
    • range, original, replacement — the patch: original is the source text between range.start and range.end (1-based, end exclusive), and replacement the text that takes its place, exactly as the human report's diff applies it. A replacement may cover more than it changes: removing a pipe stage rewrites the whole pipe, while line and column point at the stage. A column, here and in column, counts graphemes, as Elixir's columns do — not bytes, code points or UTF-16 units — so on a line with non-ASCII text, slice by graphemes or find original on the line;
    • duration_ms, selection — how long the mutant's test run took, and which tests it ran (tests, files, app or suite); absent for a mutant that launched no run (one no_coverage, ignored or poisoned);
    • note, reason — an advisory note from the mutator, and why a mutant was ignored or why its run reached no verdict; each absent when there is none.
  • finish — the last line, written once the reports are. stopped says why: complete, max_survivors, time_budget, sigterm, or error. All but error carry mutants (the number the run set out to test; null if a SIGTERM stopped the scan), evaluated, counts (every status, zeros included) and score, over the mutants evaluated. An error carries error, the reason's name (baseline_failed, compile_failed, …, and aborted for an abort with no reason of its own, such as a bad # mutare:ignore qualifier or --strict-ignores), and message, the text mix mutare prints. An error writes no final report, so a report file holds what it held before: an earlier run's report, or, when the error came after mutants had run (too_many_harness_errors), possibly this run's last checkpoint, with the mutants it had not tested Pending. A SIGTERM that arrives while the final reports are being written stops the writes: each report is then either its final version or the last checkpoint, which may hold fewer verdicts than the finish counts.

A file that ends without a finish belongs to a run that is still going or died without stopping cleanly — a SIGKILL or a crash; its stderr says which.

Object keys are not in a fixed order, and a later version may add fields and events: a consumer ignores the ones it does not know. A change that would mislead such a consumer bumps version.

Summary

Types

Why a run stopped, as a finish event names it.

Types

stopped()

@type stopped() :: :complete | :max_survivors | :time_budget | :sigterm

Why a run stopped, as a finish event names it.