A relation is validated by its basis. Which bases make it current:
| Type | Validated by |
|---|---|
implements | evidence or a review only, never by hand |
verifies | evidence of the failing run, a review, or a judgement |
excuses, depends_on | a judgement |
tests, refines | nothing: structural facts read from the source |
Evidence: mix test && mix surfex.confirm --evidence. It records every relation the
test runs show: verifies on a test's failing run; tests and implements once a test
has gone red against one version of the code and green against another (or is
baselined under adoption, mix surfex.info adoption).
Review: mix surfex.validate TEST SPEC_UNIT --note N. Read the unit's claims and the
test; judge whether the test checks them (cases, boundaries, what must not change); fix
the test where it falls short. It needs the verifies relation and green evidence at the
current test and code versions. The unit may be a section whose hint the test already
verifies: the review then records the section's implements for the code the test
exercises. The note says which assertion checks which claim.
Code behind an entry point: a test reaches what it calls and, transitively, what that
code calls, as the source states it. So a unit may cite the function doing the work while
its test drives the public entry point: evidence and a review credit the cited function's
implements, and the triangle counts it exercised. Reach never makes a relation: tests
names what the test calls itself. Code reached only at run time (messages, dispatch by
name) isn't reached by reading the source, with one exception: a function calling its own
module's behaviour (GenServer.call) reaches the module's hidden @impl callbacks, and
a call to a hidden definition (a Mix task's run/1) goes on through what it calls, as
does a call into a @moduledoc false module. Hidden definitions are versioned in their
module, and in the public functions that run them, so changing a callback dangles both; a
hidden module's code is versioned in every function and module that runs it, in any file.
A review's note speaks for what was read, so
validate credits reached code only when you name it: --code ID, after reading it.
A path no call shows: a test may exercise code through a route, a message to a
process, a JavaScript hook or a capture handed to a helper, which no call in the source
names. Relate the code to the claim it realises, not to every section that mentions it
(mix surfex.relate CODE CLAIM --type implements --note N), then review it naming the
path: mix surfex.validate TEST CLAIM --code ID --through "message {:feed, _}" --note N.
The entry records through, history shows it, status counts these reviews, and the
triangle takes the review's word for the path. --through is refused for code the run
reached, and without --code.
Several arities, one definition: a function with default arguments (put/3 and
put/4 from one def) is one definition, so one review credits both under its one note.
validate names them when it does: write the note for the definition, not one arity.
Many at once: --file PATH on confirm or validate records a file of relations in one
run, one per line, tab-separated, each line ending with its own note (FROM TO TYPE NOTE,
or TEST UNIT NOTE). The notes must be distinct; write each line after judging that
relation.
Evidence from CI: --merge PATH (once per file) on confirm --evidence, validate,
relate and suggest --accept reads CI jobs' evidence files beside the local run, so a
test excluded locally (it needs a service you don't run) counts from CI's run.
Checks: some claims are verified by a gate, not a test: a linter rule (no rescue),
a security scanner, a project's own task comparing a layout with a header. Declare it
under checks: (mix surfex.info config) with the units it verifies and the files it
inspects, record its verifies (mix surfex.suggest --accept), and run it with mix surfex.check. A check is evidence only once it has shown it can fail: break an input on
purpose (the rule's config, or the code it inspects), run it red, restore it, run it
green, then mix surfex.confirm --evidence. A check that has never failed is validated
only by a review whose note says what it would catch: mix surfex.validate CHECK UNIT --note N. Changing the check or any input dangles its relations; a green run
re-confirms them. CI runs mix surfex.check in the job whose evidence the check reads,
or a check-confirmed claim reads as not run. Checks verify: they never carry
implements, which is related by hand to the code the claim is about. A gate is the
right verifier for a rule over the whole code (style, safety, layout); a behaviour is a
test's.
Docstrings aren't covered. A function's version leaves out its @doc, so a docstring
that states a contract can drift from the spec unnoticed. Read them together.
Judgement: mix surfex.confirm FROM TO --type T --note N. For a spec reworded with
no change in behaviour (verifies), or an excuse. It refuses implements.
--validated on status fails on current implements/verifies relations without one of
these (unvalidated: recorded before validation existed, or a baseline no longer trusted).