Surfex.SourceScan (Surfex v0.6.2)

Generic, dependency-free primitives for reading Elixir source without compiling it: find a project root, enumerate lib sources, extract defmodule nodes, and version a definition by its content.

Not compiling is the point. A gate built on these runs before, and independently of, the code it inspects — so it can be a build gate rather than a test, and it cannot be fooled by a module that failed to load.

Stdlib only, and deps: [] by policy: anything that depends on this should not inherit a kernel along with it.

It knows nothing about what is being catalogued. Finding a use Authz.Api opener, a TLA.Spec action, or a terminal escape sequence is the scanner's job, and a scanner belongs to the project whose vocabulary it reads. This is what every scanner shares.

Extracted from agentronic's Maatronic.SourceScan, where ten surface goldens have used it in production. The algorithms below are unchanged; only the prose and the project-root marker were generalised.

Summary

Types

definition()

One public definition of a module: a name at one arity, hashed over all its clauses.

Functions

definition_hash(source)

A definition's content version: a short sha256 (8 lowercase hex chars) over the item's AST with ALL positional and formatting metadata stripped, so the hash is a stable function of the definition's STRUCTURE — not of its position in the file or its layout. Inserting a blank line or a comment above the definition, or reindenting it, leaves the hash unchanged; changing the definition's body changes it.

defmodules(ast)

Every {:defmodule, meta, [aliases, [do: body]]} node in a quoted ast, in source order — the enumeration a scanner walks to find declaring modules. Nested defmodules are included, in the order they appear.

defs(arg)

The public definitions in a defmodule node's own body, sorted by name and arity. A nested module's definitions are that module's, not this one's.

hidden_module?(arg)

Whether a defmodule node declares @moduledoc false in its own body.

lib_sources(root)

Every .ex source in a first-party lib/ under root, sorted: the sources a scanner walks. It works for a single library (lib/) and a multi-member tree (app/lib/) alike, with nothing to configure.

line_range(ast)

The first and last source lines a quoted node spans, from its metadata, or nil when it carries none. Parse with token_metadata: true so a do … end block's closing line is known. Metadata never reaches a hash, so parsing this way changes no version.

module_hash(node)

A module's version: a definition_hash/1 over its public surface, not its whole body. That covers

project_root(marker, start \\ File.cwd!())

The project root: ascend from start (the working directory by default) to the first directory containing marker, falling back to start when none is found.

types(arg)

The module's public types: each @type and @opaque, minus one under @typedoc false, as Elixir's docs leave it out. A @typep is private. Each has its name, arity, version (the hash of its declaration, so it changes with the type alone) and lines.

Types

definition()

@type definition() :: %{
  name: atom(),
  arity: non_neg_integer(),
  kind: :function | :macro,
  hash: String.t(),
  lines: {pos_integer(), pos_integer()} | nil
}

One public definition of a module: a name at one arity, hashed over all its clauses.

Functions

definition_hash(source)

@spec definition_hash(Macro.t() | String.t()) :: String.t()

This is what lets a golden's Locus be a stable path plus a version, rather than a path.ex:line that drifts whenever anything above the item moves. A line number reports movement; this hash reports change. It is also what makes a golden auto-merge across branches: two people editing different items touch different rows.

Accepts a quoted AST node, or a binary source snippet parsed with Code.string_to_quoted!/1.

defmodules(ast)

@spec defmodules(Macro.t()) :: [Macro.t()]

defs(arg)

@spec defs(Macro.t()) :: [definition()]

The public definitions in a defmodule node's own body, sorted by name and arity. A nested module's definitions are that module's, not this one's.

Clauses are grouped by {name, arity} and hashed together with definition_hash/1, so editing any clause changes the function's hash and moving it does not. A default argument (\\) declares every arity it generates, each with the function's one hash.

Public means def, defmacro, defdelegate and defguard, minus what the module hides from its docs: @doc false, and an @impl callback without an explicit @doc (Elixir hides those by default, as the behaviour's surface rather than the module's). A definition whose name is computed (def unquote(name)(…)) is not knowable without compiling and is skipped.

hidden_module?(arg)

@spec hidden_module?(Macro.t()) :: boolean()

Whether a defmodule node declares @moduledoc false in its own body.

lib_sources(root)

@spec lib_sources(String.t()) :: [String.t()]

A lib/ directory is first-party when both hold:

its parent has a mix.exs. It is a Mix project's own source. A lib/ that a test fixture or a scratch tree happens to contain is not.
no directory between root and it is deps, _build, test, tmp, or hidden (a dot-directory). The first rule alone is not enough: dependencies are Mix projects, and some test fixtures are complete ones, mix.exs and all.

Before this rule, anything matching **/lib/** counted. That included the tmp/ trees ExUnit's @tag :tmp_dir leaves behind, so a golden could drift on the machine that had just run the tests and never in CI, whose checkout is clean (#9).

line_range(ast)

@spec line_range(Macro.t()) :: {pos_integer(), pos_integer()} | nil

module_hash(node)

@spec module_hash(Macro.t()) :: String.t()

A module's version: a definition_hash/1 over its public surface, not its whole body. That covers:

its @moduledoc
its public definitions, as name, arity and kind (defs/1)
its @behaviours and uses
its defstruct or defexception fields
its @type, @opaque, @callback and @macrocallback declarations

Each list is sorted, so reordering is not a change. Function bodies are left out: each public function has its own item and version, so a body edit changes that function's version and not its module's. Private definitions and nested modules are left out too.

project_root(marker, start \\ File.cwd!())

@spec project_root(String.t(), String.t()) :: String.t()

The project root: ascend from start (the working directory by default) to the first directory containing marker, falling back to start when none is found.

start exists so that nothing has to change the working directory to ask. The working directory belongs to the whole VM: changing it for one caller changes it for every process running alongside.

marker is explicit and has no default, deliberately. A poncho's root is the directory holding its aggregate build marker ("scripts/poncho.exs"); a single library's is the directory holding "mix.exs". A default would be right for one shape and silently wrong for the other, and "silently wrong about which tree you are scanning" is the failure a drift gate is least able to notice.

project_root("mix.exs")             # a library
project_root("scripts/poncho.exs")  # a poncho
project_root("mix.exs", "lib/my_app/deep")

types(arg)

@spec types(Macro.t()) :: [
  %{
    name: atom(),
    arity: non_neg_integer(),
    hash: String.t(),
    lines: {pos_integer(), pos_integer()} | nil
  }
]