Crash diagnostics

The verum binary installs a crash reporter at process start. Every panic and every fatal signal (SIGSEGV, SIGBUS, SIGILL, SIGFPE, SIGABRT on Unix; SetUnhandledExceptionFilter on Windows) produces a structured report on disk with:

• the command the user ran, its cwd, and a filtered view of the process environment (secret-looking keys redacted);
• the frozen build identity — verum version, git SHA, build profile, target triple, rustc --version;
• the compiler phase where the fault occurred, via an RAII breadcrumb trail maintained by the pipeline;
• a Rust backtrace, which resolves to file:line when the binary ships with DWARF line tables (see profiles).

Reports live under ~/.verum/crashes/ as a matching .log (human) and .json (schema-versioned) pair. The reporter keeps the last 50 by default and rotates older ones out.

The verum diagnose subcommand is the user-facing interface for the report store — list, show, sanitise, bundle, and submit.

Report layout

~/.verum/crashes/
  verum-2026-04-19T05-08-22-d6b3-abcdef-0.log   ← human-readable
  verum-2026-04-19T05-08-22-d6b3-abcdef-0.json  ← structured (schema v1)

Filenames sort chronologically (ISO-8601 date with - in place of : for filesystem portability).

.log example

=== Verum crash report ===========================================
Report ID:   1538d-19da2a45011-0
Timestamp:   1776550170641 (unix-ms)
Kind:        fatal signal SIGSEGV (11)
Thread:      verum-main
Message:     received fatal signal SIGSEGV (11)

Build:       verum 0.1.0 (release, aarch64-apple-darwin, rustc 1.93.0, git abc1234 clean)
Host:        macos aarch64 (16 cores)
PID:         86925
Cwd:         /Users/me/projects/demo
Args:        verum build ./src/main.vr

Context:
  command:    build
  input:      ./src/main.vr

Breadcrumbs (most recent last):
  [  1611ms] compiler.run_native_compilation     ./src/main.vr   [thread=verum-main]
  [   313ms] compiler.phase.generate_native      ./src/main.vr   [thread=verum-main]
  [    12ms] compiler.codegen.vbc_to_llvm        project          [thread=verum-main]

Backtrace:
   …

Environment (filtered):
  HOME=/Users/me
  LANG=en_US.UTF-8
  TMPDIR=/Users/me/.tmp
  RUST_BACKTRACE=1
===================================================================

.json example

{
  "schema_version": 1,
  "report_id": "1538d-19da2a45011-0",
  "timestamp_ms": 1776550170641,
  "kind": { "type": "signal", "name": "SIGSEGV", "signo": 11 },
  "message": "received fatal signal SIGSEGV (11)",
  "location": null,
  "backtrace": "…",
  "thread_name": "verum-main",
  "breadcrumbs": [
    { "phase": "compiler.run_native_compilation", "detail": "./src/main.vr", "thread": "verum-main", "age_ms": 1611 },
    { "phase": "compiler.phase.generate_native",  "detail": "./src/main.vr", "thread": "verum-main", "age_ms": 313  }
  ],
  "context": { "command": "build", "input_file": "./src/main.vr" },
  "environment": {
    "verum_version": "0.1.0",
    "build_profile": "release",
    "build_target":  "aarch64-apple-darwin",
    "build_rustc":   "rustc 1.93.0",
    "build_git_sha": "abc1234",
    "build_git_dirty": "clean",
    "os": "macos",
    "arch": "aarch64",
    "cpu_cores": 16,
    "pid": 86925,
    "cwd": "/Users/me/projects/demo",
    "argv": ["verum", "build", "./src/main.vr"],
    "env": { "HOME": "/Users/me", "RUST_BACKTRACE": "1", … }
  }
}

Breadcrumbs

The pipeline instruments its major phases with RAII breadcrumbs — the trail of phase names and per-phase details leading up to the crash. Typical phases:

Phase	When
compiler.run_native_compilation	AOT build driver
compiler.phase.stdlib_loading	embedded stdlib
compiler.phase.project_modules	sibling modules
compiler.phase.load_source / .parse	front-end
compiler.phase.type_check	type inference
compiler.phase.verify	refinement / SMT
compiler.phase.cbgr_analysis	CBGR tier analysis
compiler.phase.ffi_validation	FFI boundary checks
compiler.phase.rayon_fence	wait for rayon workers before LLVM
compiler.phase.generate_native	LLVM codegen
compiler.codegen.vbc_to_llvm	inner VBC → LLVM lowering
compiler.phase.interpret	Tier 0 (VBC interpreter)

Third-party code that embeds verum_compiler / verum_vbc can push its own breadcrumbs by acquiring a scoped guard from verum_error.breadcrumb.enter("mytool.stage", file_path). The guard is automatically popped on scope exit, so a breadcrumb cannot outlive its surrounding work.

The trail is bounded (64 entries) and mirrored to a cross-thread snapshot so the signal handler can include it even when the offending thread's TLS is unreachable.

Sensitive data

The reporter is conservative about what it captures:

• argv, cwd, and the breadcrumb details are preserved verbatim in the on-disk report. They are intended for the developer who ran the build — the report is local, not uploaded.
• Environment variables are filtered. Only VERUM_*, RUST*, CARGO*, and a curated whitelist (HOME, USER, LANG, TERM, TMPDIR, LLVM_*, etc.) survive. Anything whose name contains PASSWORD, SECRET, TOKEN, APIKEY, PRIVATE, SESSION, COOKIE, CREDENTIAL, AUTH, or PASSPHRASE is replaced with <redacted> even if the name itself is whitelisted.

When sharing a report externally, use --scrub-paths to replace $HOME with ~ and the current username with <user> in the emitted output. The originals on disk are not modified.

The verum diagnose command

verum diagnose <subcommand> [options]

list

List recent reports in ~/.verum/crashes/, newest first, with a one-line summary of each (kind, message, build, last known phase).

verum diagnose list                # last 20
verum diagnose list --limit 50     # widen the window

show

Print the full report to stdout. Defaults to the most recent.

verum diagnose show                                # newest .log
verum diagnose show path/to/report.log
verum diagnose show --json                         # structured form
verum diagnose show --scrub-paths                  # safe-to-share render

bundle

Pack recent reports (both the .log and the .json) into a single .tar.gz suitable for attaching to an issue. A README inside the archive explains where to upload it.

verum diagnose bundle                              # last 5 → ./verum-crash-bundle-<ts>.tar.gz
verum diagnose bundle --recent 3 -o report.tgz
verum diagnose bundle --scrub-paths                # sanitise every file in the archive

--scrub-paths rewrites each bundled file — the originals under ~/.verum/crashes are untouched.

submit

Open a new GitHub issue via the gh CLI. Paths are always scrubbed before upload; the .tar.gz path is printed for the user to attach manually (the gh CLI does not accept attachments at issue creation time).

verum diagnose submit                                # verum-lang/verum
verum diagnose submit --repo my/fork --recent 3
verum diagnose submit --dry-run                      # print the gh invocation

Requires gh auth login.

env

Print the build/host environment snapshot that the reporter captured at install time — useful when diagnosing "which verum am I running" questions without needing a crash.

verum diagnose env
verum diagnose env --json

clean

Delete every report in ~/.verum/crashes/.

verum diagnose clean          # prompts for confirmation
verum diagnose clean --yes    # unattended

Debug-info profile

The primary [profile.release] stays stripped for binary size and runtime stability (keeping DWARF in release re-introduces an LLVM pass-registration race on macOS — see the note below). A dedicated profile keeps line tables so crash-report backtraces resolve to file:line:

cargo build --profile release-debug-tables --bin verum

This produces target/release-debug-tables/verum plus an external .dSYM (macOS) or .dwp (Linux) bundle next to the binary. The main binary size is unchanged; the extra data lives in the bundle, which the backtrace crate consults automatically when resolving frames.

Ship the debug-tables build to users who are triaging a reported bug; keep the primary release build on production paths. Do not fold debug = "line-tables-only" into [profile.release] — the extra DWARF-emitter passes expand the lazy-init surface that races rayon worker wake paths, re-introducing a ~70 % SIGSEGV rate in the phase_generate_native codegen step.

Chaining your own panic hook

crash.install chains into whatever hook was set before it. If you need custom panic metrics in addition to the crash report, install your own hook first and then call verum_error.crash.install(...) — the crash reporter will defer to the previously installed hook before doing its own work.

The Verum CLI itself does not install the stock PanicLogger from verum_error.panic_handler by default. Benchmarking showed the extra hook measurably destabilised release builds on the codegen race path; the structured report produced by the crash reporter already contains everything PanicLogger would record, plus the breadcrumb trail and environment snapshot.

Signal-safety caveats

The reporter is best-effort async-signal-safe, not strictly so. It:

• installs on an alternate signal stack (sigaltstack) so a stack overflow still reaches the handler;
• pre-creates the report directory at install time;
• uses the backtrace crate to capture frames from the signal path (pragmatic choice — not strictly sig-safe but works in practice for dev tools);
• re-raises the original signal after writing the report so the kernel still produces a core dump if ulimit -c allows.

A hard fault may leave the global allocator poisoned; in that case the JSON write may fail and only the short stderr notice survives. That notice still includes the report ID so a subsequent run can correlate the two events.

Configuration

All of the above is controlled by verum_error.crash.CrashReporterConfig:

Field	Default	What it controls
app_name	"verum"	Verbatim brand string surfaced in five places: the === {Titlecased} crash report === log header (first letter title-cased automatically), the Build: {app_name} {ver} (...) line of the human report, the app_name field of the JSON envelope's environment block, the {app_name}: internal compiler error... stderr prefix, and the run \{app_name} diagnose bundle`` hint shown after a crash. Embedders override this to rebrand every surface in one place.
app_version	env!("CARGO_PKG_VERSION")	Mirrored into EnvSnapshot.verum_version (kept under that name for schema stability) and rendered in the human report's Build line.
report_dir	None	Unset means default_report_dir(): ~/.verum/crashes/ when HOME (or USERPROFILE) resolves, and $TMPDIR/verum-crashes when it does not — a headless CI container hits the second. Created at install time so the signal handler doesn't have to.
retention	50	Older reports rotated off after every successful write — the rotator deletes oldest first by mtime.
capture_backtrace	true	Also forces RUST_BACKTRACE=1 so the symbolizer captures frames even if the user hasn't set the env var.
install_signal_handlers	true	Unix: SIGSEGV / SIGBUS / SIGILL / SIGFPE / SIGABRT on an alternate signal stack via sigaltstack. Windows: SetUnhandledExceptionFilter. Set to false if a host process owns its own handler.
redact_sensitive_env	true	When true, env vars whose name matches PASSWORD/SECRET/TOKEN/APIKEY/PRIVATE/SESSION/COOKIE/CREDENTIAL/AUTH/PASSPHRASE (case-insensitive substring) render as <redacted> even when they pass the keep-list. Set to false only in dev environments where you need the raw env.
issue_tracker_url	https://github.com/verum-lang/verum/issues/new	Rendered verbatim under the "Please file an issue at:" line on crash. Embedders point this at their own tracker.

Downstream tools that embed the compiler should install with an app_name + issue_tracker_url appropriate to them so their crash surfaces point users at the right bug tracker. The app_name flows through every user-facing rendering surface — log header, build line, JSON envelope, stderr branding, diagnose-bundle hint — so a single override rebrands the whole reporter without touching any rendering code:

In practice an embedder constructs a CrashReporterConfig, overrides app_name and issue_tracker_url to its own identifiers, leaves the remaining fields at their defaults, and calls verum_error.crash.install(cfg).

The header title-cases the first letter, so myapp renders as Myapp. The lowercased app_name is used verbatim in the stderr prefix and the diagnose bundle hint, matching the shell convention of lowercase tool names.

Render configuration knobs

Embedders that drive verum_diagnostics::Renderer directly control output shape via RenderConfig. Every documented field is honoured by the consumer (no inert defenses):

Field	Default	What it gates
colors	true	ANSI escape codes around severity / spans.
context_lines	2	Lines shown above / below each labeled line.
show_line_numbers	true	When false, the gutter pads with spaces of the same width so the pipe alignment stays consistent for downstream parsers.
max_line_width	120	Caps source-line rendering. Lines longer than the cap render with an ellipsis suffix (UTF-8-safe char-count slicing). 0 disables truncation. Bounds the diagnostic size on minified-source workloads.
show_source	true	Master switch on the file-snippet block.
show_suggestions	true	Toggles the inline fix-it suggestions.
show_doc_urls	true	Toggles verum-lang.org/errors/<code> URL footers.
unicode_output	true	Box-drawing + arrow glyphs vs ASCII fallback.
terminal_width	80	Read mirror via Renderer::terminal_width(). The renderer itself doesn't soft-wrap (terminals do that for free); the value is exposed so external composers (LSP servers framing into JSON-RPC, CI pretty-printers) can consult the configured stance.
relative_paths	false	When true, the file-snippet header strips the CWD prefix from absolute paths. Falls back to the original path on any failure (CWD lookup error, different filesystem mount, path is not a child of CWD).

See also

• Reference → CLI commands — verum diagnose — full flag reference.
• Guides → Troubleshooting — common error recipes.
• Community → Contributing — how to report a compiler bug.