Testing guide
YggdraSIM uses pytest and has strict operator rules around how the suite is invoked. This page documents the layout, the scoping conventions, and the no-mass-run policy that keeps test runs predictable.
On this page
Layout
All tests live under the repository tests/ directory. Each test file maps
to a subsystem or a cross-cutting concern. Representative file patterns:
| Pattern | Covers |
|---|---|
tests/test_scp03_*.py |
SCP03 shell and logic |
tests/test_scp11_*.py |
SCP11 family shells and helpers |
tests/test_scp11_eim_*.py |
SCP11 eIM-local shell and logic |
tests/test_scp11_local_access*.py |
SCP11 local-access shell |
tests/test_scp11_live_*.py |
SCP11 eSIM management relay |
tests/test_scp11_test_*.py |
SCP11 test compatibility namespace |
tests/test_simcard_*.py |
SIMCARD simulator backend |
tests/test_saip_*.py |
SAIP profile package codecs and tooling |
tests/test_profile_package_*.py |
Profile Package shell / saip tool |
tests/test_hil_bridge_*.py |
HIL bridge, runtime, and protocol |
tests/test_plugin_*.py |
plugin runtime behavior |
tests/test_apdu_fuzzer*.py |
APDU mutation fuzzer (allow-list, mutators, transports) |
tests/test_eum_diag*.py |
EUM / SM-DP+ session-key diagnostics + Lua dissector |
tests/test_yggdracore_*.py |
YggdraCore AUSF / AAnF stubs, FastAPI loopback, Open5GS bridge |
tests/test_yggdrasim_common_*.py |
shared helpers |
Scoping conventions
- Tests must be runnable in isolation. A single-node pytest invocation must produce a correct result.
- Tests should avoid module-level side effects. Fixtures own setup and teardown.
- Tests that touch PC/SC or HIL hardware must be clearly gated so they do not run unless the hardware is present.
- Tests that require plugin runtime behavior should drop a temporary plugin
through fixtures, not mutate the repository
plugins/tree.
The default test bootstrap redirects HOME, USERPROFILE, GNUPGHOME, XDG
directories, and YGGDRASIM_RUNTIME_ROOT into a temporary session tree before
test modules are imported. This prevents a developer's keyring, inventory, or
runtime state from selecting test outcomes. Hardware/integration harnesses may
opt into ambient operator state only with
YGGDRASIM_TEST_USE_AMBIENT_STATE=1.
Tests that bind a real loopback listener use the require_loopback_socket
fixture. A sandbox policy that prohibits socket creation is then reported as
an environment skip; protocol behavior is still tested normally on hosts that
permit loopback networking.
Running a single test
Respect the repository's memory-safety rule. Target narrowly.
python -m pytest -q --tb=short --disable-warnings --no-header --maxfail=1 \
tests/test_scp11_local_access.py
For a single node:
python -m pytest -q --tb=short --disable-warnings --no-header --maxfail=1 \
tests/test_scp11_local_access.py::test_load_profile_happy_path
Redirecting output
Noisy tests or captured-output tests should be redirected to a log file, with the log overwritten on each run:
python -m pytest -q --tb=short --disable-warnings --no-header --maxfail=1 \
tests/test_saip_transcode_tui.py > .pytest_last_run.log 2>&1
Read the log using rg FAILED .pytest_last_run.log or a tail-offset read.
Do not paste the full log into commit messages or issue threads.
Shell-level timeout cap
Every pytest invocation is expected to complete within 90 seconds of wall time. The cap keeps local and CI runs predictable and prevents a single slow test from holding the queue. If a test genuinely needs more than 90 s:
- Narrow the target first -- a single test class or node id usually fits.
- If it still does not fit, tag the offender with
@pytest.mark.slowand note the flag in the test docstring. - Only ask to raise the cap explicitly, for a single named run. Do not silently override it.
tests/test_saip_transcode_tui.py is currently the one module-wide
pytest.mark.slow surface; invoke it with --runslow during release
validation.
No mass runs
Repo-wide runs such as pytest, python -m pytest, pytest tests, or
pytest . are reserved for explicit user requests with a chunked plan. The
default is always the narrowest target possible.
Writing a new test
When adding a new test:
- Pick the closest existing filename and place the test there, or add a new
tests/test_<area>.pythat matches the patterns above. - Use plain functions and fixtures. Avoid class-based tests unless the fixture composition genuinely benefits.
- Keep assertions specific. Prefer one clear assertion that fails with a meaningful message over a cascade of prints.
- If you add a new fixture, keep it local to the file unless it is obviously useful for multiple files.
- Never introduce a test that depends on real hardware without an explicit guard.
What to test
- the public surface of a shell:
--cmdoutputs, verb parsing, error reporting - codec round trips: SAIP JSON/DER, ASN.1 builders, TLV helpers
- registry resolution against stable keys
- plugin runtime absence and failure paths
- state persistence and migration behavior
- HIL protocol framing where the hardware is emulated
Conformance vectors for duplicated wire logic
Some wire-format logic exists in more than one place. SCP11 carries three
BPP segmenter definitions plus an inheriting shim, and they drifted once
before anyone noticed, because each was asserted against separately.
Where that happens, do not write one test per copy. Write one vector table and run every implementation over it:
- A table of
(label, input, expected output), with the bytes that carry the wire risk spelled as explicit hex rather than rebuilt by a helper. - A registry of implementations, since construction usually differs.
- An AST sweep asserting the registry covers every definition in the tree, so a new copy fails the suite instead of drifting quietly beside it.
tests/test_bpp_segmentation_annex_m.py is the worked example. Validate a
table like this by mutation: reintroduce the bug it exists to prevent and
confirm the vectors fail and name the implementation. A vector table that
survives its own bug is decoration.
The same shape applies to the pinned upstream divergences in
tests/test_scp03_pysim_parity.py and tests/test_scp80_pysim_parity.py.
Related pages
- Coding Standards
- Release Checklist
tests/under the repository root