Skip to content
YggdraSIM

Testing guide

YggdraSIM uses pytest and has strict operator rules around how the suite is invoked. This page documents the layout, the scoping conventions, and the no-mass-run policy that keeps test runs predictable.

On this page

Layout

All tests live under the repository tests/ directory. Each test file maps to a subsystem or a cross-cutting concern. Representative file patterns:

Pattern Covers
tests/test_scp03_*.py SCP03 shell and logic
tests/test_scp11_*.py SCP11 family shells and helpers
tests/test_scp11_eim_*.py SCP11 eIM-local shell and logic
tests/test_scp11_local_access*.py SCP11 local-access shell
tests/test_scp11_live_*.py SCP11 eSIM management relay
tests/test_scp11_test_*.py SCP11 test compatibility namespace
tests/test_simcard_*.py SIMCARD simulator backend
tests/test_saip_*.py SAIP profile package codecs and tooling
tests/test_profile_package_*.py Profile Package shell / saip tool
tests/test_hil_bridge_*.py HIL bridge, runtime, and protocol
tests/test_plugin_*.py plugin runtime behavior
tests/test_apdu_fuzzer*.py APDU mutation fuzzer (allow-list, mutators, transports)
tests/test_eum_diag*.py EUM / SM-DP+ session-key diagnostics + Lua dissector
tests/test_yggdracore_*.py YggdraCore AUSF / AAnF stubs, FastAPI loopback, Open5GS bridge
tests/test_yggdrasim_common_*.py shared helpers

Scoping conventions

  • Tests must be runnable in isolation. A single-node pytest invocation must produce a correct result.
  • Tests should avoid module-level side effects. Fixtures own setup and teardown.
  • Tests that touch PC/SC or HIL hardware must be clearly gated so they do not run unless the hardware is present.
  • Tests that require plugin runtime behavior should drop a temporary plugin through fixtures, not mutate the repository plugins/ tree.

The default test bootstrap redirects HOME, USERPROFILE, GNUPGHOME, XDG directories, and YGGDRASIM_RUNTIME_ROOT into a temporary session tree before test modules are imported. This prevents a developer's keyring, inventory, or runtime state from selecting test outcomes. Hardware/integration harnesses may opt into ambient operator state only with YGGDRASIM_TEST_USE_AMBIENT_STATE=1.

Tests that bind a real loopback listener use the require_loopback_socket fixture. A sandbox policy that prohibits socket creation is then reported as an environment skip; protocol behavior is still tested normally on hosts that permit loopback networking.

Running a single test

Respect the repository's memory-safety rule. Target narrowly.

python -m pytest -q --tb=short --disable-warnings --no-header --maxfail=1 \
    tests/test_scp11_local_access.py

For a single node:

python -m pytest -q --tb=short --disable-warnings --no-header --maxfail=1 \
    tests/test_scp11_local_access.py::test_load_profile_happy_path

Redirecting output

Noisy tests or captured-output tests should be redirected to a log file, with the log overwritten on each run:

python -m pytest -q --tb=short --disable-warnings --no-header --maxfail=1 \
    tests/test_saip_transcode_tui.py > .pytest_last_run.log 2>&1

Read the log using rg FAILED .pytest_last_run.log or a tail-offset read. Do not paste the full log into commit messages or issue threads.

Shell-level timeout cap

Every pytest invocation is expected to complete within 90 seconds of wall time. The cap keeps local and CI runs predictable and prevents a single slow test from holding the queue. If a test genuinely needs more than 90 s:

  1. Narrow the target first -- a single test class or node id usually fits.
  2. If it still does not fit, tag the offender with @pytest.mark.slow and note the flag in the test docstring.
  3. Only ask to raise the cap explicitly, for a single named run. Do not silently override it.

tests/test_saip_transcode_tui.py is currently the one module-wide pytest.mark.slow surface; invoke it with --runslow during release validation.

No mass runs

Repo-wide runs such as pytest, python -m pytest, pytest tests, or pytest . are reserved for explicit user requests with a chunked plan. The default is always the narrowest target possible.

Writing a new test

When adding a new test:

  1. Pick the closest existing filename and place the test there, or add a new tests/test_<area>.py that matches the patterns above.
  2. Use plain functions and fixtures. Avoid class-based tests unless the fixture composition genuinely benefits.
  3. Keep assertions specific. Prefer one clear assertion that fails with a meaningful message over a cascade of prints.
  4. If you add a new fixture, keep it local to the file unless it is obviously useful for multiple files.
  5. Never introduce a test that depends on real hardware without an explicit guard.

What to test

  • the public surface of a shell: --cmd outputs, verb parsing, error reporting
  • codec round trips: SAIP JSON/DER, ASN.1 builders, TLV helpers
  • registry resolution against stable keys
  • plugin runtime absence and failure paths
  • state persistence and migration behavior
  • HIL protocol framing where the hardware is emulated

Conformance vectors for duplicated wire logic

Some wire-format logic exists in more than one place. SCP11 carries three BPP segmenter definitions plus an inheriting shim, and they drifted once before anyone noticed, because each was asserted against separately.

Where that happens, do not write one test per copy. Write one vector table and run every implementation over it:

  1. A table of (label, input, expected output), with the bytes that carry the wire risk spelled as explicit hex rather than rebuilt by a helper.
  2. A registry of implementations, since construction usually differs.
  3. An AST sweep asserting the registry covers every definition in the tree, so a new copy fails the suite instead of drifting quietly beside it.

tests/test_bpp_segmentation_annex_m.py is the worked example. Validate a table like this by mutation: reintroduce the bug it exists to prevent and confirm the vectors fail and name the implementation. A vector table that survives its own bug is decoration.

The same shape applies to the pinned upstream divergences in tests/test_scp03_pysim_parity.py and tests/test_scp80_pysim_parity.py.