Testing
Comprehensive guide for testing scuv.
Quick Reference
cargo test # Run all tests
cargo test json # Run tests containing "json"
cargo test -- --nocapture # Show println! output
cargo clippy -- -D warnings # Lint check
Test Structure
tests/
├── cli/ # CLI integration tests (one binary)
│ ├── main.rs # declares the topic modules
│ ├── support.rs # TestFixture, scoop_cmd, shared helpers
│ └── <topic>.rs # general, list, remove, color, shell, ...
└── i18n_completeness.rs # locale parity
src/
├── error/ # Unit tests for error types
├── validate/ # Unit + property tests for validate.rs (tests.rs, proptests.rs)
├── paths/tests.rs # Unit tests for paths.rs
├── python_version/ # Unit + property tests for python_version.rs
├── output/
│ └── json/tests.rs # Unit tests for JSON output
├── core/
│ ├── virtualenv/ # virtualenv service (mod.rs + tests.rs)
│ ├── version.rs # Unit tests for version service
│ ├── metadata.rs # Unit tests for metadata
│ └── doctor/ # Unit tests for doctor
├── shell/
│ ├── bash.rs # Shell script tests
│ └── zsh.rs # Shell script tests
└── uv/
├── client.rs # Unit tests for uv client
└── install/tests.rs # Unit tests for install.rs
Small test modules stay inline (#[cfg(test)] mod tests { ... }). Once one
passes roughly 450 lines it moves to a child file beside its source:
src/paths.rs declares #[cfg(test)] mod tests;, which loads
src/paths/tests.rs. The production file stays where it is, and the test
paths (paths::tests::...) do not change.
Running Tests
All Tests
# Run all tests
cargo test
# Run with all features enabled
cargo test --all-features
# Run in release mode (faster execution)
cargo test --release
How CI Runs Them
CI’s Test and MSRV jobs run the suite with cargo-nextest, which gives every test its own process:
cargo nextest run --all-features --workspace # everything except doctests
cargo test --doc --all-features # nextest does not run doctests
cargo nextest run --all-features --workspace --run-ignored only # the tests that need uv
cargo test runs tests as threads of one process, so a test that changes
PATH, HOME or SCUV_HOME changes it for every test running at that
moment. #[serial] keeps marked tests apart from each other, not from
unmarked ones. Under nextest that leak cannot happen, and coverage and
mutation testing run under nextest too, but a local cargo test still runs
in one process, so a test that reaches uv or reads those variables still
needs #[serial], or must run inside env_guard or with_temp_scoop_home,
which hold the same lock as the tests that change them.
Filtered Tests
# By name pattern
cargo test json # Tests containing "json"
cargo test error # Tests containing "error"
cargo test virtualenv # Tests containing "virtualenv"
# By module path
cargo test output::json # Tests in output/json/tests.rs
cargo test error::tests # Tests in error/tests.rs
cargo test core::version # Tests in core/version.rs
cargo test cli::commands # Tests in cli/commands/
# Single test
cargo test test_json_response_success_creates_correct_status
Test Output
# Show stdout/stderr (println!, dbg!, etc.)
cargo test -- --nocapture
# Show test names as they run
cargo test -- --nocapture --test-threads=1
# Only show failed tests
cargo test -- --quiet
Debugging
# Run single-threaded (easier to debug)
cargo test -- --test-threads=1
# Run ignored tests
cargo test -- --ignored
# Run specific test with output
cargo test test_name -- --nocapture --test-threads=1
Test Categories
Unit Tests
The bulk of the suite lives within source files using #[cfg(test)]:
#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_something() {
assert_eq!(1 + 1, 2);
}
}
}
Key test modules:
| Module | Coverage |
|---|---|
error::tests | Error types, codes, suggestions |
output::json::tests | JSON serialization, edge cases |
validate::tests | Name/version validation |
core::version::tests | Version file resolution |
core::virtualenv::tests | Virtualenv service |
paths::tests | Path utilities |
shell::*::tests | Shell scripts (shellcheck) |
Integration Tests (tests/cli/ + tests/i18n_completeness.rs)
tests/cli/ is one test binary (main.rs) with a module per topic:
general, list, remove, color, shell, dispatch, errors,
output_format, requires_uv, and shared fixtures (including a fake uv)
in support.
# Run only integration tests
cargo test --test cli
Categories:
- Error cases - Invalid inputs, missing arguments
- Output format - Help, version, JSON output
- Command behavior - list, create, use, remove
- Real shells - the
shellmodule sourcesscuv initin real bash, zsh, fish and PowerShell (pwsh) and drives the wrapper, hook and completion through them. Each test skips when its shell is not installed. The CI Test and MSRV jobs install fish and zsh (bash andpwshcome with the runner) and setSCUV_REQUIRE_FISH,SCUV_REQUIRE_ZSHandSCUV_REQUIRE_PWSH, so a missing fish, zsh orpwshfails there instead.
Some tests are marked #[ignore] because they require uv installed. The
Docker integration jobs run them with cargo nextest run --run-ignored all,
as their images carry uv and Python 3.12:
# Run ignored tests (requires uv)
cargo test -- --ignored
Doc Tests
Examples in documentation comments:
#![allow(unused)]
fn main() {
/// Validates environment name.
///
/// # Examples
///
/// ```
/// use scoop_uv::validate::is_valid_env_name;
/// assert!(is_valid_env_name("myenv"));
/// assert!(!is_valid_env_name("123bad"));
/// ```
pub fn is_valid_env_name(name: &str) -> bool { ... }
}
# Run only doc tests
cargo test --doc
Property Tests
Using proptest for randomized testing:
#![allow(unused)]
fn main() {
use proptest::prelude::*;
proptest! {
#[test]
fn prop_valid_names_accepted(name in "[a-zA-Z][a-zA-Z0-9_-]{0,49}") {
assert!(is_valid_env_name(&name));
}
}
}
Located in src/validate/proptests.rs.
Parameterized Tests
Using rstest #[case] tables for input/output matrices (e.g. version parsing,
env-name validation) so each case reports independently:
#![allow(unused)]
fn main() {
use rstest::rstest;
#[rstest]
#[case::simple("myenv", true)]
#[case::digit_start("123", false)]
#[case::reserved("activate", false)]
fn is_valid_env_name_cases(#[case] input: &str, #[case] expected: bool) {
assert_eq!(is_valid_env_name(input), expected);
}
}
Mutation Testing
cargo-mutants verifies the suite actually catches bugs (not just that lines
run). Config in .cargo/mutants.toml; CI runs it on changed lines per PR
(--in-diff) and a full pass weekly.
cargo install cargo-mutants
cargo mutants # full (scoped via mutants.toml)
git diff origin/main.. | cargo mutants --in-diff /dev/stdin # changed lines
Fuzz Testing
cargo-fuzz (libFuzzer) fuzzes the untrusted-input parsers. It lives in an
isolated fuzz/ workspace pinned to nightly, so it never affects the
MSRV-1.89 build; CI runs the targets on a weekly schedule.
cargo install cargo-fuzz
cargo +nightly fuzz run fuzz_env_name -- -max_total_time=60
Writing Tests
Unit Test Template
#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
use super::*;
// ========================================
// Test Group Name
// ========================================
#[test]
fn test_function_name_expected_behavior() {
// Arrange
let input = "test input";
// Act
let result = function_under_test(input);
// Assert
assert_eq!(result, expected_value);
}
#[test]
fn test_function_name_edge_case() {
let result = function_under_test("");
assert!(result.is_err());
}
}
}
Integration Test Template
#![allow(unused)]
fn main() {
// tests/cli/<topic>.rs (declare it in tests/cli/main.rs)
use crate::support::*;
use assert_cmd::Command;
use predicates::prelude::*;
#[test]
fn test_command_success() {
Command::cargo_bin("scuv")
.unwrap()
.args(["list"])
.assert()
.success()
.stdout(predicate::str::contains("expected output"));
}
#[test]
fn test_command_failure() {
Command::cargo_bin("scuv")
.unwrap()
.args(["use", "nonexistent"])
.assert()
.failure()
.stderr(predicate::str::contains("not found"));
}
}
JSON Output Testing
#![allow(unused)]
fn main() {
#[test]
fn test_json_serialization() {
let data = MyData { field: "value".into() };
let json = serde_json::to_string(&data).unwrap();
// Check JSON structure
let parsed: serde_json::Value = serde_json::from_str(&json).unwrap();
assert_eq!(parsed["field"], "value");
}
#[test]
fn test_optional_field_omitted() {
let data = MyData { optional: None, .. };
let json = serde_json::to_string(&data).unwrap();
// skip_serializing_if = "Option::is_none"
assert!(!json.contains("optional"));
}
}
Test Utilities
Located in src/test_utils.rs:
#![allow(unused)]
fn main() {
use scoop_uv::test_utils::*;
#[test]
fn test_with_temp_environment() {
with_temp_scoop_home(|temp_dir| {
// SCUV_HOME is set to temp_dir
// Cleanup happens automatically
});
}
#[test]
fn test_with_mock_venv() {
with_temp_scoop_home(|temp_dir| {
create_mock_venv(temp_dir, "myenv", Some("3.12"));
// Virtual environment created at temp_dir/virtualenvs/myenv
});
}
}
Coverage
Using cargo-tarpaulin
# Install
cargo install cargo-tarpaulin
# Run with HTML report
cargo tarpaulin --out Html --output-dir coverage
# Run with specific target
cargo tarpaulin --out Html --output-dir coverage --packages scoop-uv
# View report
open coverage/tarpaulin-report.html
Using cargo-llvm-cov
# Install
cargo install cargo-llvm-cov
# Run with HTML report
cargo llvm-cov --html
# View report
open target/llvm-cov/html/index.html
CI/CD Testing
Tests run automatically on:
- Every push to
main - Every pull request targeting
main
GitHub Actions workflow (.github/workflows/ci.yml):
- name: Run tests
run: cargo nextest run --profile ci --all-features --workspace
- name: Run doctests
run: cargo test --doc --all-features
- name: Run clippy
run: cargo clippy --all-targets --all-features -- -D warnings
Troubleshooting
Test Hangs
# Run single-threaded to identify hanging test
cargo test -- --test-threads=1
Flaky Tests
# Run specific test multiple times
for i in {1..10}; do cargo test test_name || break; done
Environment Issues
# Clear test artifacts
cargo clean
# Rebuild and test
cargo test
Shell Tests Fail
The real-shell tests in tests/cli/shell.rs skip a shell that is not
installed; install bash, zsh, fish and pwsh to run them all locally, or set
SCUV_REQUIRE_<SHELL> to make a missing one fail.
ShellCheck must be installed for shell script tests:
# macOS
brew install shellcheck
# Linux
apt install shellcheck
Best Practices
- Test naming:
test_<function>_<scenario>_<expected> - Arrange-Act-Assert: Clear test structure
- One assertion per test: When practical
- Test edge cases: Empty, unicode, special chars, boundaries
- No test interdependencies: Each test should be isolated
- Fast tests: Mock external dependencies
Codespaces / Devcontainer
.devcontainer/devcontainer.json boots a Rust 1.89 dev environment on
mcr.microsoft.com/devcontainers/rust:1-bookworm that matches the local
toolchain pinned by rust-toolchain.toml. Open the repo in VS Code
(“Reopen in Container”) or create a Codespace — both follow the same
lifecycle:
| Hook | Runs | Used for |
|---|---|---|
onCreateCommand | once (in Codespace prebuild) | install nextest / llvm-cov / mutants / uv, warm cargo build |
updateContentCommand | on every prebuild refresh | cargo fetch to keep registry warm |
postCreateCommand | when user creates the Codespace | prek install |
Cargo registry + git caches persist as named Docker volumes scoped per
project so two scoop-uv worktrees don’t share caches. target/ is NOT
volumed — it’s architecture-specific and warmed in the prebuild layer.
Enabling Codespace prebuilds
In repo Settings → Codespaces → “Set up prebuild”, configure for the
main branch on the “configuration change” trigger. This keeps Actions
minutes low (only rebuilds when .devcontainer/** or Dockerfile
changes) while still keeping the cargo registry warm via volume
persistence.
Multi-source Integration (Docker matrix)
The Dockerfile builds three per-source fixture stages, pyenv-fixture,
conda-fixture and venvwrapper-fixture, on top of a shared
scuv-test-base. Each carries only the source tool it migrates from,
and none depends on the source tree, so a change under src/ leaves
their layers cached. The pyenv-test, conda-test and
venvwrapper-test stages add the image’s own scuv binary on top for
the interactive shells below.
# Local (sequential)
make test-integration-pyenv
make test-integration-conda
make test-integration-venvwrapper
# Local (all three sequentially)
make test-integration-all
# Drop into a single variant for ad-hoc debugging
make docker-shell-conda
CI runs all three in parallel via
.github/workflows/integration-test.yml’s strategy.matrix with
per-source BuildKit cache scoping (cache-from/to=type=gha,scope=<src>).
Each job builds the fixture stage and compiles the workspace once, inside
the container: nextest builds the debug scuv binary with the tests, and
SCUV_ENTRYPOINT_BUILD=0 stops the entrypoint’s own release build. It
then checks with docker/scripts/check_fixture.py that
scuv migrate list --json finds that image’s fixture env ready to
migrate, and that scuv migrate @env <fixture> --dry-run --json reads its
packages and their origins with no failure: the only place a real conda,
pyenv or virtualenvwrapper env is read.
Benchmarks (Criterion)
Three bench binaries live in benches/:
| Binary | Targets |
|---|---|
parsing | clap parse, TOML manifest, JSON uv python list |
validation | is_valid_env_name across 6 representative inputs |
path_lookup | find_executable_in hit + miss |
Local workflow
# Run all benches once (no baseline diffing)
cargo bench
# Save current results as a named baseline (default: "main")
make bench-save # saves as "main"
make bench-save BENCH_BASELINE=before-X # saves as "before-X"
# Compare current results against a saved baseline
make bench-compare # vs "main"
make bench-compare BENCH_BASELINE=before-X
# Run all benches inside Docker (reproducible)
make bench
Criterion writes HTML reports to target/criterion/report/index.html
for visual diffing.
CI regression gate
.github/workflows/bench.yml runs every PR and push:
- On
main: results are pushed to thegh-pagesbranch as the new baseline. - On PRs: results are compared against that baseline.
- CPU benches (
parsing,validation): alert and fail past 250% - Filesystem bench (
path_lookup): alert past 500%, never fails
- CPU benches (
Thresholds account for GitHub-hosted runner variance (10-30% per-bench noise is normal on shared CPU). Tighten by running on a self-hosted runner with pinned hardware if needed.