Writing Automated Tests

In his 1972 essay "The Humble Programmer," Edsger W. Dijkstra said that "program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence." That doesn't mean we shouldn't try to test as much as we can!

Correctness in our programs is the extent to which our code does what we intend it to do. Auto is designed with a high degree of concern about correctness — the type system, ownership rules, and borrowing checker catch many bugs at compile time. But the type system cannot catch everything. That's where automated tests come in.

Auto's testing system is closely aligned with Rust's, since Auto transpiles to Rust and runs tests using the same test runner. This chapter covers the testing mechanics available to you in Auto.

How to Write Tests

Tests are functions that verify non-test code is functioning as expected. Test functions typically perform these three actions:

1. Set up any needed data or state.
2. Run the code you want to test.
3. Assert that the results are what you expect.

Structuring Test Functions

A test in Auto is a function annotated with #[test]. When you run automan test, the transpiler compiles your code to Rust and runs cargo test, which builds a test runner binary that executes the annotated functions.

A test function using `assert_eq!`

The #[test] attribute marks the function as a test. The assert_eq! macro checks that two values are equal. If they're not, the test panics and fails.

Checking Results with assert!

The assert! macro verifies that a condition is true. If the condition is false, the test fails:

#[test]
fn larger_can_hold_smaller() {
    let larger = Rectangle(8, 7)
    let smaller = Rectangle(5, 1)
    assert(larger.can_hold(&smaller))
}

Adding Custom Failure Messages

You can add custom messages to assert! and assert_eq! to make failures easier to debug:

A test with a custom failure message

When this test fails, the custom message is printed, showing the actual value of result — making it much easier to debug.

Checking for Panics with #[should_panic]

To test that code panics under certain conditions, use the #[should_panic] attribute:

#[test]
#[should_panic]
fn it_panics() {
    panic("expected panic")
}

You can also check for a specific panic message with expected:

#[test]
#[should_panic(expected = "less than or equal to 100")]
fn greater_than_100() {
    Guess.new(200)
}

Using !T (Result) in Tests

Tests can return !T (Result) instead of panicking. This lets you use .! (error propagation) inside test functions:

Tests with `#[should_panic]` and `!T` return type

Note: fn valid_guess() ! transpiles to fn valid_guess() -> Result<(), String>. This lets you use ? (via .!) for error propagation inside tests.

Test Syntax Quick Reference

Auto	Rust	Purpose
#[test]	#[test]	Mark function as a test
assert_eq(a, b)	assert_eq!(a, b)	Assert equality
assert_ne(a, b)	assert_ne!(a, b)	Assert inequality
assert(cond)	assert!(cond)	Assert condition is true
#[should_panic]	#[should_panic]	Test should panic
#[should_panic(expected = "...")]	#[should_panic(expected = "...")]	Panic with specific message
fn test() !	fn test() -> Result<(), String>	Return errors instead of panicking

Controlling How Tests Are Run

Run tests with automan test (which invokes cargo test on the transpiled Rust code):

$ automan test

Running Tests in Parallel or Consecutively

Tests run in parallel by default. To run them one at a time:

$ automan test -- --test-threads=1

Showing Output

To see print() output from passing tests:

$ automan test -- --show-output

Running a Subset of Tests

Run tests whose names contain a pattern:

$ automan test add       # Run all tests with "add" in the name

Run a single test:

$ automan test it_adds_two

Ignoring Expensive Tests

Use #[ignore] to skip time-consuming tests:

#[test]
#[ignore]
fn expensive_test() {
    // Code that takes a long time
}

Run only ignored tests:

$ automan test -- --ignored

Test Organization

Auto follows the same testing organization as Rust:

Unit Tests

Unit tests go in the same file as the code being tested, inside a tests module annotated with #[cfg(test)]:

Needs modules

This example depends on multi-file modules and cannot run inline — open it in the Playground

fn internal_adder(a int, b int) int {
    a + b
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn internal() {
        assert_eq(internal_adder(2, 2), 4)
    }
}

Unit tests can test private functions because the tests module is a child module with access to the parent's private items via use super::*.

Integration Tests

Integration tests go in a tests/ directory at the project root. Each file in tests/ is compiled as a separate crate, testing only the public API:

my_project/
├── auto.toml
├── src/
│   └── lib.rs
└── tests/
    └── integration_test.rs

Integration tests use use to import from your library:

Needs modules

This example depends on multi-file modules and cannot run inline — open it in the Playground

use my_project::add_two

#[test]
fn it_adds_two() {
    assert_eq(add_two(2), 4)
}

Summary

Auto's testing system leverages Rust's mature test infrastructure through transpilation. The key points:

Concept	Auto	Command
Run all tests	—	automan test
Single test	#[test]	automan test test_name
Filter tests	—	automan test pattern
Single-threaded	—	automan test -- --test-threads=1
Show output	—	automan test -- --show-output
Run ignored	#[ignore]	automan test -- --ignored

The testing macros (assert_eq!, assert!, assert_ne!) work identically in Auto and Rust. Auto's fn test() ! provides a clean way to write tests that propagate errors using the same !T syntax you use throughout your code.

Now let's combine the knowledge from previous chapters to build a project!