gs_test

Framework for test implementation + test discovery in jai

Description

This Jai Module provies a means of writing tests and building a test executable that can be used in applications such as CI runners.

Test Executable Reference

Test executables can be run individually or via run_all_tests.sh which is output in the same directory. Both tests and run_all_tests.sh have the same options:

-colors: output colored information for easier readability

-verbose: by default the output will only show test suites run, and full information on errors. Enabling verbose output indicates all tests that are run as well.

Test Harness Reference

Init_Test_Harness(suite_name: string): Call at the beginning of your test suite function to initialize the test suite.

Run_Test_Harness(): Call at the end of your test suite function to execute all tests

Before_Each(#type () -> ()): Define a function that will be run before each test

After_Each(#type () -> ()): Define a funciton that will be run after each test

Test(test_name: string, test_proc: #type () -> ()): Define a single test function

expect(received, expected): Checks for equality between the two values. Equivalent to if (received != expected) report_test_failure()

expect_true(value)

expect_false(value)

How It Works

1. You call gs_test.build_all_tests from your build script
2. gs_test scans your project directory for files ending in the specified extension (default is .test.jai)
3. For each test file found, gs_test outputs an executable
4. gs_test outputs a run_all_tests file

You can call run_all_tests in CI and it will output an error code if any tests fail.

To Try It Out:

1. Put gs_test somewhere where your jai compiler can find it. ie. C:\jai\modules\gs_test
2. Copy the two examples below into files in a sample project directory
3. Run jai -import_dir C:\jai\modules build.jai
4. Run ./tests/run_all_tests.sh (you might need to give it execute permissions first)

Example Usage:

build.jai

  #import "Compiler";
  gs_test :: #import "gs_test";

  #run build();
  build :: ()
  {
    w := compiler_create_workspace("Target Program");
    target_options := get_build_options(w);

    mc := gs_test.Config.{
      // See gs_test/module.jai::Config for information on how to configure things further
      root_path = #filepath
    }
    gs_test.build_all_tests(mc, target_options);

    set_build_options_dc(.{ do_output = false });
  }

main.test.jai

main :: () {
  Init_Test_Harness();

  Test("my test", () {
    expect(5, 5); // succeeds
    expect(5, 3); // fails
  });

  Run_Test_Harness();
}

Why One Executable Per Test File?

Because of the way Jai handles imports, we have to either have a single test executable which runs all your tests, or make an executable per test file. In the former case, we'd be forced to do a bunch of complicated things to deduplicate #load calls in the cases where you have separate tests importing the same files. This solution seemed simpler - you define your tests how you want, and for each file, just import what that test requires. I may revist this in the future.