gs_test

Framework for test implementation + test discovery in jai

Description

This Jai Module provies a means of writing tests and building a test executable that can be used in applications such as CI runners.

How It Works

1. You call gs_test.build_all_tests from your build script
2. gs_test scans your project directory for files ending in the specified extension (default is .test.jai)
3. For each test file found, gs_test outputs an executable
4. gs_test outputs a run_all_tests file

You can call run_all_tests in CI and it will output an error code if any tests fail.

To Try It Out:

1. Put gs_test somewhere where your jai compiler can find it. ie. C:\jai\modules\gs_test
2. Copy the two examples below into files in the same directory
3. Run jai -import_dir C:\jai\modules build.jai
4. Run ./tests/run_all_tests.sh (you might need to give it execute permissions first)

Example Usage:

build.jai

  #import "Compiler";
  gs_test :: #import "gs_test";

  #run build();
  build :: ()
  {
    w := compiler_create_workspace("Target Program");
    target_options := get_build_options(w);

    mc := gs_test.Config.{
      // See gs_test/module.jai::Config for information on how to configure things further
      root_path = #filepath
    }
    gs_test.build_all_tests(mc, target_options);

    set_build_options_dc(.{ do_output = false });
  }

main.test.jai

main :: () {
  Init_Test_Harness();

  Test("my test", () {
    expect(5, 5); // succeeds
    expect(5, 3); // fails
  });

  Run_Test_Harness();
}

Why One Executable Per Test File?

Because of the way Jai handles imports, we have to either have a single test executable which runs all your tests, or make an executable per test file. In the former case, we'd be forced to do a bunch of complicated things to deduplicate #load calls in the cases where you have separate tests importing the same files. This solution seemed simpler - you define your tests how you want, and for each file, just import what that test requires. I may revist this in the future.

Goals

1. Simply import the test framework into your build pipeline, pass compiler messages to it, and get tests out.
2. Easily write tests, and get useful failure information back out.
3. Reasonable Defaults:

• creates a single test executable next to your output
• runs all tests, returning failure codes on exit
• tests can be placed in *.test.jai files or inline with @test tags.

4. Customization

• name the output file
• define the pattern for test files (ie. instead of *.test.jai, use something else)
• define the test tag (ie. instead of @test, use something else)