Test.Runner

This is an "experts only" module that exposes functions needed to run and display tests. A typical user will use an existing runner library for Node or the browser, which is implemented using this interface. A list of these runners can be found in the README.

Runner

type alias Runner = { run : {} -> Array Expectation, labels : Array String }

A function which, when evaluated, produces a list of expectations. Also a list of labels which apply to this outcome.
type SeededRunners
= Plain (Array Runner)
| Only (Array Runner)
| Skipping (Array Runner)
| Invalid String

Test Runners which have had seeds distributed to them, and which are now either invalid or are ready to run. Seeded runners include some metadata:

  • Invalid runners had a problem (e.g. two sibling tests had the same description) making them un-runnable.
  • Only runners can be run, but Test.only was used somewhere, so ultimately they will lead to a failed test run even if each test that gets run passes.
  • Skipping runners can be run, but Test.skip was used somewhere, so ultimately they will lead to a failed test run even if each test that gets run passes.
  • Plain runners are ready to run, and have none of these issues.
fromTest : Int -> Seed -> Test -> SeededRunners

Convert a Test into SeededRunners.

In order to run any fuzz tests that the Test may have, it requires a default run count as well as an initial Random.Seed. 100 is a good run count. To obtain a good random seed, pass a random 32-bit integer to Random.initialSeed. You can obtain such an integer by running Math.floor(Math.random()*0xFFFFFFFF) in Node. It's typically fine to hard-code this value into your Gren code; it's easy and makes your tests reproducible.

Expectations

getFailureReason :
Expectation
-> Maybe { given : Maybe String, description : String , reason : Reason
}

Return Nothing if the given Expectation is a pass.

If it is a fail, return a record containing the expectation description, the Reason the test failed, and the given inputs if it was a fuzz test. (If it was not a fuzz test, the record's given field will be Nothing).

For example:

getFailureReason (Expect.equal 1 2)
-- Just { reason = Equal 1 2, description = "Expect.equal", given = Nothing }

getFailureReason (Expect.equal 1 1)
-- Nothing
isTodo : Expectation -> Bool

Determine if an expectation was created by a call to Test.todo. Runners may treat these tests differently in their output.

Distribution

getDistributionReport : Expectation -> DistributionReport

Returns a DistributionReport computed for a given test.

Formatting

formatLabels :
(String -> format)
-> (String -> format)
-> Array String
-> Array format

A standard way to format descriptions and test labels, to keep things consistent across test runner implementations.

The HTML, Node, String, and Log runners all use this.

What it does:

  • drop any labels that are empty strings
  • format the first label differently from the others
  • reverse the resulting list

Example:

[ "the actual test that failed"
, "nested description failure"
, "top-level description failure"
]
|> formatLabels ((++) "↓ ") ((++) "✗ ")

{-
[ "↓ top-level description failure"
, "↓ nested description failure"
, "✗ the actual test that failed"
]
-}

Fuzzers

These functions give you the ability to run fuzzers separate of running fuzz tests.

type Simplifiable a

A Simplifiable a is an opaque type that allows you to obtain a value of type a that is simpler than the one you've previously obtained.
fuzz :
Fuzzer a
-> Generator (Result String { value : a, simplifiable : Simplifiable a })

Given a fuzzer, return a random generator to produce a value and a Simplifiable. The value is what a fuzz test would have received as input.

Note that fuzzers aren't generated to succeed, which is why this function returns a Result. The String inside the Err case will contain a failure reason.
simplify :
(a -> Expectation)
-> { value : a, simplifiable : Simplifiable a }
-> Maybe { value : a, simplifiable : Simplifiable a
}

Given a Simplifiable, simplify the value further. Pass your test function to drive the simplification process: if a simplified value passes the test, it will be discarded. In this sense, you will get the simplest value that still fails your test.