Expect

A library to create Expectations, which describe a claim to be tested.

Quick Reference

Basic Expectations

type alias Expectation = Expectation

The result of a single test run: either a pass or a fail.

equal : a -> a -> Expectation

Passes if the arguments are equal.

Expect.equal 0 (Array.length [])

-- Passes because (0 == 0) is True

Failures resemble code written in pipeline style, so you can tell which argument is which:

-- Fails because the expected value didn't split the space in "Betty Botter"
String.split " " "Betty Botter bought some butter"
    |> Expect.equal [ "Betty Botter", "bought", "some", "butter" ]

{-

[ "Betty", "Botter", "bought", "some", "butter" ]
╷
│ Expect.equal
╵
[ "Betty Botter", "bought", "some", "butter" ]

-}

Do not equate Float values; use within instead.

notEqual : a -> a -> Expectation

Passes if the arguments are not equal.

-- Passes because (11 /= 100) is True
90 + 10
    |> Expect.notEqual 11


-- Fails because (100 /= 100) is False
90 + 10
    |> Expect.notEqual 100

{-

100
╷
│ Expect.notEqual
╵
100

-}
all : Array (subject -> Expectation) -> subject -> Expectation

Passes if each of the given functions passes when applied to the subject.

Passing an empty list is assumed to be a mistake, so Expect.all [] will always return a failed expectation no matter what else it is passed.

Expect.all
    [ Expect.greaterThan -2
    , Expect.lessThan 5
    ]
    (Array.length [])
-- Passes because (0 > -2) is True and (0 < 5) is also True

Failures resemble code written in pipeline style, so you can tell which argument is which:

-- Fails because (0 < -10) is False
Array.length []
    |> Expect.all
        [ Expect.greaterThan -2
        , Expect.lessThan -10
        , Expect.equal 0
        ]
{-
0
╷
│ Expect.lessThan
╵
-10
-}

Numeric Comparisons

lessThan : comparable -> comparable -> Expectation

Passes if the second argument is less than the first.

Expect.lessThan 1 (Array.length [])

-- Passes because (0 < 1) is True

Failures resemble code written in pipeline style, so you can tell which argument is which:

-- Fails because (0 < -1) is False
Array.length []
    |> Expect.lessThan -1


{-

0
╷
│ Expect.lessThan
╵
-1

-}

Do not equate Float values; use notWithin instead.

atMost : comparable -> comparable -> Expectation

Passes if the second argument is less than or equal to the first.

Expect.atMost 1 (Array.length [])

-- Passes because (0 <= 1) is True

Failures resemble code written in pipeline style, so you can tell which argument is which:

-- Fails because (0 <= -3) is False
Array.length []
    |> Expect.atMost -3

{-

0
╷
│ Expect.atMost
╵
-3

-}
greaterThan : comparable -> comparable -> Expectation

Passes if the second argument is greater than the first.

Expect.greaterThan -2 Array.length []

-- Passes because (0 > -2) is True

Failures resemble code written in pipeline style, so you can tell which argument is which:

-- Fails because (0 > 1) is False
Array.length []
    |> Expect.greaterThan 1

{-

0
╷
│ Expect.greaterThan
╵
1

-}
atLeast : comparable -> comparable -> Expectation

Passes if the second argument is greater than or equal to the first.

Expect.atLeast -2 (Array.length [])

-- Passes because (0 >= -2) is True

Failures resemble code written in pipeline style, so you can tell which argument is which:

-- Fails because (0 >= 3) is False
Array.length []
    |> Expect.atLeast 3

{-

0
╷
│ Expect.atLeast
╵
3

-}

Floating Point Comparisons

These functions allow you to compare Float values up to a specified rounding error, which may be relative, absolute, or both. For an in-depth look, see our Guide to Floating Point Comparison.

type FloatingPointTolerance
= Absolute Float
| Relative Float
| AbsoluteOrRelative Float Float

A type to describe how close a floating point number must be to the expected value for the test to pass. This may be specified as absolute or relative.

AbsoluteOrRelative tolerance uses a logical OR between the absolute (specified first) and relative tolerance. If you want a logical AND, use Expect.all.

within : FloatingPointTolerance -> Float -> Float -> Expectation

Passes if the second and third arguments are equal within a tolerance specified by the first argument. This is intended to avoid failing because of minor inaccuracies introduced by floating point arithmetic.

-- Fails because 0.1 + 0.2 == 0.30000000000000004 (0.1 is non-terminating in base 2)
0.1 + 0.2 |> Expect.equal 0.3

-- So instead write this test, which passes
0.1 + 0.2 |> Expect.within (Absolute 0.000000001) 0.3

Failures resemble code written in pipeline style, so you can tell which argument is which:

-- Fails because 3.14 is not close enough to pi
3.14 |> Expect.within (Absolute 0.0001) pi

{-

3.14
╷
│ Expect.within Absolute 0.0001
╵
3.141592653589793

-}
notWithin : FloatingPointTolerance -> Float -> Float -> Expectation

Passes if (and only if) a call to within with the same arguments would have failed.

Collections

ok : Result a b -> Expectation

Passes if the Result is an Ok rather than Err. This is useful for tests where you expect not to see an error, but you don't care what the actual result is.

(Tip: If your function returns a Maybe instead, consider Expect.notEqual Nothing.)

-- Passes
String.toInt "20"
    |> Result.fromMaybe "not an int"
    |> Expect.ok

Test failures will be printed with the unexpected Err value contrasting with any Ok.

-- Fails
String.toInt "not an int"
    |> Result.fromMaybe "not an int"
    |> Expect.ok

{-

Err "not an int"
╷
│ Expect.ok
╵
Ok _

-}
err : Result a b -> Expectation

Passes if the Result is an Err rather than Ok. This is useful for tests where you expect to get an error but you don't care what the actual error is.

(Tip: If your function returns a Maybe instead, consider Expect.equal Nothing.)

-- Passes
String.toInt "not an int"
    |> Result.fromMaybe "not an int"
    |> Expect.err

Test failures will be printed with the unexpected Ok value contrasting with any Err.

-- Fails
String.toInt "20"
    |> Result.fromMaybe "not an int"
    |> Expect.err

{-

Ok 20
╷
│ Expect.err
╵
Err _

-}
equalArrays : Array a -> Array a -> Expectation

Passes if the arguments are equal lists.

-- Passes
[ 1, 2, 3 ]
    |> Expect.equalArrays [ 1, 2, 3 ]

Failures resemble code written in pipeline style, so you can tell which argument is which, and reports which index the lists first differed at or which list was longer:

-- Fails
[ 1, 2, 4, 6 ]
    |> Expect.equalArrays [ 1, 2, 5 ]

{-

[1,2,4,6]
first diff at index index 2: +`4`, -`5`
╷
│ Expect.equalArrays
╵
first diff at index index 2: +`5`, -`4`
[1,2,5]

-}
equalDicts : Dict comparable a -> Dict comparable a -> Expectation

Passes if the arguments are equal dicts.

-- Passes
Dict.fromArray [ ( 1, "one" ), ( 2, "two" ) ]
    |> Expect.equalDicts (Dict.fromArray [ ( 1, "one" ), ( 2, "two" ) ])

Failures resemble code written in pipeline style, so you can tell which argument is which, and reports which keys were missing from or added to each dict:

-- Fails
(Dict.fromArray [ ( 1, "one" ), ( 2, "too" ) ])
    |> Expect.equalDicts (Dict.fromArray [ ( 1, "one" ), ( 2, "two" ), ( 3, "three" ) ])

{-

Dict.fromArray [(1,"one"),(2,"too")]
diff: -[ (2,"two"), (3,"three") ] +[ (2,"too") ]
╷
│ Expect.equalDicts
╵
diff: +[ (2,"two"), (3,"three") ] -[ (2,"too") ]
Dict.fromArray [(1,"one"),(2,"two"),(3,"three")]

-}
equalSets : Set comparable -> Set comparable -> Expectation

Passes if the arguments are equal sets.

-- Passes
Set.fromArray [ 1, 2 ]
    |> Expect.equalSets (Set.fromArray [ 1, 2 ])

Failures resemble code written in pipeline style, so you can tell which argument is which, and reports which keys were missing from or added to each set:

-- Fails
(Set.fromArray [ 1, 2, 4, 6 ])
    |> Expect.equalSets (Set.fromArray [ 1, 2, 5 ])

{-

Set.fromArray [1,2,4,6]
diff: -[ 5 ] +[ 4, 6 ]
╷
│ Expect.equalSets
╵
diff: +[ 5 ] -[ 4, 6 ]
Set.fromArray [1,2,5]

-}

Customizing

These functions will let you build your own expectations.

pass : Expectation

Always passes.

import Json.Decode exposing (decodeString, int)
import Test exposing (test)
import Expect


test "Json.Decode.int can decode the number 42." <|
    \_ ->
        case decodeString int "42" of
            Ok _ ->
                Expect.pass

            Err err ->
                Expect.fail err
fail : String -> Expectation

Fails with the given message.

import Json.Decode exposing (decodeString, int)
import Test exposing (test)
import Expect


test "Json.Decode.int can decode the number 42." <|
    \_ ->
        case decodeString int "42" of
            Ok _ ->
                Expect.pass

            Err err ->
                Expect.fail err
onFail : String -> Expectation -> Expectation

If the given expectation fails, replace its failure message with a custom one.

"something"
    |> Expect.equal "something else"
    |> Expect.onFail "thought those two strings would be the same"

Guide to Floating Point Comparison

In general, if you are multiplying, you want relative tolerance, and if you're adding, you want absolute tolerance. If you are doing both, you want both kinds of tolerance, or to split the calculation into smaller parts for testing.

Absolute Tolerance

Let's say we want to figure out if our estimation of pi is precise enough.

Is 3.14 within 0.01 of pi? Yes, because 3.13 < pi < 3.15.

test "3.14 approximates pi with absolute precision" <|
    \_ ->
        3.14 |> Expect.within (Absolute 0.01) pi

Relative Tolerance

What if we also want to know if our circle circumference estimation is close enough?

Let's say our circle has a radius of r meters. The formula for circle circumference is C=2*r*pi. To make the calculations a bit easier (ahem), we'll look at half the circumference; C/2=r*pi. Is r * 3.14 within 0.01 of r * pi? That depends, what does r equal? If r is 0.01mm, or 0.00001 meters, we're comparing 0.00001 * 3.14 - 0.01 < r * pi < 0.00001 * 3.14 + 0.01 or -0.0099686 < 0.0000314159 < 0.0100314. That's a huge tolerance! A circumference that is a thousand times longer than we expected would pass that test!

On the other hand, if r is very large, we're going to need many more digits of pi. For an absolute tolerance of 0.01 and a pi estimation of 3.14, this expectation only passes if r < 2*pi.

If we use a relative tolerance of 0.01 instead, the circle area comparison becomes much better. Is r * 3.14 within 1% of r * pi? Yes! In fact, three digits of pi approximation is always good enough for a 0.1% relative tolerance, as long as r isn't too close to zero.

fuzz
    (floatRange 0.000001 100000)
    "Circle half-circumference with relative tolerance"
    (\r -> r * 3.14 |> Expect.within (Relative 0.001) (r * pi))

Trouble with Numbers Near Zero

If you are adding things near zero, you probably want absolute tolerance. If you're comparing values between -1 and 1, you should consider using absolute tolerance.

For example: Is 1 + 2 - 3 within 1% of 0? Well, if 1, 2 and 3 have any amount of rounding error, you might not get exactly zero. What is 1% above and below 0? Zero. We just lost all tolerance. Even if we hard-code the numbers, we might not get exactly zero; 0.1 + 0.2 rounds to a value just above 0.3, since computers, counting in binary, cannot write down any of those three numbers using a finite number of digits, just like we cannot write 0.333... exactly in base 10.

Another example is comparing values that are on either side of zero. 0.0001 is more than 100% away from -0.0001. In fact, infinity is closer to 0.0001 than 0.0001 is to -0.0001, if you are using a relative tolerance. Twice as close, actually. So even though both 0.0001 and -0.0001 could be considered very close to zero, they are very far apart relative to each other. The same argument applies for any number of zeroes.