Expect
A library to create Expectation
s, which describe a claim to be tested.
Quick Reference
equal
(arg2 == arg1)
notEqual
(arg2 /= arg1)
lessThan
(arg2 < arg1)
atMost
(arg2 <= arg1)
greaterThan
(arg2 > arg1)
atLeast
(arg2 >= arg1)
- Floating Point Comparisons
Basic Expectations
Passes if the arguments are equal.
Expect.equal 0 (Array.length [])
-- Passes because (0 == 0) is True
Failures resemble code written in pipeline style, so you can tell which argument is which:
-- Fails because the expected value didn't split the space in "Betty Botter"
String.split " " "Betty Botter bought some butter"
|> Expect.equal [ "Betty Botter", "bought", "some", "butter" ]
{-
[ "Betty", "Botter", "bought", "some", "butter" ]
╷
│ Expect.equal
╵
[ "Betty Botter", "bought", "some", "butter" ]
-}
Do not equate Float
values; use within
instead.
Passes if the arguments are not equal.
-- Passes because (11 /= 100) is True
90 + 10
|> Expect.notEqual 11
-- Fails because (100 /= 100) is False
90 + 10
|> Expect.notEqual 100
{-
100
╷
│ Expect.notEqual
╵
100
-}
Passes if each of the given functions passes when applied to the subject.
Passing an empty list is assumed to be a mistake, so Expect.all []
will always return a failed expectation no matter what else it is passed.
Expect.all
[ Expect.greaterThan -2
, Expect.lessThan 5
]
(Array.length [])
-- Passes because (0 > -2) is True and (0 < 5) is also True
Failures resemble code written in pipeline style, so you can tell which argument is which:
-- Fails because (0 < -10) is False
Array.length []
|> Expect.all
[ Expect.greaterThan -2
, Expect.lessThan -10
, Expect.equal 0
]
{-
0
╷
│ Expect.lessThan
╵
-10
-}
Numeric Comparisons
Passes if the second argument is less than the first.
Expect.lessThan 1 (Array.length [])
-- Passes because (0 < 1) is True
Failures resemble code written in pipeline style, so you can tell which argument is which:
-- Fails because (0 < -1) is False
Array.length []
|> Expect.lessThan -1
{-
0
╷
│ Expect.lessThan
╵
-1
-}
Do not equate Float
values; use notWithin
instead.
Passes if the second argument is less than or equal to the first.
Expect.atMost 1 (Array.length [])
-- Passes because (0 <= 1) is True
Failures resemble code written in pipeline style, so you can tell which argument is which:
-- Fails because (0 <= -3) is False
Array.length []
|> Expect.atMost -3
{-
0
╷
│ Expect.atMost
╵
-3
-}
Passes if the second argument is greater than the first.
Expect.greaterThan -2 Array.length []
-- Passes because (0 > -2) is True
Failures resemble code written in pipeline style, so you can tell which argument is which:
-- Fails because (0 > 1) is False
Array.length []
|> Expect.greaterThan 1
{-
0
╷
│ Expect.greaterThan
╵
1
-}
Passes if the second argument is greater than or equal to the first.
Expect.atLeast -2 (Array.length [])
-- Passes because (0 >= -2) is True
Failures resemble code written in pipeline style, so you can tell which argument is which:
-- Fails because (0 >= 3) is False
Array.length []
|> Expect.atLeast 3
{-
0
╷
│ Expect.atLeast
╵
3
-}
Floating Point Comparisons
These functions allow you to compare Float
values up to a specified rounding error, which may be relative, absolute,
or both. For an in-depth look, see our Guide to Floating Point Comparison.
A type to describe how close a floating point number must be to the expected value for the test to pass. This may be specified as absolute or relative.
AbsoluteOrRelative
tolerance uses a logical OR between the absolute (specified first) and relative tolerance. If you
want a logical AND, use Expect.all
.
Passes if the second and third arguments are equal within a tolerance specified by the first argument. This is intended to avoid failing because of minor inaccuracies introduced by floating point arithmetic.
-- Fails because 0.1 + 0.2 == 0.30000000000000004 (0.1 is non-terminating in base 2)
0.1 + 0.2 |> Expect.equal 0.3
-- So instead write this test, which passes
0.1 + 0.2 |> Expect.within (Absolute 0.000000001) 0.3
Failures resemble code written in pipeline style, so you can tell which argument is which:
-- Fails because 3.14 is not close enough to pi
3.14 |> Expect.within (Absolute 0.0001) pi
{-
3.14
╷
│ Expect.within Absolute 0.0001
╵
3.141592653589793
-}
Passes if (and only if) a call to within
with the same arguments would have failed.
Collections
Passes if the
Result
is
an Ok
rather than Err
. This is useful for tests where you expect not to see
an error, but you don't care what the actual result is.
(Tip: If your function returns a Maybe
instead, consider Expect.notEqual Nothing
.)
-- Passes
String.toInt "20"
|> Result.fromMaybe "not an int"
|> Expect.ok
Test failures will be printed with the unexpected Err
value contrasting with
any Ok
.
-- Fails
String.toInt "not an int"
|> Result.fromMaybe "not an int"
|> Expect.ok
{-
Err "not an int"
╷
│ Expect.ok
╵
Ok _
-}
Passes if the
Result
is
an Err
rather than Ok
. This is useful for tests where you expect to get an
error but you don't care what the actual error is.
(Tip: If your function returns a Maybe
instead, consider Expect.equal Nothing
.)
-- Passes
String.toInt "not an int"
|> Result.fromMaybe "not an int"
|> Expect.err
Test failures will be printed with the unexpected Ok
value contrasting with
any Err
.
-- Fails
String.toInt "20"
|> Result.fromMaybe "not an int"
|> Expect.err
{-
Ok 20
╷
│ Expect.err
╵
Err _
-}
Passes if the arguments are equal lists.
-- Passes
[ 1, 2, 3 ]
|> Expect.equalArrays [ 1, 2, 3 ]
Failures resemble code written in pipeline style, so you can tell which argument is which, and reports which index the lists first differed at or which list was longer:
-- Fails
[ 1, 2, 4, 6 ]
|> Expect.equalArrays [ 1, 2, 5 ]
{-
[1,2,4,6]
first diff at index index 2: +`4`, -`5`
╷
│ Expect.equalArrays
╵
first diff at index index 2: +`5`, -`4`
[1,2,5]
-}
Passes if the arguments are equal dicts.
-- Passes
Dict.fromArray [ ( 1, "one" ), ( 2, "two" ) ]
|> Expect.equalDicts (Dict.fromArray [ ( 1, "one" ), ( 2, "two" ) ])
Failures resemble code written in pipeline style, so you can tell which argument is which, and reports which keys were missing from or added to each dict:
-- Fails
(Dict.fromArray [ ( 1, "one" ), ( 2, "too" ) ])
|> Expect.equalDicts (Dict.fromArray [ ( 1, "one" ), ( 2, "two" ), ( 3, "three" ) ])
{-
Dict.fromArray [(1,"one"),(2,"too")]
diff: -[ (2,"two"), (3,"three") ] +[ (2,"too") ]
╷
│ Expect.equalDicts
╵
diff: +[ (2,"two"), (3,"three") ] -[ (2,"too") ]
Dict.fromArray [(1,"one"),(2,"two"),(3,"three")]
-}
Passes if the arguments are equal sets.
-- Passes
Set.fromArray [ 1, 2 ]
|> Expect.equalSets (Set.fromArray [ 1, 2 ])
Failures resemble code written in pipeline style, so you can tell which argument is which, and reports which keys were missing from or added to each set:
-- Fails
(Set.fromArray [ 1, 2, 4, 6 ])
|> Expect.equalSets (Set.fromArray [ 1, 2, 5 ])
{-
Set.fromArray [1,2,4,6]
diff: -[ 5 ] +[ 4, 6 ]
╷
│ Expect.equalSets
╵
diff: +[ 5 ] -[ 4, 6 ]
Set.fromArray [1,2,5]
-}
Customizing
These functions will let you build your own expectations.
Always passes.
import Json.Decode exposing (decodeString, int)
import Test exposing (test)
import Expect
test "Json.Decode.int can decode the number 42." <|
\_ ->
when decodeString int "42" is
Ok _ ->
Expect.pass
Err err ->
Expect.fail err
Fails with the given message.
import Json.Decode exposing (decodeString, int)
import Test exposing (test)
import Expect
test "Json.Decode.int can decode the number 42." <|
\_ ->
when decodeString int "42" is
Ok _ ->
Expect.pass
Err err ->
Expect.fail err
If the given expectation fails, replace its failure message with a custom one.
"something"
|> Expect.equal "something else"
|> Expect.onFail "thought those two strings would be the same"
Guide to Floating Point Comparison
In general, if you are multiplying, you want relative tolerance, and if you're adding, you want absolute tolerance. If you are doing both, you want both kinds of tolerance, or to split the calculation into smaller parts for testing.
Absolute Tolerance
Let's say we want to figure out if our estimation of pi is precise enough.
Is 3.14
within 0.01
of pi
? Yes, because 3.13 < pi < 3.15
.
test "3.14 approximates pi with absolute precision" <|
\_ ->
3.14 |> Expect.within (Absolute 0.01) pi
Relative Tolerance
What if we also want to know if our circle circumference estimation is close enough?
Let's say our circle has a radius of r
meters. The formula for circle circumference is C=2*r*pi
.
To make the calculations a bit easier (ahem), we'll look at half the circumference; C/2=r*pi
.
Is r * 3.14
within 0.01
of r * pi
?
That depends, what does r
equal? If r
is 0.01
mm, or 0.00001
meters, we're comparing
0.00001 * 3.14 - 0.01 < r * pi < 0.00001 * 3.14 + 0.01
or -0.0099686 < 0.0000314159 < 0.0100314
.
That's a huge tolerance! A circumference that is a thousand times longer than we expected would pass that test!
On the other hand, if r
is very large, we're going to need many more digits of pi.
For an absolute tolerance of 0.01
and a pi estimation of 3.14
, this expectation only passes if r < 2*pi
.
If we use a relative tolerance of 0.01
instead, the circle area comparison becomes much better. Is r * 3.14
within
1%
of r * pi
? Yes! In fact, three digits of pi approximation is always good enough for a 0.1% relative tolerance,
as long as r
isn't too close to zero.
fuzz
(floatRange 0.000001 100000)
"Circle half-circumference with relative tolerance"
(\r -> r * 3.14 |> Expect.within (Relative 0.001) (r * pi))
Trouble with Numbers Near Zero
If you are adding things near zero, you probably want absolute tolerance. If you're comparing values between -1
and 1
, you should consider using absolute tolerance.
For example: Is 1 + 2 - 3
within 1%
of 0
? Well, if 1
, 2
and 3
have any amount of rounding error, you might not get exactly zero. What is 1%
above and below 0
? Zero. We just lost all tolerance. Even if we hard-code the numbers, we might not get exactly zero; 0.1 + 0.2
rounds to a value just above 0.3
, since computers, counting in binary, cannot write down any of those three numbers using a finite number of digits, just like we cannot write 0.333...
exactly in base 10.
Another example is comparing values that are on either side of zero. 0.0001
is more than 100%
away from -0.0001
. In fact, infinity
is closer to 0.0001
than 0.0001
is to -0.0001
, if you are using a relative tolerance. Twice as close, actually. So even though both 0.0001
and -0.0001
could be considered very close to zero, they are very far apart relative to each other. The same argument applies for any number of zeroes.