Fuzz

This is a library of fuzzers you can use to supply values to your fuzz tests. You can typically pick out which ones you need according to their types.

A Fuzzer a knows how to create values of type a. It can create them randomly, so that your test's expectations are run against many values. Fuzzers will often generate edge cases likely to find bugs. If the fuzzer can make your test fail, the test runner also knows how to "simplify" that failing input into more minimal examples, some of which might also cause the tests to fail. In this way, fuzzers can usually find the simplest input that reproduces a bug.

Fuzzers

type alias Fuzzer a = Fuzzer a

The representation of fuzzers is opaque. Conceptually, a Fuzzer a consists of a way to randomly generate values of type a in a way allowing the test runner to simplify those values.

examples : Int -> Fuzzer a -> Array a

Generate a few example values from the fuzzer.

Useful in REPL:

> import Fuzz
> Fuzz.examples 20 (Fuzz.intRange 20 50)
[42,45,32,26,33,29,41,45,23,45,34,23,22,42,29,27,41,43,30,50]
    : Array Int

Uses the first argument as the seed as well as the count of examples to generate.

Will return an empty array in case of rejection.

labelExamples :
Int
-> Array { label : String, predicate : a -> Bool }
-> Fuzzer a
-> Array { labels : Array String, value : Maybe a
}

Show examples of values satisfying given classification predicates (see also Test.reportDistribution and Test.expectDistribution).

Generates a given number of values and classifies them based on the predicates.

Uses the first argument as the seed as well as the count of examples to generate.

This function will always return all the given "base" labels, even if no examples of them could be found:

Fuzz.labelExamples 100
    [ ( "Lower boundary (1)", \n -> n == 1 )
    , ( "Upper boundary (20)", \n -> n == 20 )
    , ( "In the middle (2..19)", \n -> n > 1 && n < 20 )
    , ( "Outside boundaries??", \n -> n < 1 || n > 20 )
    ]
    (Fuzz.intRange 1 20)

-->
[ ( [ "Lower boundary (1)" ], Just 1 )
, ( [ "Upper boundary (20)" ], Just 20 )
, ( [ "In the middle (2..19)" ], Just 5 )
, ( [ "Outside boundaries??" ], Nothing )
]

In case of predicate overlap (eg. something is both green and big) this function will also return all the found combinations:

Fuzz.labelExamples 100
    [ ( "fizz", \n -> (n |> Math.modBy 3) == 0 )
    , ( "buzz", \n -> (n |> Math.modBy 5) == 0 )
    ]
    (Fuzz.intRange 1 20)

-->
[ ( [ "fizz" ], Just 3 )
, ( [ "buzz" ], Just 10 )
, ( [ "fizz, buzz" ], Just 15 )
]

Number fuzzers

int : Fuzzer Int

A fuzzer for int values. It will never produce NaN, Infinity, or -Infinity.

This fuzzer will generate values in the range Random.minInt .. Random.maxInt.

  • Simplifies towards 0
  • Prefers positive values over negative ones
  • Prefers smaller values over larger ones
intRange : Int -> Int -> Fuzzer Int

A fuzzer for int values between a given minimum and maximum value, inclusive. Shrunk values will also be within the range.

uniformInt : Int -> Fuzzer Int

Draw an integer between 0 and n inclusive.

Will simplify towards 0, but draws uniformly over the whole range.

Max supported value is 2^32 - 1.

intAtLeast : Int -> Fuzzer Int

A fuzzer that will generate values in range n..2^32-1.

intAtMost : Int -> Fuzzer Int

A fuzzer that will generate values in range -(2^32-1)..n.

float : Fuzzer Float

A fuzzer for float values.

Will prefer integer values, nice fractions and positive numbers over the rest.

Will occasionally try infinities and NaN. If you don't want to generate these, use Fuzz.niceFloat.

niceFloat : Fuzzer Float

A fuzzer for float values.

Will prefer integer values, nice fractions and positive numbers over the rest.

Will never try infinities or NaN.

percentage : Fuzzer Float

A fuzzer for percentage values. Generates random floats between 0.0 inclusive and 1.0 exclusive, in an uniform fashion.

Will occasionally try the boundaries.

Doesn't shrink to nice values like Fuzz.float does; shrinks towards zero.

floatRange : Float -> Float -> Fuzzer Float

A fuzzer for float values within between a given minimum and maximum (inclusive).

Shrunken values will also be within the range.

floatAtLeast : Float -> Fuzzer Float

Fuzzer generating floats in range n..Infinity.

The positive part of the range will shrink nicely, the negative part will shrink uniformly.

The fuzzer will occasionally try the minimum, 0 (if in range) and Infinity.

floatAtMost : Float -> Fuzzer Float

Fuzzer generating floats in range -Infinity..n.

The negative part of the range will shrink nicely, the positive part will shrink uniformly.

The fuzzer will occasionally try the maximum, 0 (if in range) and -Infinity.

String-related fuzzers

char : Fuzzer Char

A fuzzer for arbitrary Unicode char values.

Avoids surrogate pairs or their components (0xD800..0xDFFF).

Will prefer ASCII characters, whitespace, and some examples known to cause trouble, like combining diacritics marks and emojis.

asciiChar : Fuzzer Char

A fuzzer for simple ASCII char values (range 32..126). Skips control characters and the extended character set.

For more serious char fuzzing look at Fuzz.char which generates the whole Unicode range.

string : Fuzzer String

Generates random unicode strings of up to 10 characters.

stringOfLength : Int -> Fuzzer String

Generates random unicode strings of a given length.

Note that some unicode characters have String.length of 2. This fuzzer will make sure the String.length of the returned string is equal to the wanted length, even if it will mean there are less characters. If you instead want it to give N characters even if their String.length will be above N, you can use

Fuzz.arrayOfLength n Fuzz.char
    |> Fuzz.map String.fromArray
stringOfLengthBetween : Int -> Int -> Fuzzer String

Generates random unicode strings of length between the given limits.

Note that some unicode characters have String.length of 2. This fuzzer will make sure the String.length of the returned string is equal to the wanted length, even if it will mean there are less characters. If you instead want it to give between MIN and MAX characters even if their String.length will be above MAX, you can use

Fuzz.arrayOfLengthBetween min max Fuzz.char
    |> Fuzz.map String.fromArray
asciiString : Fuzzer String

Generates random ASCII strings of up to 10 characters.

asciiStringOfLength : Int -> Fuzzer String

Generates random ASCII strings of a given length.

asciiStringOfLengthBetween : Int -> Int -> Fuzzer String

Generates random ASCII strings of length between the given limits.

Collection fuzzers

pair : Fuzzer a -> Fuzzer b -> Fuzzer { first : a, second : b }

Create a fuzzer of pairs from two fuzzers.

triple :
Fuzzer a
-> Fuzzer b
-> Fuzzer c
-> Fuzzer { first : a, second : b , third : c
}

Create a fuzzer of triples from three fuzzers.

array : Fuzzer a -> Fuzzer (Array a)

Given a fuzzer of a type, create a fuzzer of a array of that type. Generates random arrays of varying length, up to 32 elements.

arrayOfLength : Int -> Fuzzer a -> Fuzzer (Array a)

Given a fuzzer of a type, create a fuzzer of a array of that type. Generates random arrays of exactly the specified length.

arrayOfLengthBetween : Int -> Int -> Fuzzer a -> Fuzzer (Array a)

Given a fuzzer of a type, create a fuzzer of a array of that type. Generates random arrays of length between the two given integers.

shuffledArray : Array a -> Fuzzer (Array a)

A fuzzer that shuffles the given array.

maybe : Fuzzer a -> Fuzzer (Maybe a)

Given a fuzzer of a type, create a fuzzer of a maybe for that type.

result : Fuzzer error -> Fuzzer value -> Fuzzer (Result error value)

Given fuzzers for an error type and a success type, create a fuzzer for a result.

Other fuzzers

bool : Fuzzer Bool

A fuzzer for boolean values. It's useful when building up fuzzers of complex types that contain a boolean somewhere.

We recommend against writing tests fuzzing over booleans. Write a unit test for the true and false cases explicitly.

Simplifies in order False < True.

unit : Fuzzer {}

A fuzzer for the unit value. Unit is a type with only one value, commonly used as a placeholder.

order : Fuzzer Order

A fuzzer for order values.

Simplifies in order LT < EQ < GT.

weightedBool : Float -> Fuzzer Bool

A fuzzer for boolean values, generating True with the given probability (0.0 = always False, 1.0 = always True).

Probabilities outside the 0..1 range will be clamped to 0..1.

Simplifies towards False (if not prevented to do that by using probability >= 1).

Choosing fuzzers

oneOf : Array (Fuzzer a) -> Fuzzer a

Choose one of the given fuzzers at random. Each fuzzer has an equal chance of being chosen; to customize the probabilities, use Fuzz.frequency.

This fuzzer will simplify towards the fuzzers earlier in the array (each of which will also apply its own way to simplify the values).

Fuzz.oneOf
    [ Fuzz.intRange 0 3
    , Fuzz.intRange 7 9
    ]
oneOfValues : Array a -> Fuzzer a

Choose one of the given values at random. Each value has an equal chance of being chosen; to customize the probabilities, use Fuzz.frequencyValues.

This fuzzer will simplify towards the values earlier in the array.

Fuzz.oneOfValues
    [ 999
    , -42
    ]
frequency : Array { weight : Float, fuzzer : Fuzzer a } -> Fuzzer a

Create a new Fuzzer by providing a array of probabilistic weights to use with other fuzzers. For example, to create a Fuzzer that has a 1/4 chance of generating an int between -1 and -100, and a 3/4 chance of generating one between 1 and 100, you could do this:

Fuzz.frequency
    [ ( 1, Fuzz.intRange -100 -1 )
    , ( 3, Fuzz.intRange 1 100 )
    ]

This fuzzer will simplify towards the fuzzers earlier in the array (each of which will also apply its own way to simplify the values).

There are a few circumstances in which this function will return an invalid fuzzer, which causes it to fail any test that uses it:

  • If you provide an empty array of frequencies
  • If any of the weights are less than 0
  • If the weights sum to 0

Be careful recursively using this fuzzer in its arguments. Often using Fuzz.map is a better way to do what you want. If you are fuzzing a tree-like data structure, you should include a depth limit so to avoid infinite recursion, like so:

type Tree
    = Leaf
    | Branch Tree Tree

tree : Int -> Fuzzer Tree
tree i =
    if i <= 0 then
        Fuzz.constant Leaf

    else
        Fuzz.frequency
            [ { weight = 1, fuzzer = Fuzz.constant Leaf }
            , { weight = 2, fuzzer = Fuzz.map2 Branch (tree (i - 1)) (tree (i - 1)) }
            ]
frequencyValues : Array { weight : Float, value : a } -> Fuzzer a

Create a Fuzzer by providing a array of probabilistic weights to use with values. For example, to create a Fuzzer that has a 1/4 chance of generating a string "foo", and a 3/4 chance of generating a string "bar", you could do this:

Fuzz.frequencyValues
    [ { weight = 1, value = "foo" }
    , { weight = 3, value = "bar" }
    ]

This fuzzer will simplify towards the values earlier in the array.

There are a few circumstances in which this function will return an invalid fuzzer, which causes it to fail any test that uses it:

  • If you provide an empty array of frequencies
  • If any of the weights are less than 0
  • If the weights sum to 0

Working with Fuzzers

constant : a -> Fuzzer a

Create a fuzzer that only and always returns the value provided, and performs no simplifying. This is hardly random, and so this function is best used as a helper when creating more complicated fuzzers.

invalid : String -> Fuzzer a

A fuzzer that is invalid for the provided reason. Any fuzzers built with it are also invalid. Any tests using an invalid fuzzer fail.

filter : (a -> Bool) -> Fuzzer a -> Fuzzer a

A fuzzer that only lets through values satisfying the given predicate function.

Warning: By using Fuzz.filter you can get exceptionally unlucky and get 15 rejections in a row, in which case the test will fluke out and fail!

It's always preferable to get to your wanted values using Fuzz.map, as you don't run the risk of rejecting too may values and slowing down your tests, for example using Fuzz.intRange 0 5 |> Fuzz.map (\x -> x * 2) instead of Fuzz.intRange 0 9 |> Fuzz.filter (\x -> Math.modBy 2 x == 0).

If you want to generate indefinitely until you find a satisfactory value (with a risk of infinite loop depending on the predicate), you can use this pattern:

goodItemFuzzer =
    itemFuzzer
        |> Fuzz.andThen
            (\item ->
                if isGood item then
                    Fuzz.constant item

                else
                    goodItemFuzzer
            )
map : (a -> b) -> Fuzzer a -> Fuzzer b

Map a function over a fuzzer.

map2 : (a -> b -> c) -> Fuzzer a -> Fuzzer b -> Fuzzer c

Map over two fuzzers.

map3 :
(a -> b -> c -> d)
-> Fuzzer a
-> Fuzzer b
-> Fuzzer c
-> Fuzzer d

Map over three fuzzers.

map4 :
(a -> b -> c -> d -> e)
-> Fuzzer a
-> Fuzzer b
-> Fuzzer c
-> Fuzzer d
-> Fuzzer e

Map over four fuzzers.

map5 :
(a -> b -> c -> d -> e -> f)
-> Fuzzer a
-> Fuzzer b
-> Fuzzer c
-> Fuzzer d
-> Fuzzer e
-> Fuzzer f

Map over five fuzzers.

map6 :
(a -> b -> c -> d -> e -> f -> g)
-> Fuzzer a
-> Fuzzer b
-> Fuzzer c
-> Fuzzer d
-> Fuzzer e
-> Fuzzer f
-> Fuzzer g

Map over six fuzzers.

map7 :
(a -> b -> c -> d -> e -> f -> g -> h)
-> Fuzzer a
-> Fuzzer b
-> Fuzzer c
-> Fuzzer d
-> Fuzzer e
-> Fuzzer f
-> Fuzzer g
-> Fuzzer h

Map over seven fuzzers.

map8 :
(a -> b -> c -> d -> e -> f -> g -> h -> i)
-> Fuzzer a
-> Fuzzer b
-> Fuzzer c
-> Fuzzer d
-> Fuzzer e
-> Fuzzer f
-> Fuzzer g
-> Fuzzer h
-> Fuzzer i

Map over eight fuzzers.

andMap : Fuzzer a -> Fuzzer (a -> b) -> Fuzzer b

Map over many fuzzers. This can act as mapN for N > 8. The argument order is meant to accommodate chaining:

Fuzz.constant fn
    |> Fuzz.andMap fuzzerA
    |> Fuzz.andMap fuzzerB
    |> Fuzz.andMap fuzzerC
andThen : (a -> Fuzzer b) -> Fuzzer a -> Fuzzer b

Use a generated value to decide what fuzzer to use next.

For example, let's say you want to generate a array of given length. One (not ideal) possible way to do that is first choosing how many elements will there be (generating a number), andThen generating a array with that many items:

Fuzz.intRange 1 10
    |> Fuzz.andThen
        (\length ->
            let
                go : Int -> Array a -> Fuzzer (Array a)
                go left acc =
                    if left <= 0 then
                        Fuzz.constant (Array.reverse acc)

                    else
                        itemFuzzer
                            |> Fuzz.andThen (\item -> go (length - 1) (item :: acc))
            in
            go length []
        )

This will work! Different fuzzers will have different PRNG usage patterns though and will shrink with varying success. The currently best known way to fuzz a array of items is based on a "flip a coin, andThen generate a value and repeat or end" approach, and is implemented in the Fuzz.array helpers in this module. Use them instead of rolling your own array generator!

Think of andThen as a generalization of Fuzz.map. Inside Fuzz.map you don't have the option to fuzz another value based on what you already have; inside andThen you do.

lazy : ({} -> Fuzzer a) -> Fuzzer a

A fuzzer that delays its execution. Handy for recursive types and preventing infinite recursion.

sequence : Array (Fuzzer a) -> Fuzzer (Array a)

Executes every fuzzer in the array and collects their values into the returned array.

Rejections (eg. from Fuzz.filter or Fuzz.invalid) bubble up instead of being discarded.

traverse : (a -> Fuzzer b) -> Array a -> Fuzzer (Array b)

Runs the Fuzzer-returning function on every item in the array, executes them= and collects their values into the returned array.

Rejections (eg. from Fuzz.filter or Fuzz.invalid) bubble up instead of being discarded.

Misc helpers

fromGenerator : Generator a -> Fuzzer a

(Avoid this function if you can! It is only provided as an escape hatch.)

Convert a Random.Generator into a Fuzzer.

Works internally by generating a random seed and running Random.step.

Note this will not shrink well (in fact it will shrink randomly, to smaller seeds), as Generators are black boxes from the perspective of Fuzzers. If you want meaningful shrinking, define fuzzers using the other functions in this module!