gren-lang/core - String.Parser.Advanced

type Parser payload problem value

An advanced Parser gives two ways to improve your error messages:

problem — Instead of all errors being a String, you can create a custom type like type Problem = BadIndent | BadKeyword String and track problems much more precisely.
payload — Sometimes you want to track information while you're parsing. This could be things like comments for when you want to format code, indentation levels, or what you're currently trying to parse. The payload gives you a place to store arbitrary data, should you need it.

I recommend starting with the simpler [String.Parser][String.Parser] module though, and when you feel comfortable and want better error messages, you can create a type alias like this:

import Parser.Advanced

type alias MyParser a =
  Parser.Advanced.Parser Payload Problem a

type Payload = Definition String | List | Record

type Problem = BadIndent | BadKeyword String

All of the functions from String.Parser should exist in String.Parser.Advanced in some form, allowing you to switch over pretty easily.

run :

Parser p x a

-> p

-> String

-> Result (Array ( DeadEnd p x))

a

This works just like String.Parser.run, except it requires an intial payload value and will return more precise information for each dead end.

type alias DeadEnd payload problem =

{ row : Int

, col : Int

, problem : problem

, payload : payload

}

Say you are parsing a function named viewHealthData that contains a list. You might get a DeadEnd like this:

{ row = 18
, col = 22
, problem = UnexpectedComma
, payload = List
}

We're using the payload to keep track of what we were parsing when the error occurred. So in the error message, we can say that "I ran into an issue when parsing a list. It looks like there is an extra comma." Or maybe something even better!

By tracking the row and col where the problem occurred along with the payload, we can provide helpful context about what the parser was doing. This is much better than just marking where the problem manifested without any additional information about the parsing state.

Note: Rows and columns are counted like a text editor. The beginning is row=1 and col=1. The col increments as characters are chomped. When a \n is chomped, row is incremented and col starts over again at 1.

getPayload : Parser payload x payload

This allows you to extract the payload from the parser. You might use this during parsing to consult the current, or previous, indentation level. Maybe you want the payload at the end of parsing to report some interesting statistics, or to include some optional constructs in certain cases (like comments while code formatting).

intWithPayload : Parser Payload error { int : Int, payload : Payload } intWithPayload = succeed (\integer payload -> { int = integer, payload = payload }) |> keep int |> keep getPayload

setPayload : payload -> Parser payload x a -> Parser payload x a

This is how you set the payload. For example, here is a rough outline of some code that uses setPayload to mark when you are parsing a specific definition:

type Payload
  = Definition String
  | List

definition : Parser Payload Problem Expr
definition =
  functionName
    |> andThen definitionBody

definitionBody : String -> Parser Payload Problem Expr
definitionBody name =
  setPayload (Definition name) <|
    succeed (Function name)
      |> keep arguments
      |> skip (token "=" ExpectingEquals)
      |> keep expression

functionName : Parser c Problem String
functionName =
  variable
    { start = Char.isLower
    , inner = Char.isAlphaNum
    , expecting = ExpectingFunctionName
    }

First we parse the function name, and then we parse the rest of the definition. Importantly, we call setPayload so that any dead end that occurs in definitionBody will get this extra payload information. That way you can say things like, "I was expecting an equals sign in the view definition".

Keep in mind that the payload will be changed for the entire duration of parsing, and not just for this particular parser. Take a look at scopedPayloadUpdate for more info.

updatePayload :

(payload -> payload)

-> Parser payload x a

Like setPayload, but with the option of modifying a subset of the payload instead of replacing it entirely. This might be helpful when you're using the payload to keep track of multiple things.

Like setPayload, the payload changes aren't automaticly reverted.

scopedUpdatePayload :

(payload -> val)

-> (payload -> val)

-> (val -> payload -> payload)

-> Parser payload x a

This works like updatePayload, except it allows you to revert the payload back to it's original state when a parser is done. Effectively, it allows you to scope a payload value to a specific parser.

definitionBody : String -> Parser Payload Problem Expr
definitionBody name =
  setPayload (Definition name) <|
    succeed (Function name)
      |> keep arguments
      |> skip (token "=" ExpectingEquals)
      |> keep
        ( Parser.scopedUpdatePayload
          (\payload -> payload.indentLevel + 1)
          .indentLevel
          (\nextValue payload -> { payload | indentLevel = nextValue })
          expression
        )

In the above expression, the indentLevel is increased for parsing the expression, but reverted to its original value after.

renderDeadEnds :

Output out

-> Extract payload problem

-> String

-> Array (DeadEnd payload problem)

-> Array out

Render an array of DeadEnds.

The String should be the same that was passed to the failing parser.

This returns a list of renderable "pieces", explaining what went wrong and where.

type alias Output out =

{ text : String -> out

, formatCaret : out -> out

, newline : out

, formatContext : out -> out

, linesOfExtraContext : Int

}

Describes how to output the various parts of the error message.

type alias Extract payload problem =

{ deadEndToString : DeadEnd payload problem -> { row : Int, col : Int , context : String }

, problemToString : problem -> Expected

}

Describes how to get the context stack from a DeadEnd and how to extract expectations information from a problem.

type Expected

= Expected String

| Other String

A problem is often of the form "expected something". This type is used to group those together.

int : x -> Parser c x Int

Just like String.Parser.int where you have to handle negation yourself. The only difference is that you provide a problem in case the parser fails.

token : String -> x -> Parser c x {}

Just like String.Parser.token except you provide a problem in case the parser fails.

keyword : String -> x -> Parser c x {}

Just like String.Parser.keyword except you provide a problem in case the parser fails:

let_ : Parser Payload Problem {}
let_ =
  keyword "let" ExpectingLet

Note: this would fail to chomp letter because of the subsequent characters. Use token if you do not want that last letter check.

variable :

{ start : Char -> Bool

, inner : Char -> Bool

, expecting : x

}

-> Parser c x String

Just like String.Parser.variable except you specify the problem yourself.

end : x -> Parser c x {}

Just like String.Parser.end except you provide the problem that arises when the parser is not at the end of the input.

succeed : a -> Parser c x a

Just like String.Parser.succeed

keep : Parser c x a -> Parser c x (a -> b) -> Parser c x b

Just like keep from the String.Parser module.

skip : Parser c x ignore -> Parser c x keep -> Parser c x keep

Just like skip from the String.Parser module.

lazy : ({} -> Parser c x a) -> Parser c x a

Just like String.Parser.lazy

andThen : (a -> Parser c x b) -> Parser c x a -> Parser c x b

Just like String.Parser.andThen

problem : x -> Parser c x a

Just like String.Parser.problem except you provide a custom type for your problem.

oneOf : Array (Parser c x a) -> Parser c x a

Just like String.Parser.oneOf

map : (a -> b) -> Parser c x a -> Parser c x b

Just like String.Parser.map

mapError : (errA -> errB) -> Parser c errA x -> Parser c errB x

Transform the Error of a parser.

backtrackable : Parser c x a -> Parser c x a

Just like String.Parser.backtrackable

commit : a -> Parser c x a

Just like String.Parser.commit

sequence :

{ start : Token x

, separator : Token x

, end : Token x

, spaces : Parser c x {}

, item : Parser c x a

, trailing : Trailing

}

-> Parser c x (Array a)

Just like String.Parser.sequence except with Token records for the start, separator, and end. That way you can specify your custom type of problem for when something is not found.

type alias Token x = { string : String, expecting : x }

Contains a string that we expect to chomp and the problem we'll return in case the parser fails.

type Trailing

= Forbidden

| Optional

| Mandatory

What’s the deal with trailing commas? Are they Forbidden? Are they Optional? Are they Mandatory? Welcome to shapes club!

loop :

state

-> (state -> Parser c x ( Step state a))

-> Parser c x a

Just like String.Parser.loop

type Step state a

= Loop state

| Done a

Just like String.Parser.Step

spaces : Parser c x {}

Just like String.Parser.spaces

lineComment : String -> x -> Parser c x {}

Just like String.Parser.lineComment except you provide a problem in case the parser fails.

multiComment : Token x -> Token x -> Nestable -> Parser c x {}

Just like String.Parser.multiComment except with a Token for the open and close symbols.

type Nestable

= NotNestable

| Nestable

Works just like String.Parser.Nestable to help distinguish between unnestable and nestable comments.

getChompedString : Parser c x a -> Parser c x String

Just like String.Parser.getChompedString

chompIf : (Char -> Bool) -> x -> Parser c x {}

Just like String.Parser.chompIf except you provide a problem in case a character cannot be chomped.

chompChar : Char -> x -> Parser c x {}

Just like String.Parser.chompChar except you provide a problem in case a character cannot be chomped.

chompWhile : (Char -> Bool) -> Parser c x {}

Just like String.Parser.chompWhile

chompUntil : String -> x -> Parser c x {}

Just like String.Parser.chompUntil except you provide the problem in case you chomp all the way to the end of the input without finding what you need.

chompUntilEndOr : String -> Parser c x {}

Just like String.Parser.chompUntilEndOr

mapChompedString : (String -> a -> b) -> Parser c x a -> Parser c x b

Just like Parser.mapChompedString

getPosition : Parser c x { row : Int, col : Int }

Just like String.Parser.getPosition

getRow : Parser c x Int

Just like String.Parser.getRow

getCol : Parser c x Int

Just like String.Parser.getCol

getOffset : Parser c x Int

Just like String.Parser.getOffset