Upgrading to ppxlib 0.36.0

In ppxlib 0.36.0 the internal AST was bumped from 4.14 to 5.2. The changes to the AST are described below which should help ppx authors upgrade to the latest ppxlib.

Functions

Old Representation

The main change to the parsetree is the representation of functions. In OCaml before 5.2.0 functions were always represented in the parsetree with single-arity. For example, a simple three-argument function was represented as nested Pexp_fun constructors.

(* Expression *)
fun x y z -> ...

(* Parsetree *)
Pexp_fun(x, Pexp_fun (y, Pexp_fun(z, ...)))

Additionally, pattern-matching functions used the Pexp_function node to distinguish them from other functions.

(* Expression *)
function A -> ... | B -> ...

(* Parsetree *)
Pexp_function ([ case A; case B ])

New Representation

Since OCaml 5.2.0 (and ppxlib 0.36.0) all of these functions now map to a single AST node Pexp_function (note, this is the same name as the old cases function). The first argument is a list of parameters. The function we saw earlier:

(* Expression *)
fun x y z -> ...

(* Parsetree *)
Pexp_function([x; y; z], _constraint, body)

And the body is where we can either have more expressions (Pfunction_body expr) or cases (Pfunction_cases cases). With the body able to express both expressions and cases, our function from before can be represented with an empty list of parameters follow by the Pfunction_cases function body constructor.

(* Expression *)
function A -> ... | B -> ...

(* Parsetree *)
Pexp_function([], _, Pfunction_cases ([case A; case B]))

Changes to Ast_builder and metaquot

In ppxlib 0.36.0, functions from Ast_builder will help ppx authors produce maximum arity functions. Functions from Ast_helper will not do this.

(* Building functions with Ast_builder *)
let func arg body =
  pexp_fun Nolabel None (ppat_var (str arg)) body

let add_expr = func "x" (func "y" (eapply (evar "Int.add") [ evar "x"; evar "y" ]))

(* Building functions with Ast_helper *)
let ast_helper_func arg body =
  Ast_helper.Exp.fun_ Nolabel None (ppat_var (str arg)) body

let ast_helper_expr = ast_helper_func "x" (ast_helper_func "y" (eapply (evar "Int.equal") [ evar "x"; evar "y" ]))

The above snippet of code builds something akin to fun x y -> Int.add x y using Ast_builder and Ast_helper. If we print the simplified parsetree we can see that only the first coalesced the arity.

# pp_expr add_expr;;
Pexp_function
  ( [ { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "x")
      }
    ; { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "y")
      }
    ]
  , None
  , Pfunction_body
      (Pexp_apply
         ( Pexp_ident (Ldot ( Lident "Int", "add"))
         , [ ( Nolabel, Pexp_ident (Lident "x"))
           ; ( Nolabel, Pexp_ident (Lident "y"))
           ]
         ))
  )
- : unit = ()

Above, we see a function expression with arity two. Below, we see nested function expressions each with arity one.

# pp_expr ast_helper_expr;;
Pexp_function
  ( [ { pparam_loc = __loc
      ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "x")
      }
    ]
  , None
  , Pfunction_body
      (Pexp_function
         ( [ { pparam_loc = __loc
             ; pparam_desc = Pparam_val ( Nolabel, None, Ppat_var "y")
             }
           ]
         , None
         , Pfunction_body
             (Pexp_apply
                ( Pexp_ident (Ldot ( Lident "Int", "equal"))
                , [ ( Nolabel, Pexp_ident (Lident "x"))
                  ; ( Nolabel, Pexp_ident (Lident "y"))
                  ]
                ))
         ))
  )
- : unit = ()

Metaquot will also favour building maximum arity functions (e.g. [%expr fun x -> fun y -> x + y] will be maximum arity). You should use the internal AST nodes in order to have more control over the arity of your functions if defaulting to the maximum arity is not desired.

Additionally, some functions have changed in Ast_builder. Function names in Ast_builder match the AST node names, for example pexp_apply will create a Pexp_apply node. Since Pexp_function now represents all functions and not just the special pattern-matching case, pexp_function now builds full functions and not just ones with cases. When upgrading, your easiest path is to replace your calls to pexp_function with calls to pexp_function_cases.

Ast_pattern

We have not, as of this update, added any function to help users migrate from pexp_function in the Ast_pattern module. In the meantime, the following snippet should suffice for matching on any form of "cases":

let pexp_all_cases =
    let open Ast_pattern in
    single_expr_payload @@
    alt
      (pexp_function drop drop (pfunction_cases __ drop drop) |> map1 ~f:(fun cases -> (None, cases)))
      (pexp_match __ __ |> map2 ~f:(fun e cases -> (Some e, cases)))

Local Module Opens for Types

Another feature added in OCaml 5.2.0 was the ability to locally open modules in type definitions.

module M = struct
  type t = A | B | C
end

type t = Local_open_coming of M.(t)

This has a Ptyp_open (module_identifier, core_type) AST node. Just like normal module opens this does create some syntactic ambiguity about where things come from inside the parentheses.

Value Binding Constraints

Thanks to @lukstafi for suggesting this update.

Value bindings have a new record field: pvb_constraint. Since OCaml 5.2, whenever a value binding also has a type constraint, that information is held in this field. For example:

let v : int = 1

Will have the following parsetree representation:

[ Pstr_value
    ( Nonrecursive
    , [ { pvb_pat = Ppat_var "i"
        ; pvb_expr = Pexp_constant (Pconst_integer ( "1", None))
        ; pvb_constraint =
            Some
              (Pvc_constraint
                 { locally_abstract_univars = []
                 ; typ = Ptyp_constr ( Lident "int", [])
                 })
        ; pvb_attributes = __attrs
        ; pvb_loc = __loc
        }
      ]
    )
]

In versions of OCaml before 5.2, the same code would make use of the Ppat_constraint pattern node. Concretely, when upgrading you should use Ast_builder to build an expression with (or without, by default) a type constraint on the value binding where possible. Any pattern-matching on the value_binding record that does not use a wildcard (_) pattern will need to match on the pvb_constraint field too.