Deriving abstract types

[clecat]

Hello everyone,

I have recently started doing my first ppx and I encountered a problem:
My goal was to generate functions for a record which I would keep as an abstract type in the signature of my module.
The structure part was working fine but when doing the signature part, I met one problem:
My type being an abstract one, I could not access my record fields anymore, therefore I could not generate the functions signatures.

This is why I was wondering if there was a solution to allow me to have my record information available for the ppx but remove them with the abstraction of my type right after.

What do you think of it ?

You could do that with an extension point:

[%foo type t = { ... }]

but not with the [@@deriving] attribute.

[NathanReb]

This is a common problem for derivers, I think it would make sense to support that natively in ppxlib.

The issue with [%%foo: type t = { ... }] is that you can't implement it using Extension.declare as it will expect you to transform a sig item into another single sig item where we want to generate several.

I wonder if it would make sense to support something like this:

 type%abstract t =
  { a : string
  ; b : int
  }
[@@deriving make]

which would expand to:

type t
val make : a: string -> b: int -> unit -> t

Pppxlib could take care of calling the deriver by passing it the right type definition while only exposing the abstract type. That way dervier maintainers don't have to do extra work to allow this.

[NathanReb]

@emillon raised valid concerns about the type%abstract thing potentially conflicting with existing rewriters but I'm sure we can come up with a good enough convention that would work most of the time.

I admit that I'm a little sceptical about this. Apart from [@@deriving make], what other derivers would benefit from this feature?

[NathanReb]

ppx_fields_conv and ppx_variants_conv would I think. So would ppx_factory. Any ppx which derives values based on record fields or variants. There's probably others that I don't necessarily know about.

The main issue is that currently I don't think you can express that transformation (i.e. signature_item -> signature) using our rules and you'd have to register a whole transformation which as far as I understand goes against ppxlib philosophy.

Well, you could hack something to make it work with the current API. Such as expanding type%astract t = <def> into type t [@@abstract.def type t = <def> and then looking for the definition in this attribute in the deriver.

But if that's a common pattern, I guess it makes sense to include it in ppxlib directly. Regarding ppx_fields_conv and ppx_variants_conv, we have a syntax steering group inside Jane Street, I can bring the question there to see what people think about this syntax.

Personally, I'm sceptical for two reasons:

• the processed code looks smaller and clearer than the original code
• there will be no strong check that the record in the interface is the same as the record in the implementation. For instance you could drop or add mutable keywords and if you add a wrapper around make in the implementation you could even change the type of fields

[NathanReb]

Yeah it's true that not having the type definition checked by the compiler isn't great. The generated signatures would have to match the generated implementations though, which means they probably can't differ too much.

I'm curious to know how this is dealt with at Janestreet for example, i.e. do you have a different method to derive the signatures but still keep the type abstract in the interface.

We just write the signature by hand when it comes to [@@deriving fields] or [@@deriving variants].

[rgrinberg]

Personally, this feature does not appeal to me but I've encountered this problem myself quite a few times.

The issue with [%%foo: type t = { ... }] is that you can't implement it using Extension.declare as it will expect you to transform a sig item into another single sig item where we want to generate several.

Is this a limitation with the Extension api? It would be really nice if there was a way to make this work. This is already to generate code from anonymous type definitions. E.g. [%sexp_of: (int * _) list] [1,"one"; 2,"two"]. If it's possible to extend the API to allow this, it would be nice & symmetric.

It is supported by Extension.declare_inline, which lets you generate multiple items. And in any case, you can generate a single include sig ... end.

[NathanReb]

Indeed Extension.declare_inline should work well without too much boiler plate. I didn't know about it!

CC @clecat, using https://github.com/ocaml-ppx/ppxlib/blob/master/src/extension.mli#L170, you should be able to write something like:

[%%<deriver_name>: type t = {a; ...}]

that could be expanded to what you want!

My impression was that this is a common enough pattern so that it's worth making it automatically available for any registered Deriving.Generator.t, eventually controlled by a flag but you both seem to feel otherwise. I guess as long as it's possible to implement it doesn't matter that much!

Should we close this issue then?

[rgrinberg]

My impression was that this is a common enough pattern so that it's worth making it automatically available for any registered Deriving.Generator.t, eventually controlled by a flag but you both seem to feel otherwise. I guess as long as it's possible to implement it doesn't matter that much!

As far as I understand, we have this distinction between ppx_deriving and ppx's deriving:

• In ppx, the convention is to first define an expander and then use it to register a corresponding deriver.
• In ppx_deriving, registering a deriver automatically creates the corresponding expander.

I agree with Nathan that ppx_deriving's approach is better here. It results in a more uniform API for all derivers and is also less boilerplate for implementors.

That's a topic for a separate issue :)

I'd be curious to see what a ppx_deriving like API would look like for ppx. One difference is that ppx parses arguments in a typed way and you need something to pack the typed description of the arguments together with the callback.

@NathanReb yeah, let's close this issue for now. If it turns out that type%abstract is widely used, we can reconsider adding it to ppx.

[clecat]

Thank you for your responses, I should be able to solve my problem now :)