Access to constructor params from within ADJUST and field-initialiser blocks

[leonerd]

Current state in Object::Pad

First, a little background on the current state of Object::Pad. There are ADJUST blocks, and there are field-initialiser blocks. They behave in very different ways.

The ADJUST keyword creates an entire (anonymous) function which behaves as such when invoked. It is passed a hashref to the constructor params as its first argument. It can access that with $_[0] or shift or optionally by providing a signatured parameter name:

ADJUST { say "The foo param was $_[0]->{foo}"; }
ADJUST { say "The foo param was ", shift->{foo}; }
ADJUST ($params) { say "The foo param was $params->{foo}"; }

(Of course, these examples are stupid in that they're doing things that :param would be far better suited to, but I didn't want to cloud up the examples with more complex real-world examples.)

By comparison, field-initialiser blocks are simple blocks, being parsed by no small amount of parser trickery into believing they're all just sequential blocks of the same function. That function is a single (anonymous) function stored as part of the class, used to initialise all the fields. These initialiser blocks do not currently have any access to the constructor parameters; which is somewhat limiting.

Both ADJUST and field-initialiser blocks get access to a $self lexical, the same as other methods.

Current state in feature-class branch

The current feature-class branch provides ADJUST blocks that allow either of the first two forms, but not the third form (with a signature parameter). The branch does not currently provide field-initialiser blocks at all.

The Question

And now we get on to my question. I would like to unify these two things together and at the same time make them better. In particular, I would like both ADJUST and field-initialiser to be simple blocks within one (implied) function rather than having one entersub overhead for every block. Additionally, I would like all of them to have the same access to the constructor parameters.

In particular, it would be great if we could say that a field-initialiser block is really just the same as an ADJUST block that assigns the field from its result, give or take some code that inspects the parameters hash to see if the caller already provided a value.

field $f { EXPR }   <==>   field $f;
                           ADJUST { $f = exists $params{f} ? $params{f} : do { EXPR }; }

The trouble though is how to provide this params hash(ref). It can't just come as the first value in @_ because any block would too easily be able to break access to it for all the later ones by just doing shift, or something equally dumb. It needs to be provided by something guaranteed to be visible to all the blocks, that no earlier block can get in the way of. So far I can only think of two ideas, neither of them are great:

Special %params lexical

In the same way that $self is special, make %params special in these blocks:

  field $x { delete $params{x} }

  field $y; ADJUST { $y = delete $params{y} }

It's a simple idea and easily understandable, but it does mean that no class is permitted a field called %params itself, because then there'd be no way for an ADJUST block to see it. Of course we're already in that situation at the moment with $self, so it doesn't make the problem much worse. But perhaps enough classes might themselves want a field %params, that this would become awkward.

Use the %_ superglobal

By considering analogy to the @_ superglobal already in perl, we can consider using the %_ hash as a storage of these name/value pairs:

  field $x { delete $_{x} }

  field $y; ADJUST { $y = delete $_{y} }

I think there's a certain neatness to this, and a certain symmetry with using $_[0], $_[1], etc... However, some folks might object to it on grounds that "newbies to perl might be confused by lots of symbol syntax". Personally I'm not very swayed by this particular argument, but it seems important to some. Another potential argument against doing this is that it seems weird to introduce a new use for %_ while at the same time trying to get rid of @_ in favour of function signatures.

Other Designs

Another way to look at it entirely, is to observe that the main reason for wanting the hash(ref) of params available in these blocks in the first place, was to do things with constructor parameters that aren't just simply "copy the value into a field". This was originally discussed for Object::Pad in RT137209. I say "discussed" - I thought out loud on a few ideas but nobody else has commented so far.

In many ways the current solution of just passing a hash(ref) around isn't very nice. It means that the actual parameter names used by the construction process overall are not manifestly expressed anywhere, and only become apparent in side-effects of the actual process of constructing an object. You can only find out those names by running it and seeing if it complains about missing or extra ones. It would be nice to find a nicer overall design for this sort of pattern, but so far one has not emerged.

[leonerd]

Latest thoughts: While it doesn't help the field-initialiser block, an ADJUST block could still work with something that happens superficially to look like a signature. The same parser hackery that applies to allowing each ADJUST block to just resume parsing of a single containing sub can be employed to adjust the pad names before resuming. It would permit:

ADJUST (%params) { say join "|", keys %params }
ADJUST (%args)   { say "Foo is $args{foo}" }
ADJUST           { say "params aren't visible here at all" }

which would allow ADJUST blocks to request access to the params, while neither eating up a reserved name nor using the %_ superglobal. Though it still doesn't solve the wider metadata/declarative question.

[haarg]

I don't like the idea of "Special %params lexical". %params is not obviously magic, and someone could easily want to name a field or other variable with that name. $self is only acceptable as a reserved variable because it is so well established in perl that we can't really use anything else for the instance variable. But no such consensus exists for constructor params.

%_ is better, in that it is obviously "special" and maps well to the old @_. But we're trying to move away from @_.

Of the options presented so far, something declared by the user seems the only really acceptable option. As stated, it's not obvious how to combine that with initializer blocks, but I'm not sure that combining them is actually a necessary feature.

[leonerd]

Thinking further about it, I'm not sure I like the exact syntax of this latter approach. It's nice to imagine giving an ADJUST block a way to request the name of a lexical hash created in scope that aliases the parameters hash, so it can delete on it, but I don't think the syntax given here is quite the right thing:

ADJUST (%params) { say join "|", keys %params }

Mostly because this really does something very different to the similar-looking syntax of

method f (%params) { ... }

What we're doing in the ADJUST block is really something a bit more like a refalias. Signatures don't support that yet, but if they did I imagine it'd look like method f (\%params) { ... } and so maybe that suggests similar for ADJUST. But that's what we'd always write, so it seems a bit noisy.

Another potential downside to this current notation is that it doesn't leave us any room for adding more things in there. Again by analogy to slurpy parameters, it feels like %params ought to be last, with nothing else after it. I don't know what else we might want to pass in, but currently there isn't space for it. So boo. :(

I almost wonder if the ADJUST block wants an attribute to give it the name of a new lexical to create, instead:

ADJUST :please_provide_me_the_parameters_in_a_hash_called(%params) { ... }

I'll take opinions on a nicer name ;)

[leonerd]

And now some more examples on the "declared metadata" front...

Looking around at various actual ADJUSTPARAMS blocks I have in real code right now, it seems most of them operate on a theme of taking some fixed set of named constructor arguments, and using them to set up the values of some fields by using code, rather than simply storing the values as passed in.

For example, a few operate on a theme of taking the name of a file to be opened, opening it in the ADJUST block, then storing the filehandle in a field of the instance:

https://metacpan.org/release/PEVANS/Electronics-PSU-DPSxxxx-0.02/source/lib/Electronics/PSU/DPSxxxx.pm#L67-80

Somewhat paraphrased:

field $fh :param { undef };
ADJUSTPARAMS ( $params ) {
   unless( $fh ) {
      my $path = delete $params->{path};
      # code here to open $path and store it in $fh
   }
}

It feels like this is a fairly common sort of pattern - parameters that are passed in, not to be stored directly, but instead to be used as part of a larger block of code that is evaluated at construction time which eventually yields the initial value for a field to be stored.

It's annoying that these parameters aren't declared any how, and only work as a side-effect of running the actual code. There's no predelcared metadata, so things like name-clashes aren't visible like they are for :param:

$ perl -MObject::Pad -E 'role R1 { field $x :param } role R2 { field $x :param } class C1 :does(R1) :does(R2) {}'
Named parameter 'x' clashes with the one provided by role R2 at -e line 1.

While it isn't yet a thing, it is tempting to imagine that perhaps some way to attach documentation snippets could be found for :param-annotated fields. If such a thing existed, then perhaps a sufficiently-advanced IDE would be able to display such information about named parameters being passed into constructors when they were encountered. These additional parameters being eaten by ADJUST blocks wouldn't be visible in that way either. Notice in the above example code, the string "path" only appears once, in the delete $params->{path} expression. It doesn't appear anywhere else - specifically, not in any compiletime-declared metadata about the class.

It's tempting to think that perhaps what is wanted is a new class-level keyword for declaring that named parameters exist, so that they can have such information attached.

param path :apidoc(The path to the filesystem device to open, if 'fh' is not provided);

However, that's just dangling out in space somewhere separate, there's nothing to attach it to the specific value within the ADJUST block. It'd be too easy to become separated from it.

It feels like the core of why this is hard to design comes from the fact that "named params" want to be two things at once:

• Values passed in lexical variables directly into the body of an ADJUST block
• Toplevel named declared things that are members of a class, known about at compiletime

There doesn't at the moment appear to be a syntax idea that somehow addresses both concerns.

[aquanight]

One possible answer to this: revisit the notion of "lexical fields" (at the very least still outside of methods) and through this, not needing to provide any parameters to ADJUST:

Using the file example above:

field $fh :param { undef };
{
    field $path :param { undef };
    ADJUST {
        defined($fh) != defined($path) or die;
        $path and (open $fh, "<", $path or die "$path: $!\n");
        # In theory, when this ADJUST ends, $path is no longer visible to anything.
        # Thus it could be released from the object.
    }
}

[Ovid]

I think we might revisit this by not looking at ADJUST as a phaser. The original intent was to look at this similar to Moo/se, but use different names for the methods to ensure they weren't viewed as the same thing as Moose methods. In particular, Moo/se has:

# called before new
around "BUILDARGS" => sub ( $orig, $class, @args ) { ... }; 

sub new ($class, $args) { ... } # implicit. We don't write this directly

# called after new
sub BUILD ($self, @args) { ... }

For Corinna, I envisioned something vaguely like this (with naming suggested by @thoughtstream, with NEW calling each of these in turn, but respecting inheritance by calling parent versions, too):

# before NEW
method CONSTRUCT :common (@args) {
     # $class is available, but not $self
     # must return even-sized list of key/value pairs that
     # are passed to new()
}

method NEW (%args) { ... } # called implicitly using args returned by CONSTRUCT

# after NEW
method ADJUST (%args) {
    # both $class and $self available. Do what you will before
    # new object is returned
}

This is somewhat described here in the Wiki, but I see now that it didn't make it into the official RFC. When I saw you were using ADJUST differently, I said nothing because I was tired of how divisive so many arguments about Corinna were. That's my fault. I should have said something. Burn out.

CONSTRUCT is used to to take whatever is passed to new and return K/V pairs. After all CONSTRUCT methods are called, the final result goes to NEW (internal method). It's awkward because it looks like a phaser (and those don't take args) and it kind of looks like an instance method, but it's not quite one.

ADJUST is described here and is called after NEW.

However, the Moo/se paradigm is only awkward (in my opinion) because Moo/se is limited by Perl's current syntax. The overall idea seems solid.

[leonerd]

Your suggestion there doesn't seem to any clearer answer how to handle these different params. Perhaps a concrete example is needed.

Lets imagine some UI widget that needs a title and a colour specified in red/green/blue components. The title is just stored directly as text, but the red/green/blue are actually forwarded to the constructor of a "Colour" object, which is then stored. Right now in Object::Pad or feature-class I can write the following and it technically works:

class Widget {
   field $title :param;

   field $colour;
   ADJUST ($params) {
      $colour = Colour->new( delete %{$params}{qw( red green blue )} );
   }
}

my $widget = Widget->new( title => "The text", red => 100, green => 30, blue => 0 );

This has some nice properties:

• All the ADJUST blocks across every role/component class are run, independently. Each gets to inspect the params and delete from it the ones they consume
• Any left-over params that no block deleted are known to be unhandled, so we can complain about them.

E.g. consider what happens to someone who does Widget->new( cyan => 123 ).

It has some downsides to it though. Most notably, the words red, green and blue only appear in the behavioural code; in the body of the ADJUST block in that hash-slice delete expression. They don't otherwise appear anywhere that is inspectable as declarative metadata. For example, we can imagine what might eventually happen when IDEs and code editors start to understand this syntax.

class Widget {
   field $title :param :is(Text) :apidoc("The title text to display at the top");
....

my $widget = Widget->new( title => ... 

If the user hovers the mouse over title here perhaps they'll be reminded the type and the apidoc help string for this parameter.

Where in the class declaration can we attach similar information about the red, green and blue constructor params?

With reference to your suggested methods above (CONSTRUCT, NEW, ADJUST) would you be able to find any better answers on this? Specifically, where red/green/blue are handled, where any metadata about them might be declared (currently we have nowhere), and where the unrecognised cyan param would be complained about.

[Ovid]

With reference to your suggested methods above (CONSTRUCT, NEW, ADJUST) would you be able to find any better answers on this? Specifically, where red/green/blue are handled, where any metadata about them might be declared (currently we have nowhere), and where the unrecognised cyan param would be complained about.

I'm still not seeing the problem. Currently, this can be done in Moose with MooseX::StrictConstructor and BUILDARGS. The lack of metadata about the documented arguments is either:

• Just fine because that's what the class designer intended or ...
• Terrible because they didn't think about it

So in the latter case:

class Widget {
   field $title :param;
   field ($red, $greed, $blue ) :param;
   field $colour;

   ADJUST ($params) {
      $colour = Colour->new( delete %{$params}{qw( red green blue )} );
   }
}

my $widget = Widget->new( title => "The text", red => 100, green => 30, blue => 0 );

Note that the colors don't have readers or writers, as per your original code, so I think this satisfies that and it's more correct because the user must supply them. (You can always return undef from an initializer block if they're optional).

If my code sample doesn't address your issue, can you help me understand why? If this is solvable under Moose but not Corinna, can you show me a Moose example?

[HaraldJoerg]

The issue with your code example is that the "fields" $red, $greenand $blue are just waste after the construction of the object, but can't be disposed of. This is not a showstopper, but I consider it a minor flaw.

As for the comparison with BUILDARGS: This will get more pronounced as soon as inheritance comes into play. In Moose, BUILDARGS is a method, thus can be chained to call the parent's BUILDARGS whenever it sees fit. Also, BUILDARGS can be used to define arbitrary constructor APIs, not just key/value pairs. I think we might need more procedural control in ADJUST blocks, too.

Here's an example which can be made to work with Moose, but not with Object::Pad:

use Object::Pad;

class Complex {
    field $real :param;
    field $imaginary :param;
}

class Real :isa(Complex) {
    ADJUST {
       # Here, I should be able to set $imaginary to 0.  But how?
       # $imaginary isn't available, and setting imaginary => 0
       # has no place to set it
    }
}

my $real = Real->new(real => 123);
# Required parameter 'imaginary' is missing for Complex constructor

[leonerd]

The issue with your code example is that the "fields" $red, $greenand $blue are just waste after the construction of the object, but can't be disposed of. This is not a showstopper, but I consider it a minor flaw.

It could be considered a major flaw. Right now in this particular example they're just numbers being the colour components; so it's a fairly harmless problem. But consider if those were large object references themselves. Those dead fields needlessly holding them are going to inflate the refcount, delaying their destruction. That at least wastes memory and depending on what other behaviours might be attached to destruction time of those objects, might actually be a bug, if some other code was relying on timely destruction of them.

We really mustn't store construction-time parameters as fields unless we intend to store them as fields. Using the fields just as temporary storage during construction time and then not touching them afterwards isn't really a good solution.

[leonerd]

If my code sample doesn't address your issue, can you help me understand why?

@HaraldJoerg mostly explained that in the reply; I expanded on it above.

If this is solvable under Moose but not Corinna, can you show me a Moose example?

Well I don't know Moose anywhere near enough to suggest how to do it there, but consider this in classical perl:

package Widget;
sub new ($class, %params) {
   my $title = delete $params{title};
   my $colour = Colour->new( delete %params{qw( red green blue )} );

   die "Unrecognised construction parameters - " . join(", ", sort keys %params) if keys %params;

   my $self = bless {
      title => $title,
      colour => $colour,
   }, $class;
   return $self;
}

We haven't pointlessly stored the plain red, green or blue values, other than by forwarding them to the constructor of the Colour instance. We've still complained to our caller about any other unrecognised construction-time params.

[leonerd]

@HaraldJoerg

Here's an example which can be made to work with Moose, but not with Object::Pad:

Ah, that example is showing off an equally-valid, yet quite different problem. That's more about how subclasses can alter the definitions of existing fields on their parent classes. It's quite common in Moose for example to alter an existing field; in your example case:

has '+imaginary' => default => sub { 0 };

The + symbol says not to add a new field here, but instead we're altering the definition of an existing one for members of this subclass.

For your usecase, it's as if we'd want to allow something like:

class Complex {
  field $re :param :reader :writer;
  field $im :param :reader :writer;
  ...
}

class Real :isa(Complex) {
   inherited field $im { 0 };  # give it a default value
}

though ideally you'd want to try to remove the ":param" part of it; making it a construction-time error to explicitly pass it.

However, that being the case it does sortof suggest at least this example, saying that a Real number is a subclass of a Complex one, is perhaps not the best example. What should be the correct behaviour of

my $z = Real->new( re => 4, im => 3 );

?

[HaraldJoerg]

@leonerd:

my $z = Real->new( re => 4, im => 3 );

That should be an error. I could live with a sloppy implementation where im is silently ignored, but not with one which would accept the value of 3.

I might be able to come up with a more complex, but more relevant example at some time.

I can not quote any relevant literature on that, but in my belief a class is responsible for the complete API of its constructor. Saying "ah, and in addition you can pass all parameters of the base class" is frequent practice and sort of acceptable if you don't want to duplicate documentation, but not correct. The fact that class Real is based on Complex is an implementation detail, a user of Real should not need to know about this.

So, some instance within class Real should be able to set up the construction parameters for its base class (and also, set up initialization parameters for every role it :does). If we have an explicit mechanism for that, then we can also solve the problem of conflicting :param names in base classes and roles. Unlike Moose, we do not have the problem that everything needs to live in one single hash after construction, roles and parent classes have their own spaces for fields!

[Ovid]

The issue with your code example is that the "fields" $red, $green and $blue are just waste after the construction of the object, but can't be disposed of. This is not a showstopper, but I consider it a minor flaw.

It could be considered a major flaw. Right now in this particular example they're just numbers being the colour components; so it's a fairly harmless problem. But consider if those were large object references themselves. Those dead fields needlessly holding them are going to inflate the refcount, delaying their destruction. That at least wastes memory and depending on what other behaviours might be attached to destruction time of those objects, might actually be a bug, if some other code was relying on timely destruction of them.

Now I see your concern, but I don't see it as a "major flaw" in terms of Corinna. It's a major flaw in terms of the general expressiveness of most OO systems. With those systems that define a constructor as the name of the class, you could do this (pseudo-code):

class Widget {
   field $title :param;
   field ($red, $greed, $blue ) :param;
   field $colour;

   method Widget :constructor ($params) {
      $colour = Colour->new( delete %{$params}{qw( red green blue )} );
   }
}

For those types of OO systems (e.g., Java), no field would have a value without being explicitly assigned in the Widget constructor. The problem goes away.

With current Corinna syntax, it can still be fixed (keeping in mind that ADJUST is called after the constructor:

class Widget {
   field $title :param;
   field ($red, $green, $blue ) :param;
   field $colour;

   ADJUST ($params) {
      $colour = Colour->new( delete %{$params}{qw( red green blue )} );
      undef $_ foreach $red, $green, $blue;
   }
}

In the above, we get undefine those values, but the metadata is still available. That solves the issue, yes?

I think (I could be wrong) that this is an edge case that's less likely to be a serious issue, but it's easy to solve if it arrives. I acknowledge that you might write code that often hits this issue. I'm unsure of the last time that I have. However, I do admit that I've written plenty of code where arguments are required for construction, but not post-construction.

[Ovid]

However, that being the case it does sort of suggest at least this example, saying that a Real number is a subclass of a Complex one, is perhaps not the best example. What should be the correct behaviour of

my $z = Real->new( re => 4, im => 3 );

Well, that's annoying  😃 I'm not sure of the best approach here.

In classical Perl terms, the constructor for Real might look like this:

sub new ( $class, %args ) {
    my @keys = keys %args;
    if ( 1 == @keys && 're' eq $keys[0] ) {
        return bless $class->SUPER::new(%args, im => 0), $class;
    }
    croak("...");
}

In Moose, I think this would work:

has '+im' => (
    default  => 0,
    init_arg => undef,
);

You'd probably want MooseX::StrictConstructor with that.

So for Corinna, possibly something like this?

class Complex {
  field $re :param :reader :writer;
  field $im :param :reader :writer;
  ...
}

class Real :isa(Complex) {
   field $im :overrides { 0 };  # give it a default value
}

If we wanted $im to remove the :writer, we don't have a syntax for that. But Liskov tells us that any place we have an instance of a class, we should be able to substitute an instance of the subclass and have it work. So we'd have to inherit all of the parent field behavior except if we have :overrides, it removes the :param for its definition (leaving all other attributes intact, unless otherwise specified) and if we wanted that as a :param, we'd have to add that attribute back in explicitly.

Or we could say that :overrides removes all parent attributes from the field and if we want to respect Liskov, we have to manually add them back: field $im :overrides :reader :reader {0}; That's probably a cleaner solution, even if it means that Liskov will often be unhappy if we copy the attributes incorrectly.