VIP9: Expand VCL object support

Synopsis

Currently objects are limited to global objects which have a lifetime of the entire VCL. This VIP is to have objects which can be created during the request and their scope is limited to that request. When the request is done, the objects are destructed.

Original VIP9 date: April 25, 2016

Updated VIP9 date: November 18, 2019

Why?

By allowing VMOD objects to live in the request scope, we can easily bring in OO style typed variables to VCL. For example, integers, strings, doubles, lists, hash tables, anything and everything.

How

Here is a VCL snippet which compiles and works with the patch [0] and uses a new libvmod_types [1].

vcl 4.1;

import types;

sub vcl_init
{
  //global objects, these are unchanged
  new s = types.string("Hello!");
  new reqs = types.integer(0);
}

sub vcl_recv
{
  //new req scoped objects
  new slocal = types.string("Request scoped string");
  new s2 = types.string("request string two");
  new count = types.integer(1);
}

sub vcl_backend_fetch
{
  //new bereq scoped objects
  new sbe = types.string("berequest string v1");
  set bereq.http.sbe = sbe.value();

  sbe.set("berequest string v2");
  set bereq.http.sbe2 = sbe.value();
}

sub vcl_deliver
{
  //referencing a mix of global and req scoped objects
  set resp.http.X-s = s.value();
  set resp.http.X-s-length = s.length();

  set resp.http.X-slocal = slocal.value();
  set resp.http.X-slocal-length = slocal.length();

  count.increment(10);
  set resp.http.count = count.value();
  set resp.http.reqs = reqs.increment_get(1);
}

Regarding the patch, I left the legacy global objects alone in code and syntax. I introduced 2 new variable name scopes: req.var.* and bereq.var.*. This is completely cosmetic as these variables can still be request scoped without the (be)req.var prefix. However, the reason for adding it is to give the user some kind of indication that their variable is tied to a frontend, backend, or global scope. Otherwise I have the feeling having a bunch of un-prefixed variables throwing vcc scope errors when used incorrectly will be confusing.

Discussion on the mailing list brought up the following points:

* These prefixes are too verbose, its cleaner to just have plain variables without scope prefixes.

• Without a prefix, we cannot bind variables to top_request, session, etc, scopes. We could work around this by issuing a cast like prefix when we declare the variable, ex: new (top)variable = ....

Also, the implementation is fairly simple because I piggybacked on the vmod/vrt priv_task implementation. Request scoped objects are basically given a shimmed struct vmod_priv. I had to jump thru a few small hoops in vcc code to get the priv->priv to cast into an actual struct that the VMOD expects. This may or may not be related to VIP#1, but it would be cleaner to move objects to something more priv like than trying to pass in an explicit struct. However, for the patch, I kept the object interface the same and made use of the previously mentioned vcc/vrt shims.

The patch is enough to have the examples work and give you guys an idea of how it would work.

[-1] 2016 https://github.com/rezan/varnish-cache/commit/b547bd9ad2fca9db1ef17ee73b8e9b7df9950c34

[0] 2019 https://github.com/varnishcache/varnish-cache/compare/master...rezan:req_objects

[1] https://github.com/rezan/libvmod-types