VIP9: Expand VCL object support

Synopsis

Currently objects are limited to global objects which have a lifetime of the entire VCL. This VIP is to have objects which can be created during the request and their scope is limited to that request. When the request is done, the objects are fini'ed.

Original VIP9 date: April 25, 2016

Updated VIP9 date: November 18, 2019

See also: VIP9B

Why?

By allowing VMOD objects to live in the request scope, we can easily bring in OO style types and high level programming constructs to VCL. For example: integers, strings, doubles, lists, hash tables, encryption, encoding, anything and everything.

VCL Example

Here is a VCL snippet which compiles and works with the patch [0] and uses a new libvmod_types [1].

vcl 4.1;

import types;

sub vcl_recv
{
  //new req scoped objects
  new slocal = types.string("Request scoped string");
  new s2 = types.string("request string two");
  new count = types.integer(1);
}

sub vcl_backend_fetch
{
  //new bereq scoped objects
  new sbe = types.string("berequest string v1");
  set bereq.http.sbe = sbe.value();

  sbe.set("berequest string v2");
  set bereq.http.sbe2 = sbe.value();
}

sub vcl_deliver
{
  //referencing objects
  set resp.http.X-slocal = slocal.value();
  set resp.http.X-slocal-length = slocal.length();

  count.increment(10);
  set resp.http.count = count.value();
}

Another example of using an HTTP vmod in a request scope:

import http;

sub vcl_recv
{
  new server_ping = http.init();
  server_ping.req_set_url("https://server.co/api/ping");
  server_ping.req_send();
}

sub vcl_deliver
{
  server_ping.resp_wait();
  set resp.http.X-ping = server_ping.resp_get_status();
}

Scopes

Multiple levels of scoping are supported for $Objects. These requirements come from VIP9B.

.vcc scopes

An $Object can declare upfront in the .vcc file what scopes it supports:

$Scope [any, vcl, top, client, backend, local, vcl_recv, vcl_deliver, ....]

Multiple scopes can be defined on a single line. If no $Scope is declared, the object defaults to vcl scope, which is the currently supported object scope.

VCL scopes

VCL scopes are support via a qualifier keyword when using new. As long as the .vcc allows for it, the object will now be placed in that scope. Most of the time, the the vcc will be smart enough to figure out the scope based on where you define the object. Basically, there will be implicit default scopes for each sub. But in the cases where there are multiple scopes possible is where you would use this qualifier.

The supported scope qualifiers are: (local), (top), (vcl), (client), and (backend).

Here is an example using VMOD http in vcl_init in a local scope.

import http;

sub vcl_init
{
  new (local) ping = http.init();
  ping.req_set_url("https://server.co/api/ping");
  ping.req_send();
  ping.resp_wait();
  if (ping.resp_get_status() != 200) {
    return (fail);
  }
}

If the local scope qualifier wasn't used, the object would default to vcl scope and lead to a vcc compile error since this VMOD does not support VCL scope.

Runtime scopes

VMOD object constructors can optionally add a SCOPE parameter. For example:

$Object some_object(SCOPE, STRING something)

SCOPE will be an enum value of the scope being used: vcl, top, client, backend, local.

Because its an optional parameter, existing objects will not require any changes.

Object declaration and NULL checking

Objects can be declared anywhere in a sub vcl_*, including if statements, as long as the .vcc $Scope allows for it. Before an object method is invoked, the object will be null checked. This will be done once per object, per sub, per branch level. If a method is called on a NULL object (and NULL_OK was not defined), VCL_Fail() is called and the request fails with an error saying a NULL object was invoked.

Footnotes

[-1] 2016 https://github.com/rezan/varnish-cache/commit/b547bd9ad2fca9db1ef17ee73b8e9b7df9950c34

[0] 2019 https://github.com/varnishcache/varnish-cache/compare/master...rezan:req_objects

[1] https://github.com/rezan/libvmod-types