Prioritize backends using an ACL during selection

[thomasklinger1234]

Summary

This PR aims to serve as a discussion point for #132.

The VCL interface for directory is extended with a new ACL prefer parameter that - if set - will prioritize backends matching this ACL during resolution. The idea is to implement use cases such as "prefer IPv6" or "prefer nearest datacenter".

Contrary to whitelist, the prefer parameter will only be active during the resolve method in the director and does not filter out backends. This is important in particular for varnishadm backend.list which still shows all backends available. If there are not matches for the preferred ACL, the existing logic applies.

Notable changes

• Add a new prefer parameter to dynamic.director(...)
• Extend varnishadm backend.list -j -p JSON output with the preferred field
• All dynamic_ref instances have a marker preferred
• dom_find has been updated

Open topics

• Add new VTC tests
• Update README.rst

[thomasklinger1234]

Honestly, I tried to keep the function as is but did not find a good way to easily integrate backend selection flow for preferred with the existing implementation. If there is a better way, I am open to refactor!

[thomasklinger1234]

dom->current is not heavily used for iteration. This is probably not the best way

[thomasklinger1234]

Should be renamed to dom_find

[thomasklinger1234]

Could this be extended to something like score or priority? In general, the task at hand is a special scoring problem. This would match the way that HAProxy implements their prefer-ipv6 feature.

[thomasklinger1234]

Currently in draft for discussion, feedback and verification. My tests show good and correct results but I have not tested edge cases yet.

The VCL i have used is

vcl 4.1;

import dynamic from "<workdir>/src/.libs/libvmod_dynamic.so";

import std;
import directors;

backend default {
	.host = "127.0.0.1";
	.port = "80";
	.max_connections = 30;
	.wait_limit = 60;
	.wait_timeout = 10s;
}

acl ipv6_only { "::"/0; }
#acl ipv4_only { "0"/0; }
#acl ipv_all { "0"/0; "::"/0; }

sub vcl_init {
        new dyn_default = dynamic.director(
            ttl = 30s,
            port = 80,
            domain_usage_timeout = 3s,
            first_lookup_timeout = 3s,
            retry_after = 10s,
            keep = 10,
            host_header = "example.com",
            authority = "example.com",
            prefer = ipv6_only,
        );
        new dyn_default_dir = directors.round_robin();
        dyn_default.debug(1);
        dyn_default_dir.add_backend(dyn_default.backend(
            "example.com",
            "80",
        ));
}

sub vcl_recv {

}

sub vcl_backend_fetch {
    set bereq.http.host = "example.com";
    set bereq.backend = dyn_default.backend("example.com", "80");
}

[nigoroll]

@thomasklinger1234 I do very much appreciate your efforts, thank you.

Incidentally I have now come back to this topic and would like to try an idea: Generalize this a bit and try to reuse the service code.
Please give me a moment.