Dnsdist - DNS Load Balancing for Pi-hole Setup

I thought it would be useful to use a load balancer with health check functionality to distribute the DNS traffic from devices over multiple Pi-hole instances (pointing to Unbound). I had to search for a while and came across this post. And since then I’ve been looking into dnsdist, a beautiful piece of software.

Dnsdist Configuration (dnsdist.conf)

After some research, including going through these examples, I came up with the following configuration for dnsdist:

setLocal('0.0.0.0:53')
addLocal('[::]:53')

-- pc = newPacketCache(10000, {})

pool_backend = 'backend'

newServer({
	name = 'pihole-host1',
	address = '192.168.xx.xxx:53',
	pool = pool_backend,
	useClientSubnet=true,
	healthCheckMode='lazy',
	checkInterval=30,
	lazyHealthCheckFailedInterval=30,
	rise=2,
	maxCheckFailures=3,
	lazyHealthCheckThreshold=20,
	lazyHealthCheckSampleSize=100,
	lazyHealthCheckMinSampleCount=10,
	lazyHealthCheckUseExponentialBackOff=true,
	lazyHealthCheckMode='TimeoutOnly',
})
newServer({
	name = 'pihole-host2',
	address = '192.168.xx.xxx:53',
	pool = pool_backend,
	useClientSubnet=true,
	healthCheckMode='lazy',
	checkInterval=30,
	lazyHealthCheckFailedInterval=30,
	rise=2,
	maxCheckFailures=3,
	lazyHealthCheckThreshold=20,
	lazyHealthCheckSampleSize=100,
	lazyHealthCheckMinSampleCount=10,
	lazyHealthCheckUseExponentialBackOff=true,
	lazyHealthCheckMode='TimeoutOnly',
})

setPoolServerPolicy(roundrobin, pool_backend)
setECSSourcePrefixV4(32)

-- getPool(pool_backend):setCache(pc)

addAction(
	AllRule(),
	PoolAction(pool_backend)
)

setSecurityPollSuffix('')

controlSocket('127.0.0.1:5199')
setKey('pVC5gO/HECwOfgFzQDjAy6v5mWYmpwcj2h546GjqDgg=')

Below I explain the choices I made:

setLocal(‘0.0.0.0:53’)
addLocal(‘[::]:53’)
Udp/tcp dns listening, bind on ip any.

pc = newPacketCache(10000, {})
It is good to know that it is possible to use caching, I have disabled this because that’s what I use Pi-hole for..

pool_backend = ‘backend’
Servers can be part of any number of pools. Servers that are not assigned to a specific pool get assigned to the default pool that is always present. I chose to create my own backend pool.

newServer name = ‘pihole-host1’, address = ‘192.168.xx.xxx:53’, pool = pool_backend, useClientSubnet=true,
Create a server for each Pi-hole instance. Fill in the name and ip address with port number. To make sure that the source address of devices is passed from dnsdist to Pi-hole, use useClientSubnet=true, otherwise you will only see the name of the host on which dnsdist is installed in the query log of Pi-hole. More information about the useClientSubnet option can be found here. Unfortunately, you can not use the proxy protocol with Pi-hole.

(newServer) healthCheckMode=’lazy’, checkInterval=30, lazyHealthCheckFailedInterval=30, rise=2, maxCheckFailures=3, lazyHealthCheckThreshold=20, lazyHealthCheckSampleSize=100, lazyHealthCheckMinSampleCount=10, lazyHealthCheckUseExponentialBackOff=true, lazyHealthCheckMode=’TimeoutOnly’,
These options are for the purpose of the health check. To check if the configured Pi-hole instances are up or down. You can have this done continuously or “lazy”, i.e. when search results fail. For my home situation, “lazy” is sufficient because I also want to avoid making a request every x seconds. With the ‘checkName’ field you can indicate which address is used to perform the health check. Default is a.root-servers.net and that seems no problem to me.
I’ll go through a few options:
- checkInterval The time in seconds between health checks.
- lazyHealthCheckFailedInterval seconds and double between every probe.
- rise (default is 1) Require number consecutive successful checks before declaring the backend up.
- maxCheckFailures (default is 1) Allow number check failures before declaring the backend down.
- lazyHealthCheckThreshold (default is 20) The means 20% of the last lazyHealthCheckSampleSize queries should fail for a health-check to be triggered.
- lazyHealthCheckSampleSize (default is 100) Default is 100, which means the result (failure or success) of the last 100 queries will be considered.
- lazyHealthCheckMinSampleCount (default is 1) The minimum amount of regular queries that should have been recorded before the lazyHealthCheckThreshold threshold can be applied.
- lazyHealthCheckUseExponentialBackOff (default is false) When set to true, the delay between each probe starts at lazyHealthCheckFailedInterval seconds and double between every probe, capped at lazyHealthCheckMaxBackOff seconds. Using this option is great for my home network and will lower the amount of requests again.
- lazyHealthCheckMode (default is TimeoutOrServFail) ‘TimeoutOnly’ means that only timeout and I/O errors of regular queries will be considered. This is what I need with the Pi-hole instances.

setPoolServerPolicy(roundrobin, pool_backend)
Set the load balancing policy to use. Roundrobin seems like a good choice to me.

setECSSourcePrefixV4(32)
If you use the option useClientSubnet=true: Set the source prefix-length for the EDNS Client Subnet option.

getPool(pool_backend):setCache(pc)
If you use the option pc = newPacketCache(10000, {}) then enable cache for the pool.

addAction( AllRule(), PoolAction(pool_backend) )
Matches all incoming traffic and send-it to the backend pool.

setSecurityPollSuffix(‘’)
Disable security feature polling. Personally, I don’t think security polling to the domain secpoll.powerdns.com is necessary. If this setting is made empty, no polling will take place else if the result is that a given version has a security problem, the software will report this at level ‘Error’ during startup.

controlSocket(‘127.0.0.1:5199’)
setKey(‘pVC5gO/HECwOfgFzQDjAy6v5mWYmpwcj2h546GjqDgg=’)
Use the console to manage dnsdist and enable encryption with setkey. With the makeKey() command you can generate a key.

Dnsdist Installation

There are several ways to install dnsdist. Specifically for NixOS, I have described the installation in the next chapter.

If dnsdist is running, you can use the sudo dnsdist -c command to go to the console and view the servers with the command showServers(). You can also generate a (new) key with the command makeKey(). Leave the console with quit. See also the screenshot in the next chapter.

Dnsdist NixOS Installation

Add the following to configuration.nix to install dnsdist with the above configuration:

   # DNS loadbalancer
  services.dnsdist = {
       enable = true;
       listenAddress = "0.0.0.0"; # Default
       listenPort = 53; # Default
       extraConfig =
                "-- Listen\n" +
                # This is configured with the listenAddress and listenPort options
                "-- addLocal('0.0.0.0:53')\n" +
                "addLocal('[::]:53')\n" +
                "\n" +
                "-- dns caching\n" +
                "-- pc = newPacketCache(10000, {})\n" +
                "\n" +
                "-- Pools\n" +
                "pool_backend = 'backend'\n" +
                "\n" +
                "-- members definition\n" +
                "newServer({\n" +
                  "name = 'pihole-host1',\n" +
                  "address = '192.168.xx.xxx:53',\n" +
                  "pool = pool_backend,\n" +
                  "useClientSubnet=true,\n" +
                  "healthCheckMode='lazy',\n" +
                  "checkInterval=30,\n" +
                  "lazyHealthCheckFailedInterval=30,\n" +
                  "rise=2,\n" +
                  "maxCheckFailures=3,\n" +
                  "lazyHealthCheckThreshold=20,\n" +
                  "lazyHealthCheckSampleSize=100,\n" +
                  "lazyHealthCheckMinSampleCount=10,\n" +
                  "lazyHealthCheckUseExponentialBackOff=true,\n" +
                  "lazyHealthCheckMode='TimeoutOnly',\n" +
                "})\n" +
                "newServer({\n" +
                  "name = 'pihole-host2',\n" +
                  "address = '192.168.xx.xxx:53',\n" +
                  "pool = pool_backend,\n" +
                  "useClientSubnet=true,\n" +
                  "healthCheckMode='lazy',\n" +
                  "checkInterval=30,\n" +
                  "lazyHealthCheckFailedInterval=30,\n" +
                  "rise=2,\n" +
                  "maxCheckFailures=3,\n" +
                  "lazyHealthCheckThreshold=20,\n" +
                  "lazyHealthCheckSampleSize=100,\n" +
                  "lazyHealthCheckMinSampleCount=10,\n" +
                  "lazyHealthCheckUseExponentialBackOff=true,\n" +
                  "lazyHealthCheckMode='TimeoutOnly',\n" +
                "})\n" +
                "\n" +
                "-- set the load balacing policy to use\n" +
                "setPoolServerPolicy(roundrobin, pool_backend)\n" +
                "\n" +
                "-- set the source prefix-length for the EDNS Client Subnet option\n" +
                "setECSSourcePrefixV4(32)\n" +
                "\n" +
                "-- enable cache for the pool\n" +
                "-- getPool(pool_backend):setCache(pc)\n" +
                "\n" +
                "-- Rules\n" +
                "-- matches all incoming traffic and send-it to the pool of resolvers\n" +
                "addAction(\n" +
                  "AllRule(),\n" +
                  "PoolAction(pool_backend)\n" +
                ")\n" +
                "\n" +
                "-- disable security feature polling\n" +
                "setSecurityPollSuffix('')\n" +
                "\n" +
                "-- Use the console to manage dnsdist\n" +
                "controlSocket('127.0.0.1:5199')\n" +
                "-- Generate a key with makeKey()\n" +
                "setKey('pVC5gO/HECwOfgFzQDjAy6v5mWYmpwcj2h546GjqDgg=')\n" +
                "\n"
    ;
  };

Adjust the following:

“address = ‘192.168.xx.xxx:53’,\n”
Replace with the IP address of your Pi-hole instances.

And if you use the firewall, don’t forget to open port 53:

networking.firewall.allowedTCPPorts = [ 53 80 443 8080 ]; # DNS and HTTP(S)
networking.firewall.allowedUDPPorts = [ 53 ]; # DNS

Switch to the new configuration with sudo nixos-rebuild switch and make sure the dnsdist.service is running with the command systemctl status dnsdist.service. You can find more information about my NixOS configuration here.

To start the dnsdist console, you can copy the link after cGroup and paste as command and add the parameter -c, for example:

/nix/store/b5ipbxi5r7b72rbyfgqjdxzixkvsml1z-dnsdist-1.8.3/bin/dnsdist --supervised --disable-syslog --config /nix/store/flya5h2kz75zyafn2j2445vviqx01m4w-dnsdist.conf -c

And then you can also check the dnsdist configuration, for example:

cat /nix/store/flya5h2kz75zyafn2j2445vviqx01m4w-dnsdist.conf

Testing Dnsdist

If dnsdist is running and at least one Pi-hole server is up, from another Linux client you can check whether the dns request is sent to the Pi-hole server via dnsdist as follows:

 dig pi-hole.net @192.168.xx.xxx -p 53

Adjust the following:

192.168.xx.xxx
Replace this with the IP address of your dnsdist host

You can then test the roundrobin policy by executing a second request, which should end up in the second Pi-hole instance (check this with showServers() in the dnsdist console).

Read other notes