Coder Social home page Coder Social logo

perf-tests's Introduction

CoreDNS

Documentation CodeQL Go Tests CircleCI Code Coverage Docker Pulls Go Report Card CII Best Practices

CoreDNS is a DNS server/forwarder, written in Go, that chains plugins. Each plugin performs a (DNS) function.

CoreDNS is a Cloud Native Computing Foundation graduated project.

CoreDNS is a fast and flexible DNS server. The key word here is flexible: with CoreDNS you are able to do what you want with your DNS data by utilizing plugins. If some functionality is not provided out of the box you can add it by writing a plugin.

CoreDNS can listen for DNS requests coming in over:

Currently CoreDNS is able to:

  • Serve zone data from a file; both DNSSEC (NSEC only) and DNS are supported (file and auto).
  • Retrieve zone data from primaries, i.e., act as a secondary server (AXFR only) (secondary).
  • Sign zone data on-the-fly (dnssec).
  • Load balancing of responses (loadbalance).
  • Allow for zone transfers, i.e., act as a primary server (file + transfer).
  • Automatically load zone files from disk (auto).
  • Caching of DNS responses (cache).
  • Use etcd as a backend (replacing SkyDNS) (etcd).
  • Use k8s (kubernetes) as a backend (kubernetes).
  • Serve as a proxy to forward queries to some other (recursive) nameserver (forward).
  • Provide metrics (by using Prometheus) (prometheus).
  • Provide query (log) and error (errors) logging.
  • Integrate with cloud providers (route53).
  • Support the CH class: version.bind and friends (chaos).
  • Support the RFC 5001 DNS name server identifier (NSID) option (nsid).
  • Profiling support (pprof).
  • Rewrite queries (qtype, qclass and qname) (rewrite and template).
  • Block ANY queries (any).
  • Provide DNS64 IPv6 Translation (dns64).

And more. Each of the plugins is documented. See coredns.io/plugins for all in-tree plugins, and coredns.io/explugins for all out-of-tree plugins.

Compilation from Source

To compile CoreDNS, we assume you have a working Go setup. See various tutorials if you don’t have that already configured.

First, make sure your golang version is 1.21 or higher as go mod support and other api is needed. See here for go mod details. Then, check out the project and run make to compile the binary:

$ git clone https://github.com/coredns/coredns
$ cd coredns
$ make

This should yield a coredns binary.

Compilation with Docker

CoreDNS requires Go to compile. However, if you already have docker installed and prefer not to setup a Go environment, you could build CoreDNS easily:

docker run --rm -i -t \
    -v $PWD:/go/src/github.com/coredns/coredns -w /go/src/github.com/coredns/coredns \
        golang:1.21 sh -c 'GOFLAGS="-buildvcs=false" make gen && GOFLAGS="-buildvcs=false" make'

The above command alone will have coredns binary generated.

Examples

When starting CoreDNS without any configuration, it loads the whoami and log plugins and starts listening on port 53 (override with -dns.port), it should show the following:

.:53
CoreDNS-1.6.6
linux/amd64, go1.16.10, aa8c32

The following could be used to query the CoreDNS server that is running now:

dig @127.0.0.1 -p 53 www.example.com

Any query sent to port 53 should return some information; your sending address, port and protocol used. The query should also be logged to standard output.

The configuration of CoreDNS is done through a file named Corefile. When CoreDNS starts, it will look for the Corefile from the current working directory. A Corefile for CoreDNS server that listens on port 53 and enables whoami plugin is:

.:53 {
    whoami
}

Sometimes port number 53 is occupied by system processes. In that case you can start the CoreDNS server while modifying the Corefile as given below so that the CoreDNS server starts on port 1053.

.:1053 {
    whoami
}

If you have a Corefile without a port number specified it will, by default, use port 53, but you can override the port with the -dns.port flag: coredns -dns.port 1053, runs the server on port 1053.

You may import other text files into the Corefile using the import directive. You can use globs to match multiple files with a single import directive.

.:53 {
    import example1.txt
}
import example2.txt

You can use environment variables in the Corefile with {$VARIABLE}. Note that each environment variable is inserted into the Corefile as a single token. For example, an environment variable with a space in it will be treated as a single token, not as two separate tokens.

.:53 {
    {$ENV_VAR}
}

A Corefile for a CoreDNS server that forward any queries to an upstream DNS (e.g., 8.8.8.8) is as follows:

.:53 {
    forward . 8.8.8.8:53
    log
}

Start CoreDNS and then query on that port (53). The query should be forwarded to 8.8.8.8 and the response will be returned. Each query should also show up in the log which is printed on standard output.

To serve the (NSEC) DNSSEC-signed example.org on port 1053, with errors and logging sent to standard output. Allow zone transfers to everybody, but specifically mention 1 IP address so that CoreDNS can send notifies to it.

example.org:1053 {
    file /var/lib/coredns/example.org.signed
    transfer {
        to * 2001:500:8f::53
    }
    errors
    log
}

Serve example.org on port 1053, but forward everything that does not match example.org to a recursive nameserver and rewrite ANY queries to HINFO.

example.org:1053 {
    file /var/lib/coredns/example.org.signed
    transfer {
        to * 2001:500:8f::53
    }
    errors
    log
}

. {
    any
    forward . 8.8.8.8:53
    errors
    log
}

IP addresses are also allowed. They are automatically converted to reverse zones:

10.0.0.0/24 {
    whoami
}

Means you are authoritative for 0.0.10.in-addr.arpa..

This also works for IPv6 addresses. If for some reason you want to serve a zone named 10.0.0.0/24 add the closing dot: 10.0.0.0/24. as this also stops the conversion.

This even works for CIDR (See RFC 1518 and 1519) addressing, i.e. 10.0.0.0/25, CoreDNS will then check if the in-addr request falls in the correct range.

Listening on TLS (DoT) and for gRPC? Use:

tls://example.org grpc://example.org {
    whoami
}

Similarly, for QUIC (DoQ):

quic://example.org {
    whoami
    tls mycert mykey
}

And for DNS over HTTP/2 (DoH) use:

https://example.org {
    whoami
    tls mycert mykey
}

in this setup, the CoreDNS will be responsible for TLS termination

you can also start DNS server serving DoH without TLS termination (plain HTTP), but beware that in such scenario there has to be some kind of TLS termination proxy before CoreDNS instance, which forwards DNS requests otherwise clients will not be able to communicate via DoH with the server

https://example.org {
    whoami
}

Specifying ports works in the same way:

grpc://example.org:1443 https://example.org:1444 {
    # ...
}

When no transport protocol is specified the default dns:// is assumed.

Community

We're most active on Github (and Slack):

More resources can be found:

Contribution guidelines

If you want to contribute to CoreDNS, be sure to review the contribution guidelines.

Deployment

Examples for deployment via systemd and other use cases can be found in the deployment repository.

Deprecation Policy

When there is a backwards incompatible change in CoreDNS the following process is followed:

  • Release x.y.z: Announce that in the next release we will make backward incompatible changes.
  • Release x.y+1.0: Increase the minor version and set the patch version to 0. Make the changes, but allow the old configuration to be parsed. I.e. CoreDNS will start from an unchanged Corefile.
  • Release x.y+1.1: Increase the patch version to 1. Remove the lenient parsing, so CoreDNS will not start if those features are still used.

E.g. 1.3.1 announce a change. 1.4.0 a new release with the change but backward compatible config. And finally 1.4.1 that removes the config workarounds.

Security

Security Audits

Third party security audits have been performed by:

Reporting security vulnerabilities

If you find a security vulnerability or any security related issues, please DO NOT file a public issue, instead send your report privately to [email protected]. Security reports are greatly appreciated and we will publicly thank you for it.

Please consult security vulnerability disclosures and security fix and release process document

perf-tests's People

Contributors

chrisohaver avatar johnbelamaric avatar miekg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

perf-tests's Issues

k8s perf

I'm doing some simple perf tests on a k8s cluster:

Cluster is running in packet. Nodes are type 0 machines.

net.* performance

With SO_REUSEPORT we should be able to run multiple listeners for incoming messages and handle those. This comes with a couple of ifs:

  • I don't yet know how to do this
  • Maybe Go's net.* is not fast enough - fix that?
  • There might still be lots of low hanging Go userspace code in CoreDNS and Go DNS, so that we don't need to go this route.

GCE perf

I've started playing around in GCE.

3 machines:

  • instance-1: n1-standard-2
  • instance-2: n1-standard-4
  • sender: n1-standard-4

All installed with Ubuntu-1703. CoreDNS is 007 (release from github).

Packet perf

I'm playing around on packet.net and I'm finally seeing figures that I have been seeing on my local machine. I.e. a beefy type1 server does (locally!):

Queries per second:   223763.430646 qps

Doing the same thing over the network yields (obviously) much less:

Queries per second:   47829.543277 qps

Plan: Continuous perf testing

Continous performance testing for CoreDNS

Setup

drone.coredns.io (Packet machine) runs webhook behind a Caddy server. Caddy is used for Let's Encrypt TLS certs and only proxies.

Further more mbench is running to collect performance data (go test -bench=. output) and convert into prometheus metrics.

Prometheus is running locally, scrapping mbench.

Grafana is running locally displaying graphs from Prometheus.

Of all these things, only Caddy and mbench are not installed from the Debian repos.

Integration with Github

Webhook setup on github.com point to drone.io port.

Trigger a build from Github

On a PR add a comment: @mbench OK ..... Only the OK is needed. Only some people
can trigger these.

Trigger locally

Send a REST request to another endpoint (blocks everything execpt ::1 and 127.0.0.1).

Master branch

A local cron job trigger the unauthenticated local endpoint every 30 minutes.

Future

  • Each middleware should have more benchmarks
  • Publish this data somewhere?

Hit 100000 qps

% make LIMIT=40000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 40000 -n 100000 -o test.json
[05/01/17 09:21:25] Starting...
[05/01/17 09:21:28] Messages:
	Sent....: 100000;
	Received: 100000;
	Lost....: 0.


[05/01/17 09:21:28] Exiting...
% make LIMIT=50000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 50000 -n 100000 -o test.json
[05/01/17 09:21:36] Starting...
[05/01/17 09:21:38] Messages:
	Sent....: 100000;
	Received: 100000;
	Lost....: 0.


[05/01/17 09:21:38] Exiting...
% make LIMIT=60000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 60000 -n 100000 -o test.json
[05/01/17 09:21:41] Starting...
[05/01/17 09:21:53] Messages:
	Sent....: 100000;
	Received: 99934;
	Lost....: 66.


[05/01/17 09:21:53] Exiting...
% make LIMIT=60000 mig                                           12s ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 60000 -n 100000 -o test.json
[05/01/17 09:21:56] Starting...
[05/01/17 09:21:58] Messages:
	Sent....: 100000;
	Received: 100000;
	Lost....: 0.


[05/01/17 09:21:58] Exiting...
% make LIMIT=70000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 70000 -n 100000 -o test.json
[05/01/17 09:22:02] Starting...
[05/01/17 09:22:14] Messages:
	Sent....: 100000;
	Received: 99483;
	Lost....: 517.


[05/01/17 09:22:14] Exiting...
% make LIMIT=70000 mig                                           12s ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 70000 -n 100000 -o test.json
[05/01/17 09:22:15] Starting...
[05/01/17 09:22:16] Messages:
	Sent....: 100000;
	Received: 100000;
	Lost....: 0.


[05/01/17 09:22:16] Exiting...
% make LIMIT=80000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 80000 -n 100000 -o test.json
[05/01/17 09:22:21] Starting...
[05/01/17 09:22:22] Messages:
	Sent....: 100000;
	Received: 100000;
	Lost....: 0.


[05/01/17 09:22:22] Exiting...
% make LIMIT=90000 mig                                               ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 90000 -n 100000 -o test.json
[05/01/17 09:22:26] Starting...
[05/01/17 09:22:37] Messages:
	Sent....: 100000;
	Received: 99920;
	Lost....: 80.


[05/01/17 09:22:37] Exiting...
% make LIMIT=100000 mig                                                         11s ~/git/coredns/perf-tests/local-perf master
./bin/mig-linux_amd64 -s 127.0.0.1 -p 1053 -d domain.lst.mig -l 100000 -n 100000 -o test.json
[05/01/17 09:22:42] Starting...
[05/01/17 09:22:43] Messages:
	Sent....: 100000;
	Received: 100000;
	Lost....: 0.


[05/01/17 09:22:43] Exiting...

new k8s perf test

@chrisohaver how hard/easy is it to do a new k8s performance test with the current CoreDNS master branch (or the v008 release)? Because I think we should redo it: new cache, singleinflight gone from the proxy.

Also how hard is it to do this on a continuous basis?

Local perf test

This is on my laptop (4 core, 8 GB). Go 1.8.

Using the just commited utils in the "local-perf" directory.

(pprof) top10   
0.09s of 7.06s total ( 1.27%)
Dropped 108 nodes (cum <= 0.04s)
Showing top 10 nodes out of 165 (cum >= 1.53s)
      flat  flat%   sum%        cum   cum%
         0     0%     0%      5.24s 74.22%  runtime.goexit
         0     0%     0%      2.95s 41.78%  github.com/miekg/dns.(*Server).serve
         0     0%     0%      2.46s 34.84%  github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket.func1
         0     0%     0%      2.46s 34.84%  github.com/miekg/dns.HandlerFunc.ServeDNS
     0.04s  0.57%  0.57%      2.45s 34.70%  github.com/coredns/coredns/core/dnsserver.(*Server).ServeDNS
         0     0%  0.57%      2.33s 33.00%  github.com/coredns/coredns/middleware/whoami.(*Whoami).ServeDNS
     0.04s  0.57%  1.13%      2.33s 33.00%  github.com/coredns/coredns/middleware/whoami.Whoami.ServeDNS
     0.01s  0.14%  1.27%      1.56s 22.10%  github.com/miekg/dns.(*response).WriteMsg
         0     0%  1.27%      1.53s 21.67%  github.com/coredns/coredns/core/dnsserver.(*Server).ServePacket
         0     0%  1.27%      1.53s 21.67%  github.com/mholt/caddy.startServers.func1

Corefile included in that repo. CoreDNS is running with:
./coredns -conf ~/git/coredns/perf-tests/local-perf/Corefile

profile attached.
pprof.coredns.localhost:6053.samples.cpu.001.pb.gz

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.