When using a validating resolver, we should display the result of validation (the AD bit).

Maybe '?format=RAW' could return the original response message as received from the DNS server in ("wire format"), so that the querier can do his own DNS message decoding?

Should be quite simple with DNSpython to_wire()

MIME type should be application/dns, RFC 2540.

Your distribute_setup.py module fails to retrieve whatever it's trying to download and blocks building the package. Simply removing references to it from setup.py fixes the problem.

% python setup.py build
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.16.tar.gz
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    use_setuptools()
  File "/home/matt/src/dns-lg/distribute_setup.py", line 145, in use_setuptools
    return _do_download(version, download_base, to_dir, download_delay)
  File "/home/matt/src/dns-lg/distribute_setup.py", line 124, in _do_download
    to_dir, download_delay)
  File "/home/matt/src/dns-lg/distribute_setup.py", line 193, in download_setuptools
    src = urlopen(url)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: SSL is required

See http://dns.bortzmeyer.org/dangerousrecord.broken-on-purpose.generic-nic.net/MX The / in the resource record data are not escaped and it seems Apache (at least with the conf we use) do something special with them, such as merging two consecutive slashes into one (it does not happen with other HTTP servers such as Python's BaseHTTPServer.

I had no issues getting this great project to run on Apache, but I wanted to use something more lightweight. Tried a few different servers, but application would not run properly. I am not a Python programmer, but hacked at it and got it to work on FAPWS (http://www.fapws.org/) and CherryPy (http://docs.cherrypy.org/stable/refman/wsgiserver/init.html).

Providing my diff below...such as it is, in case it is of any interest. Works for me, but I'm sure some better fixes could be made.

Patch to _init_.py

123c123,124

< raise Exception("Internal error: no / at the beginning of %s" % path)

        path = '/' + path
        # raise Exception("Internal error: no / at the beginning of %s" % path)

398c399

< return self.query(start_response, Request(environ), pure_path, client, format, resolver,

        return self.query(start_response, Request(dict(environ)), pure_path, client, format, resolver,

CherryPy test server code..

!/usr/bin/env python

from cherrypy import wsgiserver
import DNSLG

port = 8080
email_admin = "foobar@invalid"
url_doc = None
url_css = None

querier = DNSLG.Querier(email_admin, url_doc, url_css)
wsgi_app = querier.application

server = wsgiserver.CherryPyWSGIServer(('0.0.0.0', 8080), wsgi_app, request_queue_size=500, server_name='localhost')

if name == 'main':
try:
server.start()
except KeyboardInterrupt:
server.stop()

FAPWS test server code

!/usr/bin/env python

import fapws._evwsgi as evwsgi
from fapws import base
from fapws.contrib import cgiapp
import DNSLG

port = 8080
email_admin = "[email protected]"
url_doc = None
url_css = None

def start():
evwsgi.start("0.0.0.0", "8080")
evwsgi.set_base_module(base)

querier = DNSLG.Querier(email_admin, url_doc, url_css)
evwsgi.wsgi_cb(("/",querier.application))

evwsgi.set_debug(0)
evwsgi.run()

if name=="main":
start()

To do so, a RESTful solution is to use the Prefer: HTTP header (RFC approved but not yet published):

GET /example.org/AAAA HTTP/1.0
Prefer: wait=10

As a proxy DNS service, it would be useful to have an option to implement the EDNS-client-subnet proposed extension.

This way GSLB and CDNs services could treat the request as if it was actually coming from the requestor.

A dns-lg user could then easily search for differences (like a french IP using a US-based dns-lg would showcase a user experience of CDNs when using OpenDNS and Google Public DNS).

It would be nice to have a HTTP Expires: header, set to the minimum of the DNS TTLs of the answers.

500 Internal Server Error on https://dns.bortzmeyer.org/e164enum.net/NS?server=e.gtld-servers.net (but it crashes for every domain and server)

When the DNS looking glas uses several resolvers, and when they have a different use of DNSSEC (for instance, some validate and some don't), the results are not perfect. Unfortunately, since there is no DNS response code "DNSSEC validation failure", I'm not sure there is a right solution, SERVFAIL is a very ambiguous response code.

Take for instance servfail.nl, which is deliberately broken. If the DNS looking glass uses only validating resolvers, we get:

http://dnslg.generic-nic.net/servfail.nl/SOA => "No server replies for domain servfail.nl" (because the program tries the next resolvers, after a SERVFAIL. Is it a good idea?)

If the DNS looking glass uses one validating resolver then ordinary resolvers, it "succeeds":

http://dns.bortzmeyer.org/servfail.nl/SOA => "Start Of Authority: Zone administrator hostmaster.forfun.net., master server li1.forfun.net., ..." Should it stop instead at the first SERVFAIL?

Python 2 is now clearly going to be deprecated so a port to Python 3 is necessary. (Before we resume fixing bugs.)

HTTP HEAD is today refused. It would be nice to have it properly managed but it is not obvious how to do it properly inside the WSGI framework.

It would be nice to have DoH (RFC 8484) support in the server. See the work done at the IETF hackathon https://www.bortzmeyer.org/hackathon-ietf-101.html

It would be nice to have support for rrtype URI.

Does anyone know a URI in the wild, to test?

As of today, URI support is not in dnspython.

It could be cool to have a way to know automatically your (actually, your Web proxy and/or NATed address) IP address. Something like http://dns.example.net/?whoami=1 ?

Query for domain _443._tcp.www.bortzmeyer.org., type TLSA
Unknown record type (52)
(Time-to-Live of this answer is 1 day, 0 second)

A better formatting is wished.

It would be nice to have a proper database of existing DNS LG instances. Currently, it is a file in DNS zone file format (dns-lg-zone.incl) but it would be better to have a structured file (YAML) and to derive automatically Web pages and DNS zones (using records TXT or URI?)

When there are several resolvers in resolv.conf, and the first one was not used (timeout or another problem), the "resolver used" in the output is always the first one, not the one actually used.

https://dns.bortzmeyer.org/hubone.me/CAA
Unknown record type (257)
Unknown record type (257)
Unknown record type (257)
Unknown record type (257)

Better formatting wished

Currently, we produce JSON according to a proprietary format. (See JSON.txt)

It would be nice to have also the output format described in RFC 8427, tagged as application/dns+json

DNSpython has two interfaces to the resolver, a high-level one (you create a dns.resolver.Resolver() object and then call its query() method) and a low-level one (create a message with dns.message.make_query() then call dns.query.udp(the_message). The first one is easier to use but some things cannot be done (for instance, it stupidly retries when the resolver returns SERVFAIL, until the timeout). The second interface provides a better control. DNS Looking Glass uses the high-level interface.

May be we should modify DNS Looking Glass to switch to the low-level one? Some things like parsing resolv.conf or handling retries and timeouts would have to be done by us, in that case.

"No data" for /wroe.com/TXT while there are actually several TXT records.

Example is syndirag.dirag.meteo.fr.

Classes (not DNS classes) within each HTML element so that one can fine tune
presentation in the CSS stylesheet.

Several records in https://dns.bortzmeyer.org/dangerousrecord.broken-on-purpose.generic-nic.net/MX are not interpreted correctly when you click on it (lack of escape) :

Record type SRC='HTTP: does not exist

or things like that.

Currently the sample scripts have a hardcoded QType, Go is ADDR and Perl is A.

Perhaps it is worth adding an optional CLI argument to specify a QType and fallback to an ADDR query?

https://dns.bortzmeyer.org/xn--kprw13d/DNAME

Unknown record type (39)

Would be nice to be able to http://dns.bortzmeyer.org/www.illyse.org/CH (or else) and get "OpenOffice DNS server 1.0".

It's great to have the time of the test in the data, especially if you store them and read it later. JSON and XML output miss this info.

It would be nice to use HTTP content negotiation, in addition to the explicit "format" parameter.

No idea how to do it in Python: HTTP headers are surprisingly difficult to parse and the header can be as complicated as (example from the RFC):

Accept: text/plain; q=0.5, text/html,
           text/x-dvi; q=0.8, text/x-c

Possible references: http://www.xml.com/pub/a/2005/06/08/restful.html (with Python code), https://github.com/martinblech/mimerender (Python module to implement that), http://pythonpaste.org/httpencode/module-httpencode.mimeparse.html (Python module at a lower level, just does the parsing)

[Tue Dec 18 13:15:31 2012] [error] [client 88.189.152.187] NoOptionError: No option 'favicon' in section: 'DNS-LG'

(But the documentation says a favicon is optional)

DNS Python already has it http://www.dnspython.org/docs/1.10.0/html/dns.rdtypes.ANY.NSEC3PARAM.NSEC3PARAM-class.html

We need a cd=1 option in the URL, so we can try with a validating resolver and domains like www.dnssec-failed.org...

Apparently, there is no way to set the CD bit in DNSpython with the high-level resolver interface. This is why this bug depends on #3.

The two following example queries trigger a server error :

labels > 63 characters :

http://dns.bortzmeyer.org/abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz.abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz/SOA

domains > 253 characters :

http://dns.bortzmeyer.org/abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz.abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz.abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz.abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz.abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz/NS

It would be nice to have the name of the DNSSEC algorithm, not just the number.

Two possible strategies: hardcode by hand the most common, or retrieve http://www.iana.org/assignments/dns-sec-alg-numbers/dns-sec-alg-numbers.xml (it's in XML) and parse it.

I was hoping (without looking at the code) that the server would allow me to specify a non-standard content-type:

curl -H 'Accept: application/zone' http://dnslg.prox:8080/example.org/A
A server error occurred.  Please contact the administrator.

But it doesn't:

Traceback (most recent call last):
  File "/usr/lib64/python2.6/wsgiref/handlers.py", line 93, in run
    self.result = application(self.environ, self.start_response)
  File "/home/jpm/dns-lg/DNSLG/__init__.py", line 397, in application
    do_dnssec, tcp, cd, edns_size, reverse)
  File "/home/jpm/dns-lg/DNSLG/__init__.py", line 283, in query
    formatter.format(answer, qtype, qclass, answer.flags, self)
UnboundLocalError: local variable 'formatter' referenced before assignment
172.16.153.1 - - [24/Feb/2013 17:59:03] "GET /example.org/A HTTP/1.1" 500 59

;-)

Info: latest version, launched with

python test-server.py

Because that's what available in Python :-(

Example: under old IDN, www.straße.de was converted to www.strasse.de. Under IDNAbis, it is now www.xn--strae-oqa.de. These two names have different IP addresses and the LG is showing the old one.

Error bodies are currently always in text/plain. They should use the requested format, whether it has been indicated by the "format" parameter, or negotiated with HTTP (see issue #10).