martinheidegger / hyper-dns

DNS lookup for dat/hyper archives

hyper-dns

Advanced resolving of decentralized web links using different name systems such as DNS Txt entries and .well-known https lookups locations. It implements various naming systems such as dat, hyper and cabal but is extensible to support other systems as well.

🚀 Basic API

After installing it through npm, you can simply run it like this:

const { resolveProtocol, resolve, resolveURL } = require('hyper-dns')
const protocol = 'dat'
const domain = 'dat-ecosystem.org'
const key = await resolveProtocol(protocol, domain)

Note: You may need to wrap it in a async function until top level async/await lands.

That's it! 🎉 - in the key variable you will get the dat key or null, if it can't be found.

🧙‍♀️ What is this magic?

Different decentralized web systems have different means to resolve "names" to a decentralized document.

hyper-dns contains a variety of implementations. Many are using DNS TXT records that contain a key of specified pattern, but other means are possible as well. − (more in the Protocol Guide)

The power of hyper-dns in comparison to other, protocol-specific implementations is that it has a shared cache for all protocols, it works in he browser and does a list of things well. − (more in the Architecture Overview)

👩‍🎓 Further reading

• Architecture Overview …to learn how caching and other things work. 🕵️‍♀️
• API documentation …for getting to know the API in detail. 🧑‍💻
• Contribution Guide …because help is always welcome. 🥳
• Protocol Guide …for current protocols and adding new ones. 🤠

📜 License

MIT License

hyper-dns's Issues

Add warning for conflicting entries

Conflicting entries are currently deterministically resolved using the "highest relevant" node:

https://github.com/martinheidegger/hyper-dns/blob/main/resolve-context.js#L84-L88

however, as this may be unexpected behavior it may be better to add a warning to the output (like CORS configuration) and even with debugging disabled it may be a good idea throw an error message in that case.

Remove sqlite3 from default import

It's been really hard to get stuff working cross platform while better-sqlite3 is in the dependency tree since it doesn't have prebuilds the way other native dependencies in the hyper stack do.

It'd be nice if this dependency could be marked as optional or opt-in to unblock development.

Create Web test interface

Provide simple interface that allows to test hyper-dns in a browser.

Document how to write a good protocol specification

Both technically and in terms of specifications it would be good to write down the findings of what gotcha's one might find in a document.

Turn multiple lookups into async iterator

Currently the lookup of dns entries (.resolve) for multiple protocols uses one async handler.

As the various protocols may resolve at different speeds, it may be a good idea to turn this into an asynchronous iterator that servers each protocol as they resolve, with an automatic abort.

Add Command line API

It would be nice to have a good command line API that functions pretty simply like:

$ hyper-dns hyper://datproject.org
{
  "hyper": "ab123c...404eff"
}

etc.

Considering #4, it may be good (optionally?) return a JSON stream instead of an object, as the results may "trickle" in.

Considering #3, it may be good to optionally start a hyper-dns compatible lookup server?

Add dnslink compatibility

dnslink is necessary for implementing an IPFS/IPNS based lookup for websites.

hyper-dns lookup server

When looking-up dns-entries-over-https (DOH) we are running into the issue that we have to trust the servers, which are hosted by google/cloudflare at the moment as they can handle any load (more servers can be found here - though they may not support json).

With this in mind I am wondering to "offload" the name lookup for decentralized names to a federated service that behaves equally but reduces complexity by doing things like the .well-known lookups, conflict resolution (multiple entries) and decentralized lookups (etherns, ...).

This way we can serve the same (possibly better) performance than we can get with a simple DoH lookup, while at the same time supporting decentralization.

Document flushing of (SQLite) cache

Currently the SQLite cache can be flushed only manually. This behavior (and the apis) needs to be documented!

hyper-dns/cache-sqlite.js

Lines 24 to 42 in 2ab60f4

	async clear () {
	await lru.clear()
	run(Q_CLEAR, {})
	},
	async clearName (name) {
	await lru.clearName(name)
	run(Q_CLEAR_NAME, { name })
	},
	async close () {
	if (db !== null && db.open) {
	db.close()
	db = null
	}
	},
	async flush (timestamp) {
	timestamp = typeof timestamp === 'number' ? timestamp : Date.now()
	await lru.flush(timestamp)
	run(Q_FLUSH, { now: timestamp })
	},

Add support for DoH without JSON

Several servers (most notably opendns) don't support https://tools.ietf.org/html/rfc8427 (DoH over JSON) but fall back to using dns packets. There is the dns-packet library which should help with the encoding of dns packets as used in the prior work of many such as https://github.com/hildjj/dohdec/blob/master/lib/doh.js

Document DNS-over-HTTPS configuration recommendations

DNS-over-HTTPS uses google/cloudflare servers by default, which is important to note if you offer the dns-over-https option to users. They may want to configure that. There should also be a information on where to get other servers (https://dnscrypt.info/public-servers/) Additionally windows 10 supports the setup of dns-over-https on a system level (and it should probably be using the system's dns in that case) and there are system add-ons for linux. This means it needs to be possible to disable the dns-over-https on those systems.

Add "hyper-dns in 2 mins" section

The readme needs to explain the basics of hyperdns and the most important implications of using hyperdns in the browser / in node.