multiformats / go-multiaddr-dns

Go library and CLI tool for /dns4, /dns6, /dnsaddr multiaddr resolution

multiformats

The main repository for discussing multiformats

Multiformats is a set of self-describing protocol values. These values are foundational in that they are low-level building blocks for both data and network layers of the composable protocols making up IPFS, IPLD, libp2p, and many other decentralized data systems. This repository's issues and pull requests are currently the primary venue for the coordination between the various registries making up the group, each of which is separately being hardened as specifications and public, formal registries over time.
See contributing.md for more details on governance and process.

Current Registries

Currently, we have the following formats, each of which corresponds to a specification and a registry. More formats are being discussed and may be added over time, but the following are the mature ones to date:

Repo	Status	Specification
multiaddr	stable	TBD
multibase	stable	W3C CCG
multicodec	stable	TBD
multihash	stable	W3C CCG

See the project directory, below, for implementations and other related repositories.

Table of Contents

• Current Registries
• Table of Contents
• Background
  • A note on the word Multiformats
• Project Directory
  • Implementations
• Maintainers
• Contribute
• License

Background

Every choice in computing has a tradeoff. This includes formats, algorithms, encodings, and so on. And even with a great deal of planning, decisions may lead to breaking changes down the road, or to solutions which are no longer optimal. Allowing systems to evolve and grow is important.

Multiformats is a collection of protocols which aim to future-proof systems, today. They do this mainly by allowing data to be self-describable. This allows interoperability, protocol agility, and helps us avoid various forms of lock-in. Currently, these interlocking protocols (both works in progress and implemented) cover the following areas:

• multiaddr: network addresses
• multibase: base encodings
• multicodec: serialization codes
• multihash: cryptographic hashes
• multikey: cryptographic keys and artifacts

Several of the multiformats are stable, and work on the others is ongoing. Implementers and refiners of the drafts of any one of these registries or their tooling are welcome to contribute, without needing to understand deeply or track progress on the others. Across these otherwise different use-cases and mechanisms, the self-describing aspects of the protocols have a few design goals in common:

• the "prefixes" use to self-describe a value must be inline with the value (not passed out-of-band, in function calls, implicit choices, or documentation);
• they must be compact and have a binary-packed representation (as opposed to a sparser encoding) or they will hinder performance;
• they must have a human-readable representation.

A note on the word Multiformats

Multiformats is the name for the community (and the "organization" in GitHub's access control model), but multiformats can also be used to refer to protocols; for instance, in the sentence "Use one of the multiformats". Formats is interchangeable with protocols, here, as each format is designed in tandem with one or more protocols which handle those self-describing values centrally. We try to capitalize Multiformats when it refers to the organization.

Project Directory

Below, a list of all of the projects in the Multiformats organization is listed.

Maintainers are the active leads for each project, even if the specification is still under construction. Their responsibilities are to make sure that issues and pull requests are attended to in a timely manner, and general upkeep. If you have questions about a repository, or need feedback, please contact them as appropriate. If any of the specifications defining these formats are formalized in a standards body, these maintainers may continue on as Registrars of the table of entries which can keep growing after stabilizing the syntax and tooling interfaces.

Implementations

There are no official or maintained implementations of the entire set of multiformats specifications and registries. The readme file of each multiformat specification repository includes a list of known implementations, some of which are hosted in this GitHub organization.

Maintainers

Maintainers of the each multiformats specification are listed in the appropriate repositories. The external standardization of multiformats specifications and registries is currently managed and coordinated by @bumblefudge of learningProof UG.

Contribute

Check out our contributing document for more information on how we work, and about contributing in general.

License

This repository is only for documents. All of these are licensed under the CC-BY-SA 3.0 license © 2016 Protocol Labs Inc. Any code is under a MIT © 2016 Protocol Labs Inc.

go-multiaddr-dns's Issues

Make Readme good

Make a proper madns CLI

dnsaddr code?

The code for dnsaddr is not consistent.

• go uses code 12345
• js uses code 53
• protocols.csv does not mention dnsaddr but has a dns at code 53.

Can someone please clarify! cc: @diasdavid @whyrusleeping

End-to-end and CLI tests

Should be able to:

• plug in a local DNS server/resolver
• can use a different madns executable (e.g. js?)

Interface unit tests and fuzzing

Support and test /dns/

It seems that this library supports /dns4 and /dns6 but does not support /dns

Given that

• JS multiformats/js-multiaddr#253 is adding support for /dns/
• and it is a valid code present in both multiaddr/protocols.csv and multicodec/table.csv

i think we should add it here as well.

Resolution order

Reuse prior art from https://en.wikipedia.org/wiki/Happy_Eyeballs ?

Filtering by protocol stack

The -p on the CLI filters results by protocol stack. Not sure yet what the interface around Resolve() should be.

> madns -p /ip6/tcp/wss /dnsaddr/example.net
/ip6/2001:db8::a3/tcp/443/wss/ipfs/Qmfoo
/ip6/2001:db8::a4/tcp/443/wss/ipfs/Qmbar

Add DNS Cache that respects TTL of resolved DNS records

Problem

When DNS over HTTPS resolver is set up, every query produces HTTP request.

To illustrate, when using https://resolver.cloudflare-eth.com/dns-query for crypto TLDs, this adds around 1 second to EVERY resolution event (only the very first one should be that slow):

$ t ipfs name resolve /ipns/brave.crypto
/ipfs/QmWrdNJWMbvRxxzLhojVKaBDswS4KNVM7LvjsN7QbDrvka
  0.25s user 0.03s system 21% cpu 1.274 total

$ t ipfs name resolve /ipns/brave.crypto                                                                                                                                                    /ipfs/QmWrdNJWMbvRxxzLhojVKaBDswS4KNVM7LvjsN7QbDrvka
  0.21s user 0.02s system 20% cpu 1.100 total

$ t ipfs name resolve /ipns/brave.crypto                                                                                                                                                   
/ipfs/QmWrdNJWMbvRxxzLhojVKaBDswS4KNVM7LvjsN7QbDrvka
  0.23s user 0.02s system 23% cpu 1.058 total

To illustrate, using non-DoH name leverages OS-level resolver as a cache, and only first lookup is expensive:

$ t ipfs name resolve /ipns/en.wikipedia-on-ipfs.org                                                                                                                                        
/ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze
  0.23s user 0.05s system 80% cpu 0.343 total

$ t ipfs name resolve /ipns/en.wikipedia-on-ipfs.org                                                                                                                                        /ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze
  0.04s user 0.01s system 173% cpu 0.029 total

$ t ipfs name resolve /ipns/en.wikipedia-on-ipfs.org                                                                                                                                        /ipfs/bafybeiaysi4s6lnjev27ln5icwm6tueaw2vdykrtjkwiphwekaywqhcjze
  0.03s user 0.02s system 160% cpu 0.032 total

Solution

The problem is not limited to DoH – there is no point in hitting OS-level default resolver if we could use cached record.
We need to add a global DNS Cache to resolve.go that makes sure we don't do redundant lookups until TTL from resolved record expires.

cc @aschmahmann @vyzo what would be the least painful way to solve this globally?

Add caching to default resolver

IIRC, normally the code will use the default net.DefaultResolver and has no caching layer.