Need tests for CBOR Production and Consumption

6.4.1 CBOR Production

All DID document data structures expressed by the data model MUST be serialized to the CBOR representation according to the following production rules:

Data Type	CBOR Representation Type
map	A CBOR map (major type 5), where each entry is represented as a member of the CBOR map. The entry key is expressed as a CBOR string (major type 3) as the key, and the entry value according to its type, as defined in this table.
list	A CBOR array (major type 4), where each element of the list is added, in order, as a value of the array according to its type, as defined in this table.
ordered set	A CBOR array (major type 4), where each element of the list is added, in order, as a value of the array according to its type, as defined in this table.
datetime	A CBOR string (major type 3) formatted as an XML Datetime normalized to UTC 00:00 and without sub-second decimal precision. For example: `2020-12-20T19:17:47Z`.
string	A CBOR string (major type 3).
integer	A CBOR integer (major type 0 or 1), choosing the shortest byte representation.
double	A CBOR floating-point number (major type 7). All floating point values MUST be encoded as 64-bits (additional type value 27), even for integral values.
boolean	A CBOR simple value (major type 7, subtype 24) with a simple value of 21 (true) or 20 (false).
null	A CBOR simple value (major type 7, subtype 24) with a simple value of 22 (null).

6.4.1 CBOR Production

All DID document data structures expressed by the data model MUST be serialized to the CBOR representation according to the following production rules:

Data Type	CBOR Representation Type
map	A CBOR map (major type 5), where each entry is represented as a member of the CBOR map. The entry key is expressed as a CBOR string (major type 3) as the key, and the entry value according to its type, as defined in this table.
list	A CBOR array (major type 4), where each element of the list is added, in order, as a value of the array according to its type, as defined in this table.
ordered set	A CBOR array (major type 4), where each element of the list is added, in order, as a value of the array according to its type, as defined in this table.
datetime	A CBOR string (major type 3) formatted as an XML Datetime normalized to UTC 00:00 and without sub-second decimal precision. For example: `2020-12-20T19:17:47Z`.
string	A CBOR string (major type 3).
integer	A CBOR integer (major type 0 or 1), choosing the shortest byte representation.
double	A CBOR floating-point number (major type 7). All floating point values MUST be encoded as 64-bits (additional type value 27), even for integral values.
boolean	A CBOR simple value (major type 7, subtype 24) with a simple value of 21 (true) or 20 (false).
null	A CBOR simple value (major type 7, subtype 24) with a simple value of 22 (null).

Indefinite-length items are not allowed and MUST be made a CBOR definite length.

All CBOR tags MUST be retained regardless of whether they are optional.

All four Canonical CBOR rules listed in [RFC8949] MUST be applied to all relevant data types.

All entries of a DID document MUST be included in the root CBOR map (major type 5).

When serializing a DID document to its CBOR representation, a conforming producer MUST specify a media type of application/did+cbor to downstream applications such as described in § 7.1.2 DID Resolution Metadata.

6.4.2 CBOR Consumption
All data structures expressed by a CBOR representation MUST be deserialized into the data model according to the following consumption rules:

CBOR Representation Type	Data Type
CBOR map (major type 5)	An map, where each data item of the CBOR map is added as an entry to the map with the entry key being the data item name and the value converted based on the CBOR type and, if available, entry definition, as defined here; as no order can be enforced for general CBOR maps, no insertion order is guaranteed.
CBOR array (major type 4), where the data model entry value is a list or unknown	A list, where each value of the CBOR array is added to the list in order, converted based on the CBOR type of the array value, as defined in this table.
CBOR array (major type 4), where the data model entry value is an ordered set	An ordered set, where each value of the CBOR array is added to the ordered set in order, converted based on the CBOR type of the array value as defined in this table.
CBOR string (major type 3) where the data model entry value is a datetime	A datetime.
CBOR string (major type 3), where the data model entry value type is string or unknown	A string.
CBOR integer (major type 0 or 1), choosing the shortest byte representation	An integer.
CBOR floating-point number (major type 7)	A double.
CBOR simple value (major type 7, subtype 24) with a simple value of 21 (True) or 20 (False)	A boolean.
CBOR simple value (major type 7, subtype 24) with a simple value of 22 (Null)	A null value.

CBOR indefinite-length items are not allowed and MUST produce an error.

A duplicate key in the same CBOR map MUST produce an error.

All CBOR tags MUST be retained for CBOR production regardless of whether they are optional.

If media type information is available to a conforming consumer and the media type value is application/did+cbor, then the data structure being consumed is a DID document, and the root element MUST be a CBOR map (major type 5) where all members of the object are entries of the DID document.

A conforming consumer for a CBOR representation that is consuming a DID document with a root element that is not a CBOR map (major type 5) MUST report an error.

Clean up normative statements test `it()` blocks

We need to audit the entire test suite against the specification. In order to do so, we need to know (easily) which section certain tests come from. The easiest way to do this would be to state the exact section name right before the normative language in each it() block in the test:

For example, it's easy to know where this normative language came from:

6.3.1 JSON-LD Production - The DID document and any DID document data structures expressed by the data model MUST be serialized to the JSON-LD representation according to the JSON representation production rules as defined in § 6.2 JSON.

To contrast, it's difficult to know where the following normative language came from:

This input is REQUIRED, but the structure MAY be empty.
A conformant DID URL as a single string.

Can each of you assigned to this issue please go through your tests again and make sure you're specifying which section the normative statement you're testing is coming from?

Anyone not doing this for their tests will be volunteered to painstakingly go through the entire specification to match up unmarked tests against normative statements in the specification. I'm only half-joking. :P

Proposal to create a suite per major section of the spec

This will reduce conflicts and improve codeownership, it will also enable better testing in isolation, which should make it easier to work on tests.

Need tests for Core Properties

5.1.1 DID Subject
The value of id MUST be a string that conforms to the rules in § 3.1 DID Syntax and MUST exist in the root map of the data model for the DID document.

5.1.2 DID Controller
The controller property is OPTIONAL. If present, the value MUST be a string or an ordered set of strings that conform to the rules in § 3.1 DID Syntax.

5.1.3 Also Known As
The alsoKnownAs property is OPTIONAL. If present, the value MUST be an ordered set where each item in the set is a URI conforming to [RFC3986].

5.2 Verification Methods
The verificationMethod property is OPTIONAL. If present, the value MUST be an ordered set of verification methods, where each verification method is expressed using a map.

The verification method map MUST include the id, type, controller, and specific verification material properties that are determined by the value of type and are defined in § 5.2.1 Verification Material.

The value of the id property for a verification method MUST be a string that conforms to the rules in Section § 3.2 DID URL Syntax.

The value of the type property MUST be a string that references exactly one verification method type.

The value of the controller property MUST be a string that conforms to the rules in § 3.1 DID Syntax.

The publicKeyBase58 property is OPTIONAL. This feature is non-normative. If present, the value MUST be a string representation of a [BASE58] encoded public key.

The publicKeyJwk property is OPTIONAL. If present, the value MUST be a map representing a JSON Web Key that conforms to [RFC7517].

The map MUST NOT contain "d", or any other members of the private information class as described in Registration Template.

5.2.1 Verification Material
A verification method MUST NOT contain multiple verification material properties for the same material. For example, expressing key material in a verification method using both publicKeyJwk and publicKeyBase58 at the same time is prohibited.

5.3.1 Authentication
The authentication property is OPTIONAL. If present, the associated value MUST be an ordered set of one or more verification methods.

5.3.2 Assertion
The assertionMethod property is OPTIONAL. If present, the associated value MUST be an ordered set of one or more verification methods.

5.3.3 Key Agreement
The keyAgreement property is OPTIONAL. If present, the associated value MUST be an ordered set of one or more verification methods.

5.3.4 Capability Invocation
The capabilityInvocation property is OPTIONAL. If present, the associated value MUST be an ordered set of one or more verification methods.

5.3.5 Capability Delegation
The capabilityDelegation property is OPTIONAL. If present, the associated value MUST be an ordered set of one or more verification methods.

5.4 Services
The service property is OPTIONAL. If present, the associated value MUST be an ordered set of services, where each service is described by a map.

Each service map MUST contain id, type, and serviceEndpoint properties.

The value of the id property MUST be a URI conforming to [RFC3986].

A conforming producer MUST NOT produce multiple service entries with the same id.

A conforming consumer MUST produce an error if it detects multiple service entries with the same id.

The value of the type property MUST be a string or an ordered set of strings.

The value of the serviceEndpoint property MUST be a string, a map, or an ordered set composed of one or more strings and/or maps.

All [serviceEndpoint] string values MUST be valid URIs conforming to [RFC3986] and normalized according to the Normalization and Comparison rules in RFC3986 and to any normalization rules in its applicable URI scheme specification.

Normative statements checklist: 6. Representations

6.1 Production and Consumption

6.2.1 [JSON] Production

6.2.2 [JSON] Consumption

6.3.1 [JSON-LD] Production

The DID document, DID document data structures, and representation-specific entries map MUST be serialized to the JSON-LD representation according to the JSON representation production rules as defined in § 6.2 JSON.
In addition to using the JSON representation production rules, JSON-LD production MUST include the representation-specific @context entry.
The serialized value of @context MUST be the JSON String https://www.w3.org/ns/did/v1, or a JSON Array where the first item is the JSON String https://www.w3.org/ns/did/v1 and the subsequent items are serialized according to the JSON representation production rules.
all JSON-LD Contexts and their terms SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
A conforming producer that generates a JSON-LD representation SHOULD NOT produce a DID document that contains terms not defined via the @context as conforming consumers are expected to remove unknown terms.
When serializing a JSON-LD representation of a DID document, a conforming producer MUST specify a media type of application/did+ld+json to downstream applications such as described in § 7.1.2 DID Resolution Metadata.

6.3.2 [JSON-LD] Consumption

The DID document and any DID document data structures expressed by a JSON-LD representation MUST be deserialized into the data model according to the JSON representation consumption rules as defined in § 6.2 JSON.
Conforming consumers that process a JSON-LD representation SHOULD drop all terms from a DID document that are not defined via the @context.

6.4.1 [CBOR] Production

6.4.2 [CBOR] Consumption

Report generation is brittle and needs some development work

We have need for additional developer resources to help finish the automatic generation of test implementation report.

Much work has been done:
https://github.com/w3c/did-test-suite/blob/main/packages/did-core-test-server/package.json#L11https://github.com/w3c/did-test-suite/tree/main/packages/did-core-test-server#usagehttps://github.com/w3c/did-test-suite/blob/main/packages/did-core-test-server/generate-latest-respec-data.js
https://github.com/w3c/did-test-suite/blob/main/index.html#L686

Please comment here if you may be able to help, and reach out to @OR13 for more detail.

Should implementation files specify `representation` or `didDocumentStream`?

As noted in #55 (review), there is disagreement on whether implementation files should specify representation or didDocumentStream. This issue is here to have that discussion and hopefully come to a conclusion. For now, the test suite is using representation.

need clear instructions for implementers

Implementers need a clear set of instructions that includes:

a pointer to the tests themselves and general information about what is tested
guidance on how to test and submit an implementation report
How to view the summary of submitted reports

I think number three is probably covered, but I'm not finding the first two.

Better re-use test code between sections

After merging #86, all the statements in DID Resolution and DID URL Dereferencing will have tests, but there is still some duplication.

E.g. a test from the DID Resolution section like the one in #86 (review) should re-use test code from the Metadata Structure section (see https://github.com/w3c/did-test-suite/blob/main/packages/did-core-test-server/suites/did-core-properties/did-metadata-structure.js).

Need tests for Conforming Consumer

6.1 Production and Consumption - A conforming consumer MUST take a representation and Media Type string as input into the consumption process. A conforming consumer MAY accept additional options as input into the consumption process.

6.1 Production and Consumption - A conforming consumer MUST determine the representation of a DID document using the Media Type input string.

6.1 Production and Consumption - A conforming consumer MUST detect any representation-specific entry across all known representations and place the entry into a representation-specific entries map which is returned after the consumption process completes. A list of all known representation-specific entries is available in the DID Specification Registries [DID-SPEC-REGISTRIES].

6.1 Production and Consumption - A conforming consumer MUST add all non-representation-specific entries that do not have explicit processing rules for the representation being consumed to the DID document data model using only the representation's data type processing rules and return the DID document data model after the consumption process completes.

6.1 Production and Consumption - A conforming consumer MUST produce errors when consuming non-conforming DIDs or DID documents.

Test suite URL is not correct

Should be https://w3c.github.io/did-test-suite/
Currently points to use cases.

Normative statements checklist: Section 3. Identifiers

3.1 DID Syntax

All DIDs MUST conform to the DID Syntax ABNF Rules.

3.2 DID URL Syntax

All DID URLs MUST conform to the DID URL Syntax ABNF Rules.

3.2.1 DID Parameters

service: If present, the associated value MUST be an ASCII string.
relativeRef: If present, the associated value MUST be an ASCII string and MUST use percent-encoding for certain characters as specified in RFC3986 Section 2.1.
versionId: If present, the associated value MUST be an ASCII string.
versionTime: If present, the associated value MUST be an ASCII string which is a valid XML datetime value, as defined in section 3.3.7 of W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes [XMLSCHEMA11-2].
...This datetime value MUST be normalized to UTC 00:00:00 and without sub-second decimal precision.
hl: If present, the associated value MUST be an ASCII string.
For maximum interoperability, it is RECOMMENDED that DID parameters use the DID Specification Registries mechanism [DID-SPEC-REGISTRIES]

3.2.2 Relative DID URLs

When resolving a relative DID URL reference, the algorithm specified in RFC3986 Section 5: Reference Resolution MUST be used.

Transmute is testing the entire specification

See https://transmute-industries.github.io/did-core/

Need tests for JSON Production and Consumption

6.2.1 JSON Production
The DID document and any DID document data structures expressed by the data model MUST be serialized to the JSON representation according to the following production rules:

Data Type	JSON Representation Type
map	A JSON Object, where each entry is serialized as a member of the JSON Object with the entry key as a JSON String member name and the entry value according to its type, as defined in this table.
list	A JSON Array, where each element of the list is serialized, in order, as a value of the array according to its type, as defined in this table.
ordered set	A JSON Array, where each element of the set is added, in order, as a value of the array according to its type, as defined in this table.
datetime	A JSON String serialized as an XML Datetime normalized to UTC 00:00:00 and without sub-second decimal precision. For example: `2020-12-20T19:17:47Z`.
string	A JSON String.
integer	A JSON Number without a decimal or fractional component.
double	A JSON Number with a decimal and fractional component.
boolean	A JSON Boolean.
null	A JSON null literal.

All entries of a DID document MUST be included in the root JSON Object.

When serializing a DID document, a conforming producer MUST specify a media type of application/did+json to downstream applications such as described in § 7.1.2 DID Resolution Metadata.

6.2.2 JSON Consumption
The DID document and any DID document data structures expressed by a JSON representation MUST be deserialized into the data model according to the following consumption rules:

JSON Representation Type	Data Type
JSON Object	An map, where each member of the JSON Object is added as an entry to the map. Each entry key is set as the JSON Object member name. Each entry value is set by converting the JSON Object member value according to the JSON representation type as defined in this table. Since order is not specified by JSON Objects, no insertion order is guaranteed.
JSON Array where the data model entry value is a list or unknown	A list, where each value of the JSON Array is added to the list in order, converted based on the JSON representation type of the array value, as defined in this table.
JSON Array where the data model entry value is an ordered set	An ordered set, where each value of the JSON Array is added to the ordered set in order, converted based on the JSON representation type of the array value, as defined in this table.
JSON String where data model entry value is a datetime	A datetime.
JSON String, where the data model entry value type is string or unknown	A string.
JSON Number without a decimal or fractional component	An integer.
JSON Number with a decimal and fractional component, or when entry value is a double regardless of inclusion of fractional component	A double.
JSON Boolean	A boolean.
JSON null literal	A null value.

If media type information is available to a conforming consumer and the media type value is application/did+json, then the data structure being consumed is a DID document, and the root element MUST be a JSON Object where all members of the object are entries of the DID document.

A conforming consumer for a JSON representation that is consuming a DID document with a root element that is not a JSON Object MUST report an error.

Need tests for DID Parameters

3.2.1 DID Parameters

Parameter Name	Description
`service`	If present, the associated value MUST be an ASCII string.
`relativeRef`	If present, the associated value MUST be an ASCII string and MUST use percent-encoding for certain characters as specified in RFC3986 Section 2.1.
`versionId`	If present, the associated value MUST be an ASCII string.
`versionTime`	If present, the associated value MUST be an ASCII string which is a valid XML datetime value, as defined in section 3.3.7 of W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes [XMLSCHEMA11-2]. This datetime value MUST be normalized to UTC 00:00:00 and without sub-second decimal precision. For example: `2020-12-20T19:17:47Z`.
`hl`	If present, the associated value MUST be an ASCII string.

Should implementation files contain `representationSpecificEntries` alongside each representation?

Should representationSpecificEntries and properties be stored with each DID entry in implementation files, or should they be split -- with properties being at a higher level, and representationSpecificEntries going beside each representation.

@OR13 suggested that we should store representationSpecificEntries alongside properties here: #55 (comment)

@peacekeeper suggested that they belong with each representation (which is what was decided on the 2021-04-13 telecon): #55 (comment)

Tracking the disagreement here.

The outcome of the decision has no bearing on the output from the test suite.

How will be produce visualization of the test results

At minimum we want to see Green checkmarks Red X's for tests in the spec.
List fo implementers for every test.
Goal is in each row to have at least 2 greens.
Having a human readable description of what each test does it a nice to have.
Also good to have a few words on each implementation

State of the test server

We would like to run the testsuite as a hosted service at some point. The docker image is running and according to the docs should be callable with this curl:

curl -s -X POST http://localhost:8080/test-suite-manager/generate-report \
-H "Content-Type: application/json" \
-d @./suites/did-spec/default.json \
| jq '.suitesReportJson[0]'

These default.json files got replaced by .js files with PR #52 and replacing them with e.g. -d @./suites/did-resolution/default.js returns following error:

{"statusCode":400,"code":"FST_ERR_CTP_EMPTY_JSON_BODY","error":"Bad Request","message":"Body cannot be empty when content-type is set to 'application/json'"}

We also tried it with suitesInput.json's content as data

[
  {
    "name": "did-resolution"
  }
]

and could run tests but got the error

{
  "error": "Cannot read property 'ancestorTitles' of undefined"
}

The testserver logs following:

did-core-test-server_1  |   ● Test suite failed to run
did-core-test-server_1  |
did-core-test-server_1  |     TypeError: Cannot read property 'forEach' of undefined
did-core-test-server_1  |
did-core-test-server_1  |        7 | const utils = require('../resolution-utils');
did-core-test-server_1  |        8 |
did-core-test-server_1  |     >  9 | suiteConfig.resolvers.forEach((implementation) => {
did-core-test-server_1  |          |                       ^
did-core-test-server_1  |       10 |   let implementationName = `7.1.x DID Resolution - ${implementation.implementation} - ${implementation.implementer}`;
did-core-test-server_1  |       11 |
did-core-test-server_1  |       12 |   describe(implementationName, () => {
did-core-test-server_1  |
did-core-test-server_1  |       at Object.<anonymous> (packages/did-core-test-server/suites/did-resolution/did-resolution.spec.js:9:23)
did-core-test-server_1  |
did-core-test-server_1  | Test Suites: 1 failed, 1 total
did-core-test-server_1  | Tests:       0 total
did-core-test-server_1  | Snapshots:   0 total
did-core-test-server_1  | Time:        0.317 s
did-core-test-server_1  | Ran all test suites.

Our attempts to add implementations as input failed so far.

Is this endpoint working atm? If so pls give some example inputs.

We need to refactor fixtures for implementers

We need to clearly show the difference between abstract and concrete representations.

there should be did document data models for both abstract and concrete examples

Testability of normative DID Spec Registries statement on metadata

@shigeya wrote:

Let me quote the statement again here:

All metadata property definitions registered in the DID Specification Registries [DID-SPEC-REGISTRIES] MUST define the value type, including any additional formats or restrictions to that value (for example, a string formatted as a date or as a decimal integer).

From the core spec point of view, the above statement is not testable within the core spec. This spec text needs to be placed in DID-SPEC-REGISTRIES, not within core spec. I honestly have no idea whether we can treat this statement as "human testable."

From a different perspective: since the current test implementation runs the whole test against the transcript of interaction with multiple method implementations, it is desirable to test properties defined in DID-SPEC-REGISTRIES. In this case, we need a machine-readable -- at least partial -- DID-SPEC-REGISTRY.

I think there are three options:

Eliminating the above statement
Rewrite the above statement to move the test to DID-SPEC-REGISTRIES
Rewrite the above statement to remove (direct) dependency on DID-SPEC-REGISTRIES.

It is possible to create an optional configuration parameter that may hold machine-readable (possibly partial) DID-SPEC-REGISTRIES, which can be usable for the above option 3.

(Note: due to schedule conflict, I can't attend the special topic call this week)

Test suite failing?

I don't know if it's just a problem I have, but I tried cloning the repo, installing the dependencies, and running the test suite that comes with the repo, and the tests are failing.

The error is the following:

    TypeError: require(...).didCorePropertiesTests is not a function

      10 |     `5.x Core Properties - ${didMethod} - ${implementation} - ${implementer}`;
      11 |   describe(suiteName, () => {
    > 12 |     require('./did-core-properties').didCorePropertiesTests(
         |                                      ^
      13 |       didMethodSuiteConfig);
      14 |   });
      15 |

      at Suite.<anonymous> (suites/did-core-properties/did-core-properties.spec.js:12:38)
      at forEach (suites/did-core-properties/did-core-properties.spec.js:11:3)
          at Array.forEach (<anonymous>)
      at Object.<anonymous> (suites/did-core-properties/did-core-properties.spec.js:7:24)

which happens with each DID method in the Core Properties test suite. I am running npm run test from within did-core-test-server as instructed in the test README.

Is there any solution to this?

Use of better jest matchers

By using matchers provided by packages like jest-extend, we can simplify, like this

expect(utils.isAsciiString(method)).toBe(true);

to:

expect(method).toBeString();

There are a set of jest matchers available for primitives like arrays and maps.

Also, we can implement customized jest matchers (like the one matches with DID or DID URL string pattern) which will greatly simplify the tests with better readability.

Note: Adding the library make the whole test slightly slower (1.5sec to 1.8sec in my environment), but it will be negligible once the whole set of tests are available.

If this way of the use of matchers is acceptable, I can write a PR for that.

Need tests for DID Resolution

7. Resolution
All conformant DID resolvers MUST implement the DID resolution functions for at least one DID method and MUST be able to return a DID document in at least one conformant representation.

7.1 DID Resolution
did - This is the DID to resolve. This input is REQUIRED and the value MUST be a conformant DID as defined in § 3.1 DID Syntax.

resolutionOptions - A metadata structure containing properties defined in § 7.1.1 DID Resolution Options. This input is REQUIRED, but the structure MAY be empty.

didResolutionMetadata - A metadata structure consisting of values relating to the results of the DID resolution process which typically changes between invocations of the resolve and resolveRepresentation functions, as it represents data about the resolution process itself. This structure is REQUIRED, and in the case of an error in the resolution process, this MUST NOT be empty.

If resolveRepresentation was called, this structure MUST contain a contentType property containing the Media Type of the representation found in the didDocumentStream.

If the resolution is not successful, this structure MUST contain an error property describing the error.

didDocument - If the resolution is successful, and if the resolve function was called, this MUST be a DID document abstract data model (a map) as described in § 4. Data Model that is capable of being transformed into a conforming DID Document (representation), using the production rules specified by the representation.

The value of id in the resolved DID document MUST match the DID that was resolved.

If the resolution is unsuccessful, this value MUST be empty.

didDocumentStream - If the resolution is successful, and if the resolveRepresentation function was called, this MUST be a byte stream of the resolved DID document in one of the conformant representations.

If the resolution is unsuccessful, this value MUST be an empty stream.

didDocumentMetadata - If the resolution is successful, this MUST be a metadata structure.

If the resolution is unsuccessful, this output MUST be an empty metadata structure.

7.1.1 DID Resolution Options
accept - The Media Type MUST be expressed as an ASCII string.

This property is OPTIONAL for the resolveRepresentation function and MUST NOT be used with the resolve function.

7.1.2 DID Resolution Metadata
contentType - The Media Type of the returned didDocumentStream. This property is REQUIRED if resolution is successful and if the resolveRepresentation function was called.

This property MUST NOT be present if the resolve function was called.

The value of this property MUST be an ASCII string that is the Media Type of the conformant representations.

The caller of the resolveRepresentation function MUST use this value when determining how to parse and process the didDocumentStream returned by this function into the data model.

error - The error code from the resolution process. This property is REQUIRED when there is an error in the resolution process.

The value of this property MUST be a single keyword ASCII string.

**invalidDid
**The DID supplied to the DID resolution function does not conform to valid syntax. (See § 3.1 DID Syntax.)
**notFound
**The DID resolver was unable to find the DID document resulting from this resolution request.
**representationNotSupported
**This error code is returned if the representation requested via the accept input metadata property is not supported by the DID method and/or DID resolver implementation. \

7.1.3 DID Document Metadata
The possible properties within this structure and their possible values SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES]. This specification defines the following common properties.

7.1.3 DID Document Metadata
created - The value of the property MUST be a string formatted as an XML Datetime normalized to UTC 00:00:00 and without sub-second decimal precision. For example: 2020-12-20T19:17:47Z.

updated - The value of the property MUST follow the same formatting rules as the created property.

deactivated - If a DID has been deactivated, DID document metadata MUST include this property with the boolean value true. If a DID has not been deactivated, this property is OPTIONAL, but if included, MUST have the boolean value false.

nextUpdate - The value of the property MUST follow the same formatting rules as the created property.

versionId - The value of the property MUST be an ASCII string.

nextVersionId - The value of the property MUST be an ASCII string.

The value of equivalentId MUST be a set where each item in the list is a string that conforms to the rules in Section § 3.1 DID Syntax. Each equivalentId DID value MUST be produced by, and a form of, the same DID Method as the id property value.

A conforming DID Method specification MUST guarantee that each equivalentId value is logically equivalent to the id property value.

equivalentId is a much stronger form of equivalence than alsoKnownAs because the equivalence MUST be guaranteed by the governing DID method.

The value of canonicalId MUST be a string that conforms to the rules in Section § 3.1 DID Syntax. A canonicalId value MUST be produced by, and a form of, the same DID Method as the id property value.

A conforming DID Method specification MUST guarantee that the canonicalId value is logically equivalent to the id property value.

Normative statements checklist: 8. Methods

NOTE: none of these are machine testable as they involve looking over DID method spec text, so I don't expect any to be checked off as part of the test suite, but including for completeness.

8.1 Method Syntax

Normative statements checklist: 4. Data Model

4.1 Extensibility

For maximum interoperability, it is RECOMMENDED that extensions use the W3C DID Specification Registries mechanism [DID-SPEC-REGISTRIES].
Representations MAY define other extensibility mechanisms, including ones that do not require the use of the DID Specification Registries. Such extension mechanisms SHOULD support lossless conversion into any other conformant representation.
Extension mechanisms for a representation SHOULD define a mapping of all properties and representation syntax into the data model and its type system.

PROPOSAL: The test suite will be dockerized

This will allow for anyone to easily run the test suite, on their own machines, without installing the developer dependencies required to run the test suite... if they choose to use docker.

Fix did:3 resolver implementation

@oed you added a resolver implementation for did:3 in #57, and it was temporarily disabled in c138027, since a number of tests are failing.

Some things to consider that are currently causing tests to fail on your implementation:

DID Core says in section "7. Resolution": "All conformant DID resolvers MUST implement the DID resolution functions for at least one DID method and MUST be able to return a DID document in at least one conformant representation".

--> If your implementation results only contain the "resolve" function, but not the "resolveRepresentation" function, then this test will fail.

DID Core says in section "7.1.2 DID Resolution Metadata" about "contentType": "This property MUST NOT be present if the resolve function was called."

--> If your implementation results include the "contentType" DID resolution metadata property for the "resolve" function, then this test will fail. This also causes a number of additional test failures.

DID Core says in section "7.1 DID Resolution" about "did": "This input is REQUIRED and the value MUST be a conformant DID as defined in § 3.1 DID Syntax."

--> If your implementation has something other than a plain DID (e.g. a DID URL with parameters) as input to the "resolve" or "resolveRepresentation" function, then this test will fail. In this case, you should probably consider the dereferencer test suite, not the resolver test suite.

Hope this helps. The example implementation reports may also clarify this further. Thanks for your efforts to submit an implementation report! Feel free to reply here if you need more help.

Unclear how to run the tests

There is no clear run command that I could find, how to do you run all the tests?

Need tests for Metadata Structure

7.3 Metadata Structure

The structure used to communicate this metadata MUST be a map of properties.

Each property name MUST be a string.

Each property value MUST be a string, map, list, ordered set, boolean, or null.

The values within any complex data structures such as maps and lists MUST be one of these data types as well.

All metadata property definitions registered in the DID Specification Registries [DID-SPEC-REGISTRIES] MUST define the value type, including any additional formats or restrictions to that value (for example, a string formatted as a date or as a decimal integer).

The entire metadata structure MUST be serializable according to the JSON serialization rules in the [INFRA] specification.

Need test for Conforming Producer

6.1 Production and Consumption - A conforming producer MUST take a DID document data model and a representation-specific entries map as input into the production process. The conforming producer MAY accept additional options as input into the production process.

6.1 Production and Consumption - A conforming producer MUST serialize all entries in the DID document data model, and the representation-specific entries map, that do not have explicit processing rules for the representation being produced using only the representation's data type processing rules and return the serialization after the production process completes.

6.1 Production and Consumption - A conforming producer MUST return the Media Type string associated with the representation after the production process completes.

6.1 Production and Consumption - A conforming producer MUST NOT produce non-conforming DIDs or DID documents.

PROPOSAL: The test suite structured input will be in yaml and structured output will be in json

yaml is easy to read
json is easy to transform for rendering in web pages.

PROPOSAL: the test suite will support strings first and callback second.

Because callback tests are complex to implement, they should be tackled after we have tackled strings.

This means we don't need to worry about "integrations with external systems / blockchains / libraries" at the start... but we can tackle support for them after we have shown the simple strings versions to be providing the kind of coverage we need.

Tests are confusing, consider refactoring

I have improved on the initial approach taken here, in https://github.com/transmute-industries/vc.transmute.world

I am currently hosting a test server, which can be used to run the tests, their interfaces are substantially simpler:

https://github.com/transmute-industries/vc.transmute.world/blob/master/packages/jest-test-server/suites/issuer-api/issuer-api.spec.js

If its desired, I can refactor this repo to support this pattern AND host a version fo the suite that can be used to request tests on demand.

Fix did:ethr resolver implementation

@awoie you added a resolver implementation for did:ethr in #68, and it was temporarily disabled in c138027, since a number of tests are failing.

Some things to consider that are currently causing tests to fail on your implementation:

DID Core says in section "7. Resolution": "All conformant DID resolvers MUST implement the DID resolution functions for at least one DID method and MUST be able to return a DID document in at least one conformant representation".

--> If your implementation results only contain the "resolve" function, but not the "resolveRepresentation" function, then this test will fail.

DID Core says in section "7.1.2 DID Resolution Metadata" about "contentType": "This property MUST NOT be present if the resolve function was called."

--> If your implementation results include the "contentType" DID resolution metadata property for the "resolve" function, then this test will fail. This also causes a number of additional test failures.

DID Core says in section "7.1 DID Resolution" about "did": "This input is REQUIRED and the value MUST be a conformant DID as defined in § 3.1 DID Syntax."

--> If your implementation has something other than a plain DID (e.g. a DID URL with parameters) as input to the "resolve" or "resolveRepresentation" function, then this test will fail. In this case, you should probably consider the dereferencer test suite, not the resolver test suite.

Hope this helps. The example implementation reports may also clarify this further. Thanks for your efforts to submit an implementation report! Feel free to reply here if you need more help.

'jest-did-matcher@^0.0.1' is not in the npm registry

Getting this error when I try to run the tests:

npm ERR! 404 Not Found - GET https://registry.npmjs.org/jest-did-matcher - Not found
npm ERR! 404 
npm ERR! 404  'jest-did-matcher@^0.0.1' is not in the npm registry. 
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 It was specified as a dependency of 'did-core-test-server'

Producer tests should be of the ADM not resolution result

We need the producer tests to be of an ADM representation, and not a resolution result

Normative statements checklist: 7. Resolution

All conformant DID resolvers MUST implement the DID resolution functions for at least one DID method
and MUST be able to return a DID document in at least one conformant representation.

7.1 DID Resolution

7.1.1 DID Resolution Options

accept: The Media Type MUST be expressed as an ASCII string.
...The DID resolver implementation SHOULD use this value to determine the representation contained in the returned didDocumentStream if such a representation is supported and available.
...This property is OPTIONAL for the resolveRepresentation function
...and MUST NOT be used with the resolve function.

7.1.2 DID Resolution Metadata

contentType: This property is REQUIRED if resolution is successful and if the resolveRepresentation function was called.
...This property MUST NOT be present if the resolve function was called.
...The value of this property MUST be an ASCII string that is the Media Type of the conformant representations.
...The caller of the resolveRepresentation function MUST use this value when determining how to parse and process the didDocumentStream returned by this function into the data model.
error: This property is REQUIRED when there is an error in the resolution process.
...The value of this property MUST be a single keyword ASCII string.
...The possible property values of this field SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].

7.1.3 DID Document Metadata

7.2 DID URL Dereferencing

7.2.1 DID URL Dereferencing Options

The possible properties within this structure and their possible values SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
accept: The Media Type MUST be expressed as an ASCII string.
...The DID URL dereferencing implementation SHOULD use this value to determine the contentType of the representation contained in the returned value if such a representation is supported and available.
...The Media Type of the returned contentStream SHOULD be expressed using this property if dereferencing is successful.
...The Media Type value MUST be expressed as an ASCII string.

7.2.2 DID URL Dereferencing Metadata

contentType: The Media Type of the returned contentStream SHOULD be expressed using this property if dereferencing is successful.
...The Media Type value MUST be expressed as an ASCII string.
error: This property is REQUIRED when there is an error in the dereferencing process.
...The value of this property MUST be a single keyword expressed as an ASCII string.
...The possible property values of this field SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].

7.3 Metadata Structure

The structure used to communicate this metadata MUST be a map of properties.
Each property name MUST be a string.
Each property value MUST be a string, map, list, set, boolean, or null.
The values within any complex data structures such as maps and lists MUST be one of these data types as well.
All metadata property definitions registered in the DID Specification Registries [DID-SPEC-REGISTRIES] MUST define the value type, including any additional formats or restrictions to that value (for example, a string formatted as a date or as a decimal integer).
It is RECOMMENDED that property definitions use strings for values.
The entire metadata structure MUST be serializable according to the JSON serialization rules in the [INFRA] specification.

Danube Tech is testing these sections of the spec

Additional sections TBD....

did-syntax https://w3c.github.io/did-core/#did-syntax (draft PR in progress, see #12)
did-url-syntax https://w3c.github.io/did-core/#did-url-syntax

Call for DID Method Implementations

The W3C DID Working Group achieved another milestone last week; all tests for the specification are now complete.

This means that we are ready for DID Method implementers to submit their DID Method implementations for conformance testing against the specification.

The deadline for submitting DID Method implementations is in two weeks (by May 25th, 2021).

The instructions on how to submit a DID Method implementation can be found here:

https://github.com/w3c/did-test-suite/#adding-your-did-implementation

If you are a DID Method implementer, please get your DID Method implementations submitted in the next two weeks.

The following individuals have registered a DID Method over the past several years. The expectation at this point is that you submit an implementation for conformance testing if you are in a position to do so:

@kdenhartog
@atz3n
@jonnycrunch
@oed
@dhh1128
@identify202020
@picardcapt1212
@awoie
@OntologyTech
@chunningham
@DaegiMin
@MajdT51
@mikelodder7
@rmeissner
@troyronda
@Weelink-team
@anzhy-chernyavski
@wyc
@aaitor
@alexozno
@andreataglia
@bcessa
@bocheng0000
@alvin-reyes
@cbruguera
@clehner
@csuwildcat
@davidlj95
@Diiaablo95
@dmitrizagidulin
@dsemenovsky
@egidiocasati
@ilanolkies
@jdv-ibm
@zo-el
@DavidRicardoWilde
@jcnelson
@tralcanx
@pranavkirtani88
@KilianKae
@lucastetreault
@innoprenuer
@wdmmaaland
@maudnals
@MetadiumRelease
@mirkostanic
@mrinalwadhwa
@nklomp
@OmniOneID
@Sateetje

Can we remove the BSD Digital Bazaar License?

jest-did-matcher not published

I'm aware that this is still work in progress but I'm experimenting to integrate this test-suite to a CI process and would like to deploy it as a standalone server.

Running npm install or building the docker image for did-core-test-server (which does the same) fails with 'jest-did-matcher@^0.0.1' is not in the npm registry.

Is there a plan to publish jest-did-matcher to the npm registry anytime soon?

Missing tests in DID Resolution and DID URL Dereferencing

#43 has been merged to address #29 and #30, but there are still a few TODOs in the DID Resolution and DID URL Dereferencing tests.

All conformant DID resolvers MUST implement the DID resolution functions for at least one DID method and MUST be able to return a DID document in at least one conformant representation.
- Test not implemented yet.
If the resolution is successful, and if the resolve function was called, this MUST be a DID document abstract data model (a map) as described in § 4. Data Model that is capable of being transformed into a conforming DID Document (representation), using the production rules specified by the representation.
- Test if the DID document abstract data model is capable of being transformed into a conforming DID Document (representation). This should re-use code from other tests (production).
The caller of the resolveRepresentation function MUST use this value when determining how to parse and process the didDocumentStream returned by this function into the data model.
- This should re-use code from other tests (consumption).
A conforming DID Method specification MUST guarantee that each equivalentId value is logically equivalent to the id property value.
- I don't think this is testable.
equivalentId is a much stronger form of equivalence than alsoKnownAs because the equivalence MUST be guaranteed by the governing DID method.
- I don't think this is testable.
A conforming DID Method specification MUST guarantee that the canonicalId value is logically equivalent to the id property value.
- I don't think this is testable.
To dereference a DID fragment, the complete DID URL including the DID fragment MUST be used.
- Not sure yet how to test this.
If the contentStream is a DID document, this MUST be a didDocumentMetadata structure as described in DID Resolution.
- This should re-use code from other tests (metadata structure).

Validate command cli

While implementation of validate command on the cli application what should the logic be. As in the README file it is not clearly mentioned.

Normative statements checklist: 5. Core properties

5.1 Identifiers

5.1.1 DID Subject

The value of id MUST be a string that conforms to the rules in § 3.1 DID Syntax
and MUST exist in the root map of the data model for the DID document.

5.1.2 DID Controller

The controller property is OPTIONAL.
If present, the value MUST be a string or a set of strings that conform to the rules in § 3.1 DID Syntax.
The corresponding DID document(s) SHOULD contain verification relationships that explicitly permit the use of certain verification methods for specific purposes.
Any verification methods contained in the DID documents for those DIDs SHOULD be accepted as authoritative, such that proofs that satisfy those verification methods are to be considered equivalent to proofs provided by the DID subject.

5.1.3 Also Known As

The alsoKnownAs property is OPTIONAL.
If present, the value MUST be a set where each item in the set is a URI conforming to [RFC3986].

5.2 Verification Methods

The verificationMethod property is OPTIONAL.
If present, the value MUST be a set of verification methods, where each verification method is expressed using a map.
The verification method map MUST include the id, type, controller, and specific verification material properties that are determined by the value of type and are defined in § 5.2.1 Verification Material.
Verification methods SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
The value of the id property for a verification method MUST be a string that conforms to the rules in Section § 3.2 DID URL Syntax.
The value of the type property MUST be a string that references exactly one verification method type.
the verification method type SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].
The value of the controller property MUST be a string that conforms to the rules in § 3.1 DID Syntax.

5.2.1 Verification Material

The publicKeyBase58 property is OPTIONAL.
...If present, the value MUST be a string representation of a [BASE58] encoded public key.
The publicKeyJwk property is OPTIONAL.
...If present, the value MUST be a map representing a JSON Web Key that conforms to [RFC7517].
...The map MUST NOT contain "d", or any other members of the private information class as described in Registration Template.
...It is RECOMMENDED that verification methods that use JWKs [RFC7517] to represent their public keys use the value of kid as their fragment identifier.
...It is RECOMMENDED that JWK kid values are set to the public key fingerprint [RFC7638].
A verification method MUST NOT contain multiple verification material properties for the same material.

5.3 Verification Relationships

any [verification method] properties used SHOULD be registered in the DID Specification Registries [DID-SPEC-REGISTRIES].

5.3.1 Authentication

The authentication property is OPTIONAL.
If present, the associated value MUST be a set of one or more verification methods.

5.3.2 Assertion

The assertionMethod property is OPTIONAL.
If present, the associated value MUST be a set of one or more verification methods. Each verification method MAY be embedded or referenced.

5.3.3 Key Agreement

The keyAgreement property is OPTIONAL.
If present, the associated value MUST be a set of one or more verification methods.

5.3.4 Capability Invocation

The capabilityInvocation property is OPTIONAL.
If present, the associated value MUST be a set of one or more verification methods.

5.3.5 Capability Delegation

The capabilityDelegation property is OPTIONAL.
If present, the associated value MUST be a set of one or more verification methods.

5.4 Services

rename master branch

Summary

We should rename the branch master to main and use that going forward for our work.

From Problematic Terminology in Open-Source on master/slave terminology in software:

Use of this term is problematic. It references slavery to convey meaning about the relationship between two entities.

Removing this terminology from our workflow is a small gesture to include more people from marginalized groups in this project.

(I’m open to names other than main)

Technical Steps

create main branch from master
make main the default GitHub branch
modify github/central to use main for release notes reloading
redirect PRs to main in w3c/did-test-suite
move branch protections from master to main
modify docs to reference main instead of master
delete master branch to avoid confusion?

Feedback?

I request to be made an editor of this repo

Proposal to refactor the input json into js files

so we can build larger configurations from smaller more atomic ones.

Need tests for DID URL Dereferencing

7.2 DID URL Dereferencing
didUrl - A conformant DID URL as a single string. This is the DID URL to dereference. To dereference a DID fragment, the complete DID URL including the DID fragment MUST be used. This input is REQUIRED.

dereferencingOptions - This input is REQUIRED, but the structure MAY be empty.

dereferencingMetadata - This structure is REQUIRED and in the case of an error in the dereferencing process, this MUST NOT be empty.

If the dereferencing is not successful, this structure MUST contain an error property describing the error.

contentStream - If the dereferencing function was called and successful, this MUST contain a resource corresponding to the DID URL.

If the dereferencing is unsuccessful, this value MUST be empty.

contentMetadata - If the dereferencing is successful, this MUST be a metadata structure, but the structure MAY be empty.

If the contentStream is a DID document, this MUST be a didDocumentMetadata structure as described in DID Resolution.

If the dereferencing is unsuccessful, this output MUST be an empty metadata structure.

accept - The Media Type that the caller prefers for contentStream. The Media Type MUST be expressed as an ASCII string.

contentType - The Media Type value MUST be expressed as an ASCII string.

error - This property is REQUIRED when there is an error in the dereferencing process.

The value of this property MUST be a single keyword expressed as an ASCII string. This specification defines the following common error values: invalidDidUrl, notFound**
**

Need tests for Syntax and DID URL Syntax

3.1 DID Syntax
All DIDs MUST conform to the DID Syntax ABNF Rules.

The DID Syntax ABNF Rules
`did = "did:" method-name ":" method-specific-id` `method-name = 1method-char` `method-char = %x61-7A / DIGIT` `method-specific-id = ( idchar ":" ) 1idchar` `idchar = ALPHA / DIGIT / "." / "-" / "_"`

3.2 DID URL Syntax

All DID URLs MUST conform to the DID URL Syntax ABNF Rules.

The DID URL Syntax ABNF Rules
`did-url = did path-abempty [ "?" query ] [ "#" fragment ]`

3.2.2 Relative DID URLs

When resolving a relative DID URL reference, the algorithm specified in RFC3986 Section 5: Reference Resolution MUST be used. The base URI value is the DID that is associated with the DID subject, see § 5.1.1 DID Subject. The scheme is did. The authority is a combination of <method-name>:<method-specific-id>, and the path, query, and fragment values are those defined in § Path, § Query, and § Fragment, respectively.

Need tests for JSON-LD Production and Consumption

6.3.1 JSON-LD Production
The DID document and any DID document data structures expressed by the data model MUST be serialized to the JSON-LD representation according to the JSON representation production rules as defined in § 6.2 JSON.

In addition to using the JSON representation production rules, JSON-LD production MUST include the representation-specific @context entry.

The serialized value of @context MUST be the JSON String https://www.w3.org/ns/did/v1, or a JSON Array where the first item is the JSON String https://www.w3.org/ns/did/v1 and the subsequent items are serialized according to the JSON representation production rules.

When serializing a JSON-LD representation of a DID document, a conforming producer MUST specify a media type of application/did+ld+json to downstream applications such as described in § 7.1.2 DID Resolution Metadata.

6.3.2 JSON-LD Consumption
The DID document and any DID document data structures expressed by a JSON-LD representation MUST be deserialized into the data model according to the JSON representation consumption rules as defined in § 6.2 JSON.

w3c / did-test-suite Goto Github PK

did-test-suite's Introduction

DID Test Suite

Adding Your DID Implementation

Adding and Modifying Tests

Code of Conduct

Getting Started

If tests are failing

DID Working Group Repositories

did-test-suite's People

Contributors

Stargazers

Watchers

Forkers

did-test-suite's Issues

6.1 Production and Consumption

6.2.1 [JSON] Production

6.2.2 [JSON] Consumption

6.3.1 [JSON-LD] Production

6.3.2 [JSON-LD] Consumption

6.4.1 [CBOR] Production

6.4.2 [CBOR] Consumption

3.1 DID Syntax

3.2 DID URL Syntax

3.2.1 DID Parameters

3.2.2 Relative DID URLs

8.1 Method Syntax

4.1 Extensibility

7.1 DID Resolution

7.1.1 DID Resolution Options

7.1.2 DID Resolution Metadata

7.1.3 DID Document Metadata

7.2 DID URL Dereferencing

7.2.1 DID URL Dereferencing Options

7.2.2 DID URL Dereferencing Metadata

7.3 Metadata Structure

5.1 Identifiers

5.1.1 DID Subject

5.1.2 DID Controller

5.1.3 Also Known As

5.2 Verification Methods

5.2.1 Verification Material

5.3 Verification Relationships

5.3.1 Authentication

5.3.2 Assertion

5.3.3 Key Agreement

5.3.4 Capability Invocation

5.3.5 Capability Delegation

5.4 Services

Summary

Technical Steps

Recommend Projects

Recommend Topics

Recommend Org