dret / webconcepts

Web Concepts: Concepts that matter for the the Web.

Web Concepts

This is the repository for the Web Concepts site published at webconcepts.info.

Setup

There probably should be a bit of documentation about how all of this works... A very high-level overview is this:

• New categories (concepts or specification organizations/series) get added via XML configuration files (concepts.xml and specs.xml).

• New specification data gets added to the specification source directory /src/specs as individual XML files. These specification files contain information about the concept values that are defined by the specifications.

You can find additional information on how to fork the repo and how to add or remove Web Concepts. After forking, adding, or removing, you also need information on building Web Concepts.

License

Please respect the license.

webconcepts's Issues

HTTP Prefer

with http://tools.ietf.org/html/draft-snell-http-prefer, it might be interesting to look at preferences as another mechanism how a service can design/expose its interface. it basically follows the model of other mechanisms that have some general framework, and then a registry to support expressions in that framework. there is no clear "extension model" for preferences; unknown ones simply are ignored by a server.

Renaming "HTTP Header Field" to "HTTP Field"

wondering what to do about this. renaming the concept could lead to a lot of breakage out there, but then again the reality is that with HTTP/2 and HTTP/3, the "header field" concept now had been replaced by "field" because there can be trailers as well.

remove XML identifiers

it seems that the @xml:id identifiers add little or no value, and make management and the language quite a bit more complex. it should be possible to simply remove them, and rely on the identified concept itself (the media type, the link relation, and so forth). this needs a little testing, but ideally, all @xml:id should be removed from the XSD and all instanced (and then of course the XSLT needs a little love to reflect this change).

add organization URI to JSON

the current specification JSON data structure specs.json does not include the link to an organization's home page. this could be added easily by using the data that's available in the XML configuration file.

Media Type Parameters

Currently, there is no support for capturing media type parameters. It would be relatively easy to add this to the Sedola schema, if required and requested.

Add more "Web surface" concepts

right now the only concepts supported are media types, HTTP header field types, and link relation types. There definitely should be HTTP status codes as well. This issue serves to capture ideas about additional concepts that "belong into the web surface" and should be documented in a similar way as media types, HTTP header field types, and link relation types are documented right now.

add mouseover link text to concept values

on concept pages, in the right-hand column of the table of values, the specification link does have the specification abstract as mouseover link text. it would be useful to have the concept value documentation text as mouseover link text in the left-hand column of the concept page tables.

Terms/data definitions as web concepts?

Would something like
http://www.w3.org/TR/prov-dm/
be appropriate for web concepts?

Handle ABNF

the current sedola schema allows ABNF (only in HTTP header fields so far), but there is no support in the current MD transformation. for properly formatting ABNF, whitespace management might become important. ABNF can be multi-line. maybe a good starting point would be to break into lines, then to remove any leading and trailing whitespace, and then use the ABNF.
35fa0f2 just added one example of a multi-line ABNF.

support for well-known URI

maybe add support for http://www.iana.org/assignments/well-known-uris/well-known-uris.xml would be useful, allowing services to say how they support this mechanism. this could be another top-level mechanism, just like media types, HTTP headers, and all the other current top-level mechanisms in sedola.

add RFC 3688 concepts

https://tools.ietf.org/html/rfc3688 defines registries for DTD public identifiers, XML namespaces, XML schemas, and RDF schemas. all of these are intended as registries for identifiers, and (in case of URI identifiers) intentionally are designed to use URNs, so that it is clear that they are intended to be identifiers.
all of these concepts probably would make sense to be included in web concepts. the registries of DTD public identifiers and RDF schemas are effectively unused (one respectively no entry), but the other ones have seen more uptake:

• XML namespaces: 207 entries
• XML schemas: 124 entries

vertical bar (|) in text breaks MD tables

when using vertical bar characters (|) in tables, they are copied unescaped to the generated MD files. this breaks tables which are using vertical bars to separate table columns. vertical bars should be escaped before copying text into MD tables.

add usage information for individual values

there should be a way how data can be added that talks about how individual concept values are used in concrete services: it would be something along the lines of being able to say, "API X uses HTTP request method GET and HTTP status codes 200 and 404."
This has been discussed with @mitraman and @RubenVerborgh and it would be great to get some feedback or even use cases from them. for now, the simple assumption is that this data would be a useful way to link back from web concepts to concrete APIs, and thus could turn web concepts into a useful discovery tool of an existing API landscape, if the usage data for that landscape is fed into web concepts.
for the "original" web concepts repo and site, this probably does not even matter. but any fork might benefit from this if the fork could be easily turned into an overview page for an existing landscape of APIs (driven by the usage data for these specific APIs).

Missing RFC 3986 - Uniform Resource Identifier

I was very surprised that the web concepts pages do not include any URI specification. I do see that there isn't a good category for that as a concept.

Maybe there should be a new category created such as Basic concepts or similar?

Add Content Encodings

spawned from #13.

how to handle concept values with multiple definitions

as mentioned in #25 (comment), currently it is possible for a concept value to have more than one definition (http://webconcepts.info/concepts/http-header/Authorization is an example with 3 definitions). in order to better find and analyze these cases, it would be useful to have a page that simply lists all of these values. this page could remain unlinked and thus hidden, but would be useful to have.

Add HTML5

1382085 removed an "HTML5 description" that was just a copy of html4. A proper description of HTML5 is required.

Switch site and URIs to HTTPS?

while i am working on making the JSON more stable and adding documentation to the site, i am wondering if switching everything to HTTPS would be an appropriate move. it seems that HTTP URIs (and sites supporting plain HTTP) are becoming a bit old-fashioned. when switching the site to HTTPS, then probably all URIs should be switched as well, so that they still work as dereferencable URIs (see #24 for a related discussion). any opinions/preference about the site and the URIs?

Add HTTP Transfer Codings

http://www.iana.org/assignments/http-parameters/http-parameters.xhtml#transfer-coding should be added as well. this is related to #13.

Add HTTP Status Codes

spawned from #13.

Downlink header isn't linked correctly

Not sure where it's documented now though...

JSON Format for each concept value

We can get all concepts in JSON with this URL http://webconcepts.info/concepts/concepts.json, but it would be a great addition to be able to access each concept value individually as a json file.
For example: http://webconcepts.info/concepts/http-status-code/409.json

improve title generation for MD lists

156b0aa shows how the current way of generating titles from spec URIs does not work reliably enough. some improvement is needed, either better rules, or the ability to explicitly define a title.

spec.json should identify concepts and concept values by URI

currently spec.json "identifies" concepts and values by string value only:

{"ietf-httpbis-encryption-encoding": {
  "id": "http://webconcepts.info/specs/IETF/I-D/ietf-httpbis-encryption-encoding",
  "title": "Encrypted Content-Encoding for HTTP",
  "name": "Internet Draft ietf-httpbis-encryption-encoding",
  "URI": "urn:ietf:id:ietf-httpbis-encryption-encoding",
  "URL": "http://tools.ietf.org/html/draft-ietf-httpbis-encryption-encoding",
  "abstract": "This memo introduces a content coding for HTTP that allows message payloads to be encrypted.",
  "concepts": [
    {"http-content-coding": "aesgcm"},
    {"http-header": "Crypto-Key"},
    {"http-header": "Encryption"}]}}

this seems wrong because both concept and concept value should be identified by their ID (their URIs) as documented at http://webconcepts.info/JSON. i am inclined to just fix this, but wanted to see if anybody is using these values, and what they think about switching from plain strings to proper URIs.

explicitly reuse concepts when using include

just including another sedola document should not automatically use all of the concepts in that document. instead, the including document should have to explicitly list all of those concepts that it intends to reuse. this allows to also not use concepts when they are not required and should not be considered as being used.
this could be augmented by a "include-all-concepts" attribute on that by default is "no", but could be set to "yes", and then means that all concepts are intended to be used.

add OAuth concepts and values

add OAuth concepts/values as listed here: http://www.iana.org/assignments/oauth-parameters/oauth-parameters.xhtml

• OAuth Access Token Types
• OAuth Authorization Endpoint Response Types
• OAuth Dynamic Client Registration Metadata
• OAuth Extensions Error Registry
• OAuth Parameters
• OAuth Token Endpoint Authentication Methods
• OAuth Token Introspection Response
• OAuth Token Type Hints
• OAuth URI
• PKCE Code Challenge Methods

New I-D: draft-ietf-httpbis-digest-headers

I wish

To reference https://github.com/httpwg/http-extensions/blob/master/draft-ietf-httpbis-digest-headers.md in your list of web-concepts.

This spec

Will obsolete RFC 3230, which defines Digest and Want-Digest.

escape web concept values

individual web concept values should be escaped in the web concept URIs. this would avoid problems with values that have slashes (such as media types) or hash signs (such as some profile URIs). while not so hard to do in principle, this causes problems with jekyll and github pages. jekyll/jekyll#6052 has been raised to address these problems, but this might take a while to fix because there seem to be various problems with jekyll itself, and with the web server and its ability to properly respond to requests for URIs containing escaped characters.

add "Token Binding Protocol" registries/concepts (RFC 8471)

three new registries/concepts have been introduced by https://tools.ietf.org/html/rfc8471 :

• Token Binding Key Parameters
• Token Binding Types
• Token Binding Extensions

all of these could be added as new web concept types.

Add license

I can't find the license for this repository. Can you add one?

Add HTTP Request Methods

spawned from #13.

add JSON-LD profile URIs

3b3d3f7 adds JSON-LD profile URIs but only as comments because currently these cannot be processed. this might have to wait until #40 has been resolved or at least addressed in some way that makes it possible to use web concept values that contain reserved URI characters.

Include spec id in concepts.json

I'd like to include the spec signature with each concept in concepts.json (see pmhsfelix/slack-web-concept#6) so that it's not necessary to fetch and process specs.json every time. Ofc it would be possible to cache it but its structure isn't too friendly for finding the right spec anyway.

I would add a simple key/value to each spec in the format "{organization} {series} {spec}" (not sure I'm getting the naming right, so don't bash me). For example basic authentication details would become

{
  "description": "[...]",
  "documentation": "http://tools.ietf.org/html/rfc7617#section-2",
  "specification": "http://webconcepts.info/specs/IETF/RFC/7617",
  "specification-signature": "IETF RFC 7617"
}

I don't particularly like the specification-signature. Any better name?

What do you think?

Add URI Schemes

adding URI schemes would make a lot of sense. another addition to the general issue #13.

Add HTTP Preferences

http://www.iana.org/assignments/http-parameters/http-parameters.xhtml#preferences should be added as well. this is related to #13.

ABNF dependencies

the simple ABNF (just for HTTP headers for now) markup added to sedola.xsd (fea7cac) does not allow to capture dependencies. but many of the specs reuse existing production rules from the HTTP spec. it would be possible to (a) require that all ABNF much be self-contained (resulting in a lot of duplication), or (b) add some mechanism that allows to capture dependencies so that it is represented where to find referenced rules. for now, the plan is to capture ABNF information, and then we'll see if somebody is asking for (a) or (b).

Add Well Known URI: change-password

As described in https://wicg.github.io/change-password-url/

checking for concept value correctness

a1f412d was a case where a concept value (a media type in this case) contained spurious whitespace. ideally, this would be caught while processing, either by trimming whitespace automatically, or probably even better by checking concept values against some form of value pattern (maybe a regex?). that way, it would be easier to find and fix this kind of problem.

description for "http-opportunistic" well-known URI

i wasn't able to glean a good one or two sentences description about the http-opportunistic well-known URI from RFC 8164. @mnot, if you think you can come up with one to appear on http://webconcepts.info/concepts/well-known-uri/http-opportunistic, that would be great. thanks!
the source file is https://github.com/dret/webconcepts/blob/gh-pages/src/specs/rfc8164.xml but i can of course take care of this if there is some text to be added.

identifiers for web concepts

what kind of identifier should web concepts have? there are probably three general design choices:

• a simple non-URI identifier such as http-method:GET
• a dereferencable URI such as http://webconcepts.info/concepts/http-method/GET
• a non-dereferencable URI such as urn:webconcept:http-method:GET

all these methods have pros and cons, i'd be interested to hear opinions.

If/how to add "non-standard" concepts

would it make sense to start adding "non-standard concepts" such as the X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset HTTP header fields defined by https://developer.github.com/v3/#pagination? it might be useful to have a place where these are made available, but maybe the main site is not the best place? would it make sense to add a complete "parallel universe" of proprietary/non-standard concepts?

structure media type documentation

ideally, media type documentation should be more structured than just one link. it is important to document aspects of extensibility, the processing model, and possible interactions with media type documents. all of these aspects should be covered by a media type definition and its documentation, and thus sedola probably should be able to link to those individually.

Add JWT as access token type

In the page Oauth Access Token Types Bearer and MAC are listed, but I think JWT is missing. RFC 7519 specifies it.

This section registers the value "token-type:jwt" in the IANA "OAuth
   URI" registry established by "An IETF URN Sub-Namespace for OAuth"
   [RFC6755], which can be used to indicate that the content is a JWT.

   o  URN: urn:ietf:params:oauth:token-type:jwt
   o  Common Name: JSON Web Token (JWT) Token Type
   o  Change Controller: IESG
   o  Specification Document(s): RFC 7519

Defining API Standards with Web Concepts

I wanted to share a little idea I had a while back that feels inline with the spirit of this project. It came up as we were defining API standards at work. My observations were:

1. We rarely used an entire standard. For instance, for conditional requests, we said that Etag was a MUST and last modified was a MAY.
2. The standards were made to be broader than what was used in APIs. Back to my previous example, the API standards internally said MUST for Etag and MAY for last modified, but an API might only use the Etag and not list the last modified so client developers didn't have to support it.

We were providing company standards while also letting APIs share which of those standards they were using. After I sat on in many of the discussions, I had the thought of some format for describing company standards.

https://gist.github.com/smizell/d3419fcb358d057807cc18d8898d94b2

To sum up then, I have found that it's not enough to say what standards or concepts are used when defining some internal standards. People need to define a subset of what they allow and a subset of that of what a particular API used. Maybe then there is an opportunity to be able to narrow down concepts and standards in this way.

Add /.well-known

http://www.iana.org/assignments/well-known-uris/well-known-uris.xhtml is another registry that is a part of today's web toolbox. per #13, this probably should be added to the list of web concepts. the relevant spec is https://tools.ietf.org/html/rfc5785.

Add Auth scheme concept?

I was looking for a list of Authorization schemes. There is the IANA registry: http://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml

Is there a good reason to duplicate that list on webconcepts? If there are other well-known schemes around, maybe so?

Support for updated specification

RFC 4918 registers all the headers of RFC 2518 (which it updates), except for one (Status-URI, just added here: a58dff3). currently, there is no way to properly represent this dependency, so that one spec can say that it updates another, and then overwrites whatever also has been specified in the updated spec.
This might get as complicated as distinguishing between "obsoleting" and "updating", so fixing this issue probably is not going to be trivial.

Add HTTP Cache Directives

spawned from #13.

make specification JSON more self-contained

the current way how specifications list concepts is by URIs (as fixed in #36). this is correct, but means that there no easy way to get concept information from the JSON.

{ "concepts": [
  { "http://webconcepts.info/concepts/urn-namespace": "http://webconcepts.info/concepts/urn-namespace/ietf" }]}

the proposal is to make this a bit more self-contained by adding the concept name and value, resulting in something like this:

{ "concepts": [
    { "id": "http://webconcepts.info/concepts/urn-namespace",
      "name": "URN Namespace",
      "value-id": "http://webconcepts.info/concepts/urn-namespace/ietf",
      "value": "ietf" }]}

this would be a breaking change. @kinlane @mogsie @tpluscode @mitraman @BigBlueHat @pmhsfelix, anybody using this and disliking this proposal?

Add HTTP Forwarded Parameters

http://www.iana.org/assignments/http-parameters/http-parameters.xhtml#forwarded should be added as well. this is related to #13.