metanorma / metanorma-registry

The Metanorma Registry software (lite) for self-hosting

Metanorma: the standard for standards

Metanorma is dedicated to harmonizing standard documents produced by different standard-setting bodies in a manner that maintains correct semantics while allowing each standard publisher to define appropriate semantic extensions.

Simply put, it allows standards bodies or any other organization to create their own standard or specification document in a best practices manner.

Metanorma is composed of a number of specifications and software implementations. The Metanorma document model is based on the SecureDoc document model.

For more on Metanorma and who uses it, refer to https://www.metanorma.org

Installation on supported platforms

See https://www.metanorma.org/install/

Installing individual components

The Metanorma workflow can be utilized via the metanorma-cli Ruby gem.

gem install metanorma-cli

Usage

Refer to Metanorma man page and Metanorma usage

Threaded execution

Metanorma has threaded execution, to generate output documents from the same Presentation XML input more quickly. Similar to relaton, the METANORMA_PARALLEL environment variable can be used to override the default number of parallel fetches used.

Origin of name

Meta- is a prefix of Greek origin ("μετα") for “with” “after”. In English, it has ended up meaning "about (its own category)"; e.g. meta-discussion (a discussion about discussion). (For the roundabout way it ended up with that meaning, see https://en.wikipedia.org/wiki/Meta#Etymology.)

Norma is Latin for “rule” and “standard”; hence English norm, but also German Norm "standard".

The Metanorma project is for setting a standard for standard documents created by standards-setting organizations (which is a meta thing to do); hence this name.

Metanorma seeks to embrace all standards documents standards, but not possess any: it can give rise to many "standard" standards, but not limit the extension of any of those standards.

The motto of the project is Aequitate verum, "Truth through equity". Dealing with all standards fairly (aequitate), we seek not an abstract virtue (veritas), but a practical reality on the ground (verum), that can be used by stakeholders of multiple standards.

metanorma-registry's Issues

Graphql: List metanorma documents

GraphQL: filter for bibdata status

In https://github.com/metanorma/mn-registry-sample-data there are multiple stages of documents. They are all located within bibdata.

I want to be able to filter to "return all documents with status = 'published'", e.g.

{
  documents (type: "csd-standard", bibdata.status: "published") {
    id
  }
}

GraphQL endpoint to support filtering by standard type

In https://github.com/metanorma/mn-registry-sample-data there are multiple types of documents.

I want to be able to filter to "return all type = 'csd-standard' documents", e.g.

{
  documents (type: "csd-standard") {
    id
  }
}

Update code to create a Git-backed metanorma registry

Architecture

• Lambda portion for fetch, upload
• Git portion for storage

Need to handle:

• Metanorma format version migration (Metanorma XMLs must have a way of transitioning to a newer version)

Basically we want to deploy one instance for standards.calconnect.org. This means we need to provide all functionality it currently provides.

@skalee this is the end goal, let's discuss the steps in between...

Support csa-caiq-3.0.1.xml

The csa-caiq-3.0.1.xml file is a special one that contains "requirements". The GraphQL interface should support fetching requirements directly.

GraphQL: `preface` should be fully parsed

Today's preface is like this:

From this query

{
  documents {
    preface
  }
}

Response:

{
  "data": {
    "documents": [
      {
        "preface": "{\"foreword\"=>{\"obligation\"=>\"informative\", \"title\"=>\"Foreword\", \"p\"=>[\"The Calendaring and Scheduling Consortium (“CalConnect”) is a global non-profit   \\norganization with the aim to facilitate interoperability of technologies across   \\nuser-centric systems and applications.\", \"CalConnect works closely with liaison partners including international   \\norganizations such as ISO, OASIS and M3AAWG.\", \"The procedures used to develop this document and those intended for its further   \\nmaintenance are described in the CalConnect Directives.\", \"In particular the different approval criteria needed for the different types of   \\nISO documents should be noted. This document was drafted in accordance with the   \\neditorial rules of the CalConnect Directives.\", \"Attention is drawn to the possibility that some of the elements of this   \\ndocument may be the subject of patent rights. CalConnect shall not be held responsible   \\nfor identifying any or all such patent rights. Details of any patent rights   \\nidentified during the development of the document will be in the Introduction   \\nand/or on the CalConnect list of patent declarations received (see   \\nwww.calconnect.com/patents).\", \"Any trade name used in this document is information given for the convenience   \\nof users and does not constitute an endorsement.\", \"This document was prepared by Technical Committee .\"]}}"
...

But the "preface" should be full parsed, allowing queries like:

{
  documents {
    preface {
      foreword {
        title
      }
    }
  }
}

GraphQL: support "Return all terms and definitions" of this document

I need a method to say: "provide all terms and definitions" given in this document.

These are all contained in a <clause id="terms">.

The response should be something like:

{
  "data": {
    "terms":  [
      { "term": "abc", "definition": "xyz" },
      { "term": "def", "definition": "ijk" },
      ...
    ]
  }
}

Text response issues

In the text returned many places we see empty (). The reason is this:

data/cc-10001.xml:  <definition><p id="_5d4fdc0e-144e-40f5-a00a-46fc043675f1">publishable <em>document</em> (<xref target="term-document"/>) that relates to standardization</p></definition>

This line gets rendered like this:

publishable   () that relates to standardization

Clearly:

1. <em> should not disappear
2. <xref> shall somehow remain, or be rendered as a link with a clause number. (ping @opoudjis thoughts?)

The real question is: in what textual syntax does text get returned? In BasicDoc (https://github.com/CalConnect/csd-lightweight-doc)? In XML? In HTML? Or maybe this is selectable e.g. p(format: html)?

Confirm requirements

From @abunashir

Metanorma Registry

• Create metanorma resistry to manage metanorma documents
• This service will manage the lifecycle of a doucumnt
• Submit / Create a new documnet - How exactly they submit a document?
• Modify a document / what will be modified here, upload a new version?
• Delete a document - do we need to keep track of histroy or something?
• A static part that will list the documents, what about the design?
• The dynamic part, who can upload a documents a owner or organization?
• Current site is based on jekyll, do we still keep it that way?
• What exactly do we show to the user from this site and how?
• Issue: metanorma/metanorma-cli#26
• If I could handle this, what are the timeframe, urgency?
• Pricing / what might be good based on the details scope
• Restructure the calconnect site, is there any relation between this and new repository apart from haaving similiar functionality?
• Move all the pages, sass and static file to where exactly?

GraphQL: supports fetching "example", "note" elements

"example", "note" elements exist within clauses. We need to ensure that the GraphQL interface supports "get me all the examples within this document".

GraphQL: `bibdata` must be fully parsed

Today's response is like this:

{
  "data": {
    "documents": [
      {
        "bibdata": {
          "contributor": "[{\"role\"=>nil, \"organization\"=>{\"name\"=>\"CalConnect\"}}, {\"role\"=>nil, \"person\"=>{\"name\"=>{\"completename\"=>\"Thomas Schäfer, 1&1 Mail&Media Development and Technology GmbH\"}}}, {\"role\"=>nil, \"person\"=>{\"name\"=>{\"completename\"=>\"Jesse Thompson, University of Wisconsin-Madison\"}}}, {\"role\"=>nil, \"organization\"=>{\"name\"=>\"CalConnect\"}}]",
          "copyright": "{\"from\"=>\"2019\", \"owner\"=>{\"organization\"=>{\"name\"=>\"CalConnect\"}}}",
          "date": "{\"type\"=>\"published\", \"on\"=>\"2019-01-18\"}",
          "docidentifier": "CC/R 18003:2019",
          "docnumber": "18003",
          "edition": "1",
          "editorialgroup": "{\"technical_committee\"=>\"CALSPAM\"}",
          "language": "en",
          "script": "Latn",
          "status": "published",
          "title": "Calendar operator practices — Guidelines to protect against calendar abuse",
          "version": "{\"revision_date\"=>\"2019-01-18\"}"
        }
      },
...

Notice that contributor, copyright, date, editorial group, version are all unparsed JSON.