Extension Schema: Services about specification

[llamahunter]

In microservice deployments, having some way to track which version of what other microservices are used is critical for stable deployments. Ideally there would be some way to automagically generate that bom-like information (monitoring the service mesh?). But, even if it were manually maintained somewhere, it could help validate that a deployment environment is 'sufficient' to run a new microservice instance.

[stevespringett]

Agreed. So if we keep it lightweight (which I'm in favor of) and drop deployments and add a namespace to the service, we end up with something like this:

<svc:services>
  <svc:service bom-ref="b2a46a4b-8367-4bae-9820-95557cfe03a8">
    <svc:provider>
      <svc:organization>Example Inc</svc:organization>
      <svc:url>http://www.example.com</svc:url>
      <svc:email>[email protected]</svc:email>
    </svc:provider>
    <svc:license>
      <svc:name>Example, Inc enterprise license</svc:name>
      <svc:quota>10,000,000 requests per month</svc:quota>
    </svc:license>
    <svc:endpoints>
        <svc:endpoint>URI or some other location description</svc:endpoint>
    </svc:endpoints>
    <svc:namespace>kube-my-namespace</svc:namespace>
    <svc:authenticated>true/false - Whether or not authentication is required</svc:authenticated>
    <svc:x-trust-boundary>true/false - Whether or not communication with service cross a trust boundary</svc:x-trust-boundary>
    <svc:data>
      <svc:classification flow="inbound/outbound/both">What kind</svc:classification>
      <svc:classification flow="outbound">PII</svc:classification>
      <svc:classification flow="inbound">PHI</svc:classification>
    </svc:data>
  </svc:service>
</svc:services>

Is there anything else in the license section we need? Is quota useful here? Anything missing?

[stevespringett]

The component model uses the concepts of group, name, and version. We should be consistent if we introduce new types of objects.

So instead of namespace, let's have group, name, and version. The group could easily be used to represent a namespace, as both provide high-level categorization. The definition of a component group is:

The grouping name or identifier. This will often be a shortened, single name of the company or project that produced the component, or the source package or domain name. Whitespace and special characters should be avoided. Examples include: apache, org.apache.commons, and apache.org.

So something like this:

<service bom-ref="b2a46a4b-8367-4bae-9820-95557cfe03a8">
    <group>my-namespace</group>
    <name>stock-ticker-service<name>
    ...
</service>

[taleodor]

So instead of namespace, let's have group, name, and version. The group could easily be used to represent a namespace, as both provide high-level categorization.

Yes, sounds very reasonable.

[stevespringett]

I started writing the XSD and associated valid test case. Here's an example of how the service works as well as the ability to specify a dependency on a service for a component.

<?xml version="1.0"?>
<bom serialNumber="urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79" version="1" xmlns="http://cyclonedx.org/schema/bom/1.2">
  <components>
    <component type="library" bom-ref="pkg:maven/com.acme/[email protected]">
      <group>com.acme</group>
      <name>stock-java-client</name>
      <version>1.0.12</version>
      <hashes>
        <hash alg="SHA-1">e6b1000b94e835ffd37f4c6dcbdad43f4b48a02a</hash>
      </hashes>
      <licenses>
        <license>
          <id>Apache-2.0</id>
        </license>
      </licenses>
      <purl>pkg:maven/com.acme/[email protected]</purl>
    </component>
  </components>
  <services>
    <service bom-ref="b2a46a4b-8367-4bae-9820-95557cfe03a8">
      <provider>
        <name>Partner Org</name>
        <url>https://partner.org</url>
      </provider>
      <group>org.partner</group>
      <name>Stock ticker service</name>
      <description>Provides real-time stock information</description>
      <endpoints>
        <endpoint>https://partner.org/api/v1/lookup</endpoint>
        <endpoint>https://partner.org/api/v1/stock</endpoint>
      </endpoints>
      <authenticated>true</authenticated>
      <x-trust-boundary>true</x-trust-boundary>
      <data>
        <classification flow="bi-directional">pubic</classification>
      </data>
      <licenses>
        <license>
          <name>Partner license</name>
        </license>
      </licenses>
    </service>
  </services>
  <dependencies>
    <dependency ref="pkg:maven/com.acme/[email protected]">
      <dependency ref="b2a46a4b-8367-4bae-9820-95557cfe03a8"/>
    </dependency>
  </dependencies>
</bom>

I think this will work well. Please review and let me know if there's anything missing, incorrect, etc. I'll like start on the unit tests and JSON schema implementation on Monday.

[stevespringett]

Proposed namespace is:
http://cyclonedx.org/schema/ext/services/1.0

[stevespringett]

Thoughts on fields that would be beneficial to capture:

<svc:services>
  <svc:service bom-ref="b2a46a4b-8367-4bae-9820-95557cfe03a8">
    <svc:provider>
      <svc:organization>Example Inc</svc:organization>
      <svc:url>http://www.example.com</svc:url>
      <svc:email>[email protected]</svc:email>
    </svc:provider>
    <svc:license>
      <svc:name>Example, Inc enterprise license</svc:name>
      <svc:quota>10,000,000 requests per month</svc:quota>
    </svc:license>
    <svc:endpoint>URI or some other location description</svc:endpoint>
    <svc:authenticated>true/false - Whether or not authentication is required</svc:authenticated>
    <svc:x-trust-boundary>true/false - Whether or not communication with service cross a trust boundary</svc:x-trust-boundary>
    <svc:data>
      <svc:classification flow="inbound/outbound/both">What kind</svc:classification>
      <svc:classification flow="outbound">PII</svc:classification>
      <svc:classification flow="inbound">PHI</svc:classification>
    </svc:data>
  </svc:service>
</svc:services>

[stevespringett]

Note: Like BOM components themselves, the service definition should be limited to factual/verifiable data. Weaknesses, threats, countermeasures, and vulnerabilities should not be part of the service definition.

[stevespringett]

Would something like this work?

<svc:deployments>
  <svc:deployment>
    <svc:version>1.0.0</svc:version>
    <svc:published>2020-01-01T13:15:30Z</svc:published>
    <svc:buildid>53341</svc:buildid>
  </svc:deployment>
  <svc:deployment>
    <svc:version>1.0.1.canary0</svc:version>
    <svc:published>2020-01-02T10:06:10Z</svc:published>
    <svc:buildid>53349</svc:buildid>
  </svc:deployment>  
</svc:deployments>

The deployments node would appear inside the service node. As such, this would describe two deployments of a single service, all of which are optional.

Of course, there's nothing preventing you from creating more than one service of the same type, and creating a single deployment for each. In that regard it would specify a unique endpoint for each version of a given service. So it should be flexible that way.

[coderpatros]

Maybe this needs to be broken up a bit. A service can mean very different things.

We have some things that depend on another service via a message topic and subscription, REST API, incoming webhook, database, etc.

At the same time I think the example is generic enough. But the trick would be standardizing the svc:endpoint element. But then maybe that could be a bit like a name spaced package url?

[stevespringett]

@coderpatros I think pub/sub would be covered by data flow inbound and outbound. For services that depend on other services, you could use the existing dependency-graph extension to represent services dependent on other services.

One thing that might be interesting is to support services within other services, similar to what can be done with components. This assembly, subassembly heirarchy would allow you to specify an outbound data flow to a service, for example, and then within that service (possibly an API gateway), you could then represent the individual services behind the gateway that you may not directly interact with.

[stevespringett]

I'm going to reach out to the NTIA framing group to see if they've thought through these scenarios or not and if they have, try to being some of their knowledge and findings into the spec.

[stevespringett]

Since work on v1.2 is being done, I'm going to attempt to get this into the core rather than a schema extension.

[stevespringett]

Based on feedback, this is what an example service could look like.

<svc:services>
  <svc:service bom-ref="b2a46a4b-8367-4bae-9820-95557cfe03a8">
    <svc:provider>
      <svc:organization>Example Inc</svc:organization>
      <svc:url>http://www.example.com</svc:url>
      <svc:email>[email protected]</svc:email>
    </svc:provider>
    <svc:license>
      <svc:name>Example, Inc enterprise license</svc:name>
      <svc:quota>10,000,000 requests per month</svc:quota>
    </svc:license>
    <svc:deployments>
      <svc:deployment>
        <svc:version>1.0.0</svc:version>
        <svc:published>2020-01-01T13:15:30Z</svc:published>
        <svc:buildid>53341</svc:buildid>
      </svc:deployment>
      <svc:deployment>
        <svc:version>1.0.1.canary0</svc:version>
        <svc:published>2020-01-02T10:06:10Z</svc:published>
        <svc:buildid>53349</svc:buildid>
      </svc:deployment>  
    </svc:deployments>
    <svc:endpoints>
        <svc:endpoint>URI or some other location description</svc:endpoint>
    </svc:endpoints>
    <svc:authenticated>true/false - Whether or not authentication is required</svc:authenticated>
    <svc:x-trust-boundary>true/false - Whether or not communication with service cross a trust boundary</svc:x-trust-boundary>
    <svc:data>
      <svc:classification flow="inbound/outbound/both">What kind</svc:classification>
      <svc:classification flow="outbound">PII</svc:classification>
      <svc:classification flow="inbound">PHI</svc:classification>
    </svc:data>
  </svc:service>
</svc:services>

Is there anything missing or incorrect in this proposal?

For services that depend on other services, the existing dependency-graph functionality will help. In this example, the service defined above has a dependency on another service in the BOM.

<dg:dependencies>
    <dg:dependency ref="b2a46a4b-8367-4bae-9820-95557cfe03a8">
        <dg:dependency ref="c2ad6f24-b3ac-428f-9de5-74d8ed1f96c4"/>
    </dg:dependency>
</dg:dependencies>

[stevespringett]

From @coderpatros

But the trick would be standardizing the svc:endpoint element. But then maybe that could be a bit like a name spaced package url?

Agreed. I don't have an answer to that. If you look at MUD for example, it represents external services like this:

{
  "ietf-mud:mud": {
    "mud-version": 1,
    "mud-url": "https://examp.e.com/justa-fridge.json",
    "mud-signature": "https://examp.e.com/justa-fridge.p7s",
    "last-update": "2020-04-27T17:45:18+00:00",
    "cache-validity": 48,
    "is-supported": true,
    "systeminfo": "awesome sauce",
    "mfg-name": "justa",
    "documentation": "http://docs.justa",
    "model-name": "justa-fridge",
    "from-device-policy": {
      "access-lists": {
        "access-list": [
          {
            "name": "mud-79180-v4fr"
          },
          {
            "name": "mud-79180-v6fr"
          }
        ]
      }
    },
    "to-device-policy": {
      "access-lists": {
        "access-list": [
          {
            "name": "mud-79180-v4to"
          },
          {
            "name": "mud-79180-v6to"
          }
        ]
      }
    }
  },
  "ietf-access-control-list:acls": {
    "acl": [
      {
        "name": "mud-79180-v4to",
        "type": "ipv4-acl-type",
        "aces": {
          "ace": [
            {
              "name": "cl0-todev",
              "matches": {
                "ipv4": {
                  "ietf-acldns:src-dnsname": "spymaster.example.com"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "cl1-todev",
              "matches": {
                "ipv4": {
                  "ietf-acldns:src-dnsname": "spymaster-central.example.com",
                  "protocol": 6
                },
                "tcp": {
                  "ietf-mud:direction-initiated": "from-device"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "loc0-todev",
              "matches": {
                "ietf-mud:mud": {
                  "local-networks": [
                    null
                  ]
                },
                "ipv4": {
                  "protocol": 17
                },
                "udp": {
                  "source-port": {
                    "operator": "eq",
                    "port": 9000
                  },
                  "destination-port": {
                    "operator": "eq",
                    "port": 90
                  }
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            }
          ]
        }
      },
      {
        "name": "mud-79180-v4fr",
        "type": "ipv4-acl-type",
        "aces": {
          "ace": [
            {
              "name": "cl0-frdev",
              "matches": {
                "ipv4": {
                  "ietf-acldns:dst-dnsname": "spymaster.example.com"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "cl1-frdev",
              "matches": {
                "ipv4": {
                  "ietf-acldns:dst-dnsname": "spymaster-central.example.com",
                  "protocol": 6
                },
                "tcp": {
                  "ietf-mud:direction-initiated": "from-device"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "loc0-frdev",
              "matches": {
                "ietf-mud:mud": {
                  "local-networks": [
                    null
                  ]
                },
                "ipv4": {
                  "protocol": 17
                },
                "udp": {
                  "destination-port": {
                    "operator": "eq",
                    "port": 9000
                  },
                  "source-port": {
                    "operator": "eq",
                    "port": 90
                  }
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            }
          ]
        }
      },
      {
        "name": "mud-79180-v6to",
        "type": "ipv6-acl-type",
        "aces": {
          "ace": [
            {
              "name": "cl0-todev",
              "matches": {
                "ipv6": {
                  "ietf-acldns:src-dnsname": "spymaster.example.com"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "cl1-todev",
              "matches": {
                "ipv6": {
                  "ietf-acldns:src-dnsname": "spymaster-central.example.com",
                  "protocol": 6
                },
                "tcp": {
                  "ietf-mud:direction-initiated": "from-device"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "loc0-todev",
              "matches": {
                "ietf-mud:mud": {
                  "local-networks": [
                    null
                  ]
                },
                "ipv6": {
                  "protocol": 17
                },
                "udp": {
                  "source-port": {
                    "operator": "eq",
                    "port": 9000
                  },
                  "destination-port": {
                    "operator": "eq",
                    "port": 90
                  }
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            }
          ]
        }
      },
      {
        "name": "mud-79180-v6fr",
        "type": "ipv6-acl-type",
        "aces": {
          "ace": [
            {
              "name": "cl0-frdev",
              "matches": {
                "ipv6": {
                  "ietf-acldns:dst-dnsname": "spymaster.example.com"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "cl1-frdev",
              "matches": {
                "ipv6": {
                  "ietf-acldns:dst-dnsname": "spymaster-central.example.com",
                  "protocol": 6
                },
                "tcp": {
                  "ietf-mud:direction-initiated": "from-device"
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            },
            {
              "name": "loc0-frdev",
              "matches": {
                "ietf-mud:mud": {
                  "local-networks": [
                    null
                  ]
                },
                "ipv6": {
                  "protocol": 17
                },
                "udp": {
                  "destination-port": {
                    "operator": "eq",
                    "port": 9000
                  },
                  "source-port": {
                    "operator": "eq",
                    "port": 90
                  }
                }
              },
              "actions": {
                "forwarding": "accept"
              }
            }
          ]
        }
      }
    ]
  }
}

The above was created with mudmaker and is just an example. MUD supports TCP, UDP, and ANY. in the case of devices that speak MQTT which can run over TCP, but doesn't require it. It can also run over other protocols.

So coming up with an endpoint that meets all requirements may be challenging. If we use URI, I think it will support most cases. In the case of MQTT for example, the URI would be something liek this:

mqtt[s]://[username][:password]@host.domain[:port]

Thoughts? Should endpoint be a URI?

[taleodor]

Thoughts? Should endpoint be a URI?

Yes, that makes a lot of sense.

Other than that, would it be possible to add namespace property for services and deployments? This would be beneficial to organize complex deployments by namespace (one example would be k8s namespaces).

[stevespringett]

so namespace would be a simple string field? Yeah, that can be added to the service and deployment nodes.

[taleodor]

so namespace would be a simple string field?

Yes, simple string would be fine, thanks.

[coderpatros]

Should endpoint be a URI?

Yes, I think so.

Do you think environment is something that should be supported?

[stevespringett]

Do you think environment is something that should be supported?

I think that starts to fall into the configuration management space which I want to avoid. I'm a little on the fence about supporting deployments themselves. And environments on top of that would be a bit much IMO.

is there something specific you had in mind?

[taleodor]

Do you think environment is something that should be supported?

I think if we have endpoint as URI, it should be enough to identify instance / environment.

[coderpatros]

I think that starts to fall into the configuration management space which I want to avoid. I'm a little on the fence about supporting deployments themselves. And environments on top of that would be a bit much IMO.

I agree. I thought having deployments was a bit odd too. For a first cut it's probably better to keep it as lightweight as possible until real use cases are uncovered.

[coderpatros]

I'm not sure about quota either. namespace seems a bit specific to a particular ecosystem too. Everything else looks pretty good though.

[taleodor]

namespace seems a bit specific to a particular ecosystem too.

We use the concept of namespace in multiple cases, not only in k8s. We also use tags to denote services. Tags are clearly more heavyweight, so not suggesting that for 1.2 schema.

However, if there is no concept of namespacing and no concept of tagging, it would be pretty hard to apply schema to many real-word scenarios. I.e., for cases where a single instance holds multiple services belonging to different products. If we can't distinguish such products by namespaces or other means - all services would appear like they serve single purpose, which is highly inaccurate.

[coderpatros]

I guess I normally come across the namespace being part of the name, i.e. domain names, package names, etc. I use the same approach for my services too.

I don't have a strongly held opinion on namespace being part of the spec or not.

[stevespringett]

I'm also not a fan of quota because once you go there, then it's easy to start doing other things like sla.

Should the service object support licenses?

If so, there's already a license definition that can accept SPDX license IDs, SPDX expressions, or simple license names.

[coderpatros]

I think being consistent where appropriate is a good stance @stevespringett

[coderpatros]

I think it looks good.