HTTP Administration API to Manage Jetstream about jetstream

[ripienaar]

It would be easy enough to do. We do think a set of CRDs will work better for k8s though and in the mean while we have terraform support which might also be better for you I bet - there are terraform operators for a GitOps style flow

[ripienaar]

If it was a HTTP API how would you make calls to it?

Reason I ask is, the NATS API is very very similar to a HTTP API and if you're going to be writing some client - you can just access the API over NATS protocol and the outcome would be very similar?

[jon-whit]

@ripienaar whether it's an HTTP API or a NATS API it shouldn't matter. A set of CRDS would be good, if those could be managed programmatically over the k8s API (remotely). The issue I'm trying to solve is dynamic configuration.

Either way though, is there an ETA for when something like this may be available?

[ripienaar]

We have the NATS API now, that's why I am asking what the major advantage of a HTTP wrapper around that would be :)

the jsm.go package wraps the NATS API and the README.md here somewhat covers all the API has. It's a very HTTP like API with JSON in and JSON out, we have json schema documents for all the requests and responses (see schemas in this repo) if you want to do some other language

We do not have a timeline around the CRDs

[jon-whit]

@ripienaar ah, I see! I did not realize that :). Thanks for the info 👍

[ripienaar]

happy to help you out with some examples if needed :)

[jon-whit]

That would be wonderful! If you could shoot some Gist(s) my way that'd be awesome 👍

[ripienaar]

At the basic level we can do:

// connect to nats
nc, _ := nats.Connect(url)

// create a stream if it doesnt exist 
_, err := jsm.LoadOrNewStream("SCOUT_TAGS", jsm.FileStorage(), jsm.MaxMessages(10000), jsm.Subjects("scout.tags.>"), jsm.StreamConnection(jsm.WithConnection(nc)))
if err != nil {
	return fmt.Errorf("could not create SCOUT_TAGS stream: %s", err)
}

This is basically whats the nats cli does using https://github.com/nats-io/jsm.go

On the CLI you can do:

$ nats str create SCOUT_TAGS --storage=file --max-msgs=1000 --subjects="scout.tags.>" --retention=limits --discard=old --max-bytes=-1 --max-age=-1 --max-msg-size=-1

to do the same, lets see what that does under the covers by adding --trace the nats utility shows you what it sends where and what it gets back:

$ nats str create .... --trace
17:03:17 >>> $JS.API.STREAM.CREATE.SCOUT_TAGS
{"name":"SCOUT_TAGS","subjects":["scout.tags.\u003e"],"retention":"limits","max_consumers":-1,"max_msgs":1000,"max_bytes":-1,"max_age":0,"max_msg_size":-1,"storage":"file","discard":
"old","num_replicas":1}

17:03:17 <<< $JS.API.STREAM.CREATE.SCOUT_TAGS
{
  "type": "io.nats.jetstream.api.v1.stream_create_response",
  "config": {
    "name": "SCOUT_TAGS",
    "subjects": [
      "scout.tags.\u003e"
    ],
    "retention": "limits",
    "max_consumers": -1,
    "max_msgs": 1000,
    "max_bytes": -1,
    "discard": "old",
    "max_age": 0,
    "max_msg_size": -1,
    "storage": "file",
    "num_replicas": 1
  },
  "created": "2020-07-02T15:03:17.789612641Z",
  "state": {
    "messages": 0,
    "bytes": 0,
    "first_seq": 0,
    "first_ts": "0001-01-01T00:00:00Z",
    "last_seq": 0,
    "last_ts": "0001-01-01T00:00:00Z",
    "consumer_count": 0
  }
}

In other words this sent a io.nats.jetstream.api.v1.stream_create_request message and got a io.nats.jetstream.api.v1.stream_create_response back

You can read about the API subjects, design etc here https://github.com/nats-io/jetstream#nats-api-reference it mentions constants like api.JSApiStreamList those are the ones from https://github.com/nats-io/jsm.go/tree/master/api

[jon-whit]

Awesome! Thanks for those great resources :). I appreciate it.