Integrate documentation at https://github.com/nats-io/jetstream here. Enhance and add as necessary.

See #178 (The Jetstream readme is being translated into documentation)

This includes:

• General Description and Applicable Use Cases
• Concepts
• Getting Started Guide - Simple setup based on our recommendations.
• Server documentation
• Basic parameters/configuration options
• Security
• Developing clients with Jetstream
• Monitoring
• Administration and usage (CLI)
• Securing Jetstream

Section could be under: https://github.com/nats-io/nats.docs/tree/master/jetstream.

Of course, add additional documentation as necessary.

/CC @gcolliso

Monitoring endpoints have additional account parameters and the documentation needs to be updated.

e.g.
https://github.com/nats-io/nats-server/blob/34caf5aaa56daf63ef496bd6c5e29a7086e74d27/server/monitor.go#L88

The format is not correct in this page
https://nats-io.github.io/nsc/nsc_edit_user.html

https://docs.nats.io/nats-tools/nsc/nsc#creating-an-operator claims nsc add operator -n Test output should look like

Generated operator key - private key stored “~/.nkeys/Test/Test.nk”
Success! - added operator "Test"

but it looked like

$ nsc add operator -n Test
[ OK ] generated and stored operator key "OAXE7CKC44VD2ENMV5G5M7FH4OWSEVWF2NC3AYWLIGHKYZ6KHL46RGWF"
[ OK ] added operator "Test"

.

Also, ~/.nkeys/Test/Test.nk doesn't exist on my machine:

$ tree ~/.nkeys
/Users/jrantil/.nkeys
├── creds
└── keys
    └── O
        └── AX
            └── OAXE7CKC44VD2ENMV5G5M7FH4OWSEVWF2NC3AYWLIGHKYZ6KHL46RGWF.nk

4 directories, 1 file

In documentation page https://docs.nats.io/developing-with-nats/tutorials/reqreply code uses subject foo while in the description it's referred as help.please. Another mismatch is with "request payload" and "some message".

The organization should provide a short overview of the different ways we can create groups of servers
Tutorials for each of the sections should be on their own page as we do for the configuration pages.

Followed the docs

It gives an error

no matches for kind "Deployment" in version "extensions/v1beta1"

Same doesn't work with apps/v1, which gives an error saying selector missing.

Please update

The organization should provide a short overview of the different ways we can create groups of servers
Tutorials for each of the sections should be on their own page as we do for the configuration pages.

In https://docs.nats.io/nats-server/nats_admin/sys_accounts/sys_accounts after:

The only thing we don't have handy is the public key for the system account. We can get it easy enough:
(easily enough?)
there is this command-line invocation:

nsc list accounts -W

That flag is not supported by that sub-command. It looks like nsc describe has a -W for --long-ids but this example should be re-examined by someone familiar with how things should be.

Describe major features and rationale for version 2.2.

e.g. a 2.2 version of https://docs.nats.io/whats_new_20

I'm not sure where to put this right now so am opening an issue. We should provide guidance on how to track down missing messages / mismatched subscriptions.

Check for Errors

First search the server logs for authorization errors.

Test the subscription

http://demo.nats.io:8222/subsz?subs=1&test=yoursubject

If there's a listener on your subject, you'll get a list returned like:

{
  "server_id": "NCPRDHZCF4V54OQWYICZIFNCXECVZGS7WITJ7UVIFDT6IJZRSSMOMCPV",
  "now": "2021-01-25T11:29:03.914444926-05:00",
  "num_subscriptions": 39,
  "num_cache": 7,
  "num_inserts": 3643482,
  "num_removes": 3643443,
  "num_matches": 77212,
  "cache_hit_rate": 0.45173240075124665,
  "max_fanout": 1,
  "avg_fanout": 0.5714285714285714,
  "total": 1,
  "offset": 0,
  "limit": 1024,
  "subscriptions_list": [
    {
      "account": "$G",
      "subject": "yoursubject",
      "sid": "0",
      "msgs": 0,
      "cid": 68352
    }
  ]
}

If there are no active subscriptions on the supplied subject then no subscriptions_list field will be returned.

Inspect subscriptions

via connections endpoint: http://demo.nats.io:8222/connz?subs=1
via subscriptions endpoint: http://demo.nats.io:8222/subsz?subs=1

Check recently closed connections

You may also want to check closed connections to see if there was a disconnect at the time you are missing messages.
http://demo.nats.io:8222/connz?state=closed

If you named the connection in the application, you should be able to easily track down that application's connection closed events and the reason.

Watch published messages

nats-sub your.subject

Watch all published messages and look for a subject mismatch

nats-sub ">"

Debug/Verbose Mode

Run the server with -DV flags and watch protocol messages (or reload with debug:true verbose:true). This is intrusive and impacts performance so should be used sparingly in production.

like chinese, french etc.

Section could be under: https://github.com/nats-io/nats.docs/tree/master/nats-server as "Native MQTT Support". If this documentation should be in a different place please comment here.

In a PR, please provide enough documentation to get started:

• General Description and Applicable Use Cases (@ColinSullivan1 )
• Getting Started Guide - Simple setup based on our recommendations.
• Basic parameters/configuration options
• Basic security

Of course, add additional documentation as necessary.

/CC @gcolliso

Add documentation around subject mapping.

See the TestGlobalAccountRouteMappingsConfiguration test for reference.

I am new to nats-streaming and docker, I don't know if there is any misunderstanding.

Maybe the command in sql_store.md should be update or declare the environment.($VAR looks like a UNIX environment, but the docker run command like under Windows, or just a mistake? cuz the things after --name should be a name, and nats-streaming-local does not exist)

... docker run -d --name nats-streaming -p 4222:4222 -p 8222:32768 nats-streaming-local -SDV --store sql ...

It would be better if there is a line break between getting bridge ip and docker run.

And using this command did not see the management port exposed, I need to use the -m parameter to explicitly declare the port. Is this related to nats streaming itself?
Here is my command (WSL2 based docker,HOST_PG_PORT is another variable used to modify the mapped port of postgres):
docker run -d --name nats-streaming-local -p 4222:4222 -p 8222:8222 nats-streaming -m 8222 -SDV --store sql --sql_driver postgres --sql_source="user=postgres password=postgres host=$DOCKER_BRIDGE_IP port=$HOST_PG_PORT sslmode=disable"

log like this before add -m:

[1] 2020/11/19 08:49:55.238433 [INF] STREAM: Starting nats-streaming-server[test-cluster] version 0.19.0
[1] 2020/11/19 08:49:55.238512 [INF] STREAM: ServerID: cwjWoieQmrQGs9Ze5XCLDA
[1] 2020/11/19 08:49:55.238517 [INF] STREAM: Go version: go1.14.10
[1] 2020/11/19 08:49:55.238520 [INF] STREAM: Git commit: [c658000]
[1] 2020/11/19 08:49:55.286688 [INF] Starting nats-server version 2.1.9
[1] 2020/11/19 08:49:55.286724 [INF] Git commit [7c76626]
[1] 2020/11/19 08:49:55.287131 [INF] Listening for client connections on 0.0.0.0:4222
[1] 2020/11/19 08:49:55.287164 [INF] Server id is NDONJSDF6PGAVG2BS6GP3MQBTIUSIJOVEZ5PISWU6TDJSR7C7A6G2VOY
[1] 2020/11/19 08:49:55.287167 [INF] Server is ready
[1] 2020/11/19 08:49:55.313929 [INF] STREAM: Recovering the state...
[1] 2020/11/19 08:49:55.316508 [INF] STREAM: Recovered 0 channel(s)
[1] 2020/11/19 08:49:55.566947 [DBG] STREAM: Did not detect another server instance
[1] 2020/11/19 08:49:55.567145 [DBG] STREAM: Discover subject: _STAN.discover.test-cluster
[1] 2020/11/19 08:49:55.567209 [DBG] STREAM: Publish subject: _STAN.pub.gJzQuNy1b5QJX66sUUoBEB.>
[1] 2020/11/19 08:49:55.567212 [DBG] STREAM: Subscribe subject: _STAN.sub.gJzQuNy1b5QJX66sUUoBEB
[1] 2020/11/19 08:49:55.567215 [DBG] STREAM: Subscription Close subject: _STAN.subclose.gJzQuNy1b5QJX66sUUoBEB
[1] 2020/11/19 08:49:55.567217 [DBG] STREAM: Unsubscribe subject: _STAN.unsub.gJzQuNy1b5QJX66sUUoBEB
[1] 2020/11/19 08:49:55.567220 [DBG] STREAM: Close subject: _STAN.close.gJzQuNy1b5QJX66sUUoBEB
[1] 2020/11/19 08:49:55.567553 [INF] STREAM: Message store is SQL
[1] 2020/11/19 08:49:55.567607 [INF] STREAM: ---------- Store Limits ----------
[1] 2020/11/19 08:49:55.567610 [INF] STREAM: Channels: 100 *
[1] 2020/11/19 08:49:55.567612 [INF] STREAM: --------- Channels Limits --------
[1] 2020/11/19 08:49:55.567614 [INF] STREAM: Subscriptions: 1000 *
[1] 2020/11/19 08:49:55.567615 [INF] STREAM: Messages : 1000000 *
[1] 2020/11/19 08:49:55.567617 [INF] STREAM: Bytes : 976.56 MB *
[1] 2020/11/19 08:49:55.567619 [INF] STREAM: Age : unlimited *
[1] 2020/11/19 08:49:55.567621 [INF] STREAM: Inactivity : unlimited *
[1] 2020/11/19 08:49:55.567623 [INF] STREAM: ----------------------------------
[1] 2020/11/19 08:49:55.567625 [INF] STREAM: Streaming Server is ready

after add -m :

[1] 2020/11/19 09:08:19.937464 [INF] Starting http monitor on 0.0.0.0:8222
[1] 2020/11/19 09:08:19.937596 [INF] Listening for client connections on 0.0.0.0:4222

At present the persistence docs persistence/sql_store.md#L5 link to:
https://godoc.org/github.com/lib/pq#hdr-Connection_String_Parameters

On viewing that page there is a suggestion to view the more recent docs:

Any objections to a PR updating the link to the latest version of Go Docs: https://pkg.go.dev/github.com/lib/pq#hdr-Connection_String_Parameters ?

I read https://docs.nats.io/nats-tools/nk#generating-nkeys-and-configuring-the-server and it says

The first output line starts with the letter S for Seed.

From a mnemonics standpoint, it might be easier to remember "S" for "secret" as that's what it really is and it's crucial, from a security standpoint, that users nk don't send around the wrong token. How about changing to that?

Looks like https://docs.nats.io has dns problem

-nsc flag has been removed from nats-io/nats-account-server#62 for some time. The documentation still uses -nsc flag. Here are some links that I noticed. There might be more:

• https://docs.nats.io/nats-tools/nsc/nsc
• https://docs.nats.io/nats-tools/nas/nas_conf

Hi, you have a very little describing of how to use curl to add/update accounts with api here https://docs.nats.io/nats-tools/nas/dir_store#using-curl-to-add-or-update-accounts

Have any detailed explaining of how to manage account server (add accounts, add users or change users permissions in account, import or export topics and etc..) with api (without manual using of cli) ?

Exists in FAQ:

• Is there a message size limitation in NATS?
• Does NATS impose any limits on the # of subjects?

Missing from FAQ:

• How many clients can connect simultaneously?

I tried to find a response to this question and couldn't find anywhere. Is there a maximum number of concurrent clients? What does it depend on? Is the clients list partitioned, e.g. does it grow when a NATS server is added?

You may want to add this to your FAQs.

things are working as said in docs, except the monitoring specs

command line param work but not configuration file param not work or it is not correct, i tried http_port and http both

https://github.com/nats-io/nats.docs/blob/master/nats-streaming-concepts/monitoring/README.md

Can't open https://docs.nats.io - DNS_PROBE_FINISHED_NXDOMAIN

Frequently question on how to use: nats-io/stan.go#335

Security Documentation

This could be under: https://github.com/nats-io/nats.docs/tree/master/nats-server/configuration/securing_nats.

If this documentation should be in a different place please comment here.

In a PR, please provide documentation to address:

• CIDR Block Checking
• Time Restrictions
• More JWT Usage Examples
• Additional Metadata (e.g descriptions)
• JetStream Account Options

For these sections, please make a note that these features apply to 2.2.

Under NATS Tools:

• JWT V2 (nsc)
• Default User Permissions (nsc)
• Auto Generate System Account (nsc)

Miscellaneous Server Configuration

• Subject Mapping and Bridging

Of course, add additional documentation as necessary.

/CC @gcolliso

This document https://docs.nats.io/nats-tools/nas/mem_resolver describes a memory resolver. It is part of the nats account server documentation, but really should be part of nats-server.

I keep sending this email and answering similar questions. We need to add something like this to doc:

NATS and CI/CD

Running NATS in your CI/CD environment is fairly straightforward. The NATS server is so lightweight it's really easy to spin up and test with, for example in .NET we can run about 500 tests in under 7 minutes in azure pipelines with almost every test launching a server instance.

Installing the NATS Server

To setup CI with the latest NATS server you have a few options - you can build or run our nightly docker image.

To build: https://github.com/nats-io/nats.java/blob/master/install_deps.sh
Docker: $ docker run -p 4222:4222 synadia/nats-server:nightly

For past releases you can unzip a tar file from github and use an official release, but you won't have edge features.

Or just use github actions if you are in a github environment:
https://github.com/nats-io/jetstream-gh-action

NATS Server test helpers and Usage

Test helpers vary client to client. Here are some examples.

Java

Here is the class you can copy to run the NATS server from java:
https://github.com/nats-io/nats.java/blob/master/src/test/java/io/nats/client/NatsTestServer.java

Example tests:
https://github.com/nats-io/nats.java/blob/master/src/test/java/io/nats/client/JetstreamTests.java

Node.js

https://github.com/nats-io/nats.js/blob/master/test/support/nats_server_control.js
and usage:
https://github.com/nats-io/nats.js/blob/master/test/basics.js#L32

Others

(add .NET, deno, C, python, etc).

Check out each NATS client repository's CI/CD workflow for hints as to run the NATS server CI/CD in your environment.

In PRs, please provide documentation to address:

• No Responders (https://github.com/nats-io/nats.docs/tree/master/developing-with-nats) (@wallyqs)
• Message Headers (under https://github.com/nats-io/nats.docs/tree/master/developing-with-nats) (@wallyqs)
• Lame Duck Notifications (https://github.com/nats-io/nats.docs/tree/master/nats-server/nats_admin) (@ColinSullivan1)

For these sections, please make a note that these features apply to 2.2.

Locations are suggestions, if you feel this belongs elsewhere no worries.

Hey guys,
I'm missing an example for docker-compose.
Especially on how to set a custom configuration file. I haven't found out yet, how to do that.

We've had a few requests in this area. The doc should include a how-to setup for using stunnel, SSL/TLS tunnels, ssh tunnels, etc to bridge NATS over an established TLS connection.

See:

• https://www.geeksforgeeks.org/what-is-ssl-tunneling/
• https://dzone.com/articles/what-is-ssl-tunneling
• https://cheapsslsecurity.com/p/understanding-the-difference-ssl-tunnel-vs-ssh-tunnel/
• https://en.wikipedia.org/wiki/Stunnel
• https://github.com/ghostunnel/ghostunnel

e.g. with autoscaling groups

I am a beginner and wants to see the examples of sending messages in C/C++. But the docs only provides them in Go, Java, JavaScript,Python, Ruby,TypeScript.
Where can I find sample codes for sending messages to NATS server in C/C++ languages??
Thank you very much in advance!

Request from Twitter to add Scaling to the Compare NATS documentation.

Section could be under: https://github.com/nats-io/nats.docs/tree/master/nats-server as "Websocket Support". If this documentation should be in a different place please comment here.

In a PR, please provide enough documentation to get started:

• General Description and Applicable Use Cases
• Getting Started Guide - Simple setup based on our recommendations.
• Basic parameters/configuration options
• Basic security
• Leaf Nodes and WS

Of course, add additional documentation as necessary.

/CC @gcolliso

I want to allow certain client to certain subject. Is it possible using nats?

Hi guys,

I'm trying to install mkpasswd from this documentation.

Issue No.1 - Wrong symbol in command
Copy button copies content > go get github.com/nats-io/nats-server/util/mkpasswd with starting >. It makes command invalid and getting error: -bash: go: Is a directory

Issue No.2 - Unable to install mkpasswd

Steps to reproduce:

1. Install go: sudo snap install go --classic
2. Install mkpasswd: go get github.com/nats-io/nats-server/util/mkpasswd

Actual: getting error

~$ go get github.com/nats-io/nats-server/util/mkpasswd
package github.com/nats-io/nats-server/util/mkpasswd: build constraints exclude all Go files in /home/ubuntu/go/src/github.com/nats-io/nats-server/util/mkpasswd

Expected: mkpasswd is installed.

OS:

Description:	Ubuntu 20.04.1 LTS
Release:	20.04
Codename:	focal

Go:
go version go1.15.6 linux/amd64

Go env:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ubuntu/.cache/go-build"
GOENV="/home/ubuntu/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ubuntu/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ubuntu/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/snap/go/6745"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/snap/go/6745/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build471404296=/tmp/go-build -gno-record-gcc-switches"

The queue subscriber enables multiple clients to subscribe to the same stream with the same “queue name” and messages will be distributed to each member of the queue group.
https://nats.io/blog/use-cases-for-persistent-logs-with-nats-streaming/

This is not correct. Queue groups are designed to load balance traffic.

The previous nkey layouts for creds files was different. The current layout is .nkeys/creds/operator/account/user.creds.

For the case when multiple Kubernetes clusters exist in the same cloud (e.g. AWS, Azure, GCP)

The docs on page:
https://docs.nats.io/nats-server/configuration/securing_nats/authorization
don't describe how allow and deny lists interact: whether either changes implicit defaults, which takes precedence, etc.

On https://docs.nats.io/nats-server/flags it doesn't specify whether --tlsverify enables the verify or verify_and_map configuration options (found here: https://docs.nats.io/nats-server/configuration/securing_nats/tls). Which one is it?

Have this small snippet on getting started with NGS + Python using Docker but not sure where to add it:
https://gist.github.com/wallyqs/d2ecbec2b61dd170d795ac9855efdeed

https://github.com/nats-io/nats.docs/blob/master/nats-tools/nk.md says

go get github.com/nats-io/nk

but should be

go get github.com/nats-io/nkeys/nk

Section could be under: https://github.com/nats-io/nats.docs/tree/master/nats-server as "NATS Account Management", to cover the new NATS based account resolution. If this documentation should be in a different place please comment here.

In a PR, please provide enough documentation to get started:

• General Description and Applicable Use Cases
• Getting Started Guide - Simple setup based on our recommendations.
• Basic parameters/configuration options

Of course, add additional documentation as necessary.

/CC @gcolliso