sntnupl / devcontainers-mongodb-replica-set-with-docker

Docker based development container template to run a MongoDB replica set cluster in local machine.

Set up a secure MongoDB Replica-set for development using Docker

This repository showcases how one can setup a secure MongoDB replica set for development using Docker.
I have explained the details of the docker-compose.yml file in my blog post, link to which can be found at this link.

NOTE
This replica set is for educational and demonstration purposes ONLY.
Running multiple nodes of a Replica Set within a single machine is an anti-pattern, and MUST BE AVOIDED in Production.
Using this setup for local development though, is perfectly fine.

Getting Started

Most of the stuff here will work out-of-the-box, and the containers will be up and running once you run docker-compose up -d. However, there are a few configuration steps that you would need to incorporate - which are mentioned below.

Configure environmental variables

If you check the docker-compose.yml file, we are relying on .env file to populate the env variables for the compose. A sample.env file is provided here, use it to create your own .env file.

Create keyfiles for MongoDB nodes to authenticate themselves

For this we need to create a keyfile.

In Linux, the following are the commands:

openssl rand -base64 700 > file.key
chmod 400 file.key
sudo chown 999:999 file.key

I have kept a sample file in .docker/mongodb path, but you must follow the steps above to generate a new file.key.

Update hostnames

Once the replica set is up, you will need to update hostnames in local /etc/hosts file.

Sample entry

127.0.0.1 localhost mongo1 mongo2 mongo3

This change is needed to be done in all client machine, from where you would like to connect to this Replica Set.
So if you are running this replica set in your local development machine, and want to connect to it, you would have to update /etc/hosts file in your local dev machine ONLY.
However, if you want to connect to this replica set from a different machine, you would need to update the /etc/hosts file in that other machine as well.

Sample entry on a different machine

10.20.30.40 mongo1 mongo2 mongo3

Here, 10.20.30.40 is the public-ip of the machine where this replica set is running.

NOTE: In windows, the hosts file is located at C:\Windows\System32\drivers\etc\hosts

Result

Once the MongoDB replica set is up and running, Once everything comes up, you can run docker-compose status, and see something like this:

To connect to the replica set, you can use mongo client.

Connect to replica set running on local machine

$ mongo "mongodb://localhost:30001,localhost:30002,localhost:30003/<MONGO_INITDB_DATABASE>?replicaSet=rs0" -u <MONGO_INITDB_USERNAME> --authenticationDatabase admin

Connect to replica set running on local or remote machine

$ mongo "mongodb://mongo1:30001,mongo2:30002,mongo3:30003/<MONGO_INITDB_DATABASE>?replicaSet=rs0" -u <MONGO_INITDB_USERNAME> --authenticationDatabase admin

As mentioned above, you MUST update the local hosts file in the client machine (running this command), to point to the public IP address of the machine where the replica set is running.

Remember to replace the <MONGO_INITDB_DATABASE> and <MONGO_INITDB_USERNAME> tags with appropriate values that you've put in the .env file.

Once connected you can use the more succinct db.isMaster() command to debug details like hosts of the replicaset a client must connect to, if the node is primary, etc.

db.isMaster()

Full documentation of the command is available at this link.

devcontainers-mongodb-replica-set-with-docker's Issues

Couldn't connect to server mongo1:30001

I followed the instructions but I always get the result like this. Please help me fix this.

docker-compose up

Creating network "devcontainers-mongodb-replica-set-with-docker_netApplication" with the default driver
Creating mongo1 ... done
Creating mongo3 ... done
Creating mongo2 ... done
Creating mongo-setup ... done
Attaching to mongo1, mongo3, mongo2, mongo-setup
mongo-setup | ********************************************** mongo1
mongo-setup | Waiting for startup..
mongo3 exited with code 1
mongo2 exited with code 1
mongo1 exited with code 1
mongo2 exited with code 1
mongo1 exited with code 1
mongo3 exited with code 1
mongo2 exited with code 1
mongo-setup | done
mongo-setup | SETUP.sh time now: 15:31:39
mongo-setup | MongoDB shell version v4.2.15
mongo-setup | connecting to: mongodb://mongo1:30001/?compressors=disabled&gssapiServiceName=mongodb
mongo-setup | 2021-08-02T15:31:40.675+0000 E QUERY [js] Error: couldn't connect to server mongo1:30001, connection attempt failed: SocketException: Error connecting to mongo1:30001 (172.18.0.2:30001) :: caused by :: Connection refused :
mongo-setup | connect@src/mongo/shell/mongo.js:353:17
mongo-setup | @(connect):2:6
mongo-setup | 2021-08-02T15:31:40.677+0000 F - [main] exception: connect failed
mongo-setup | 2021-08-02T15:31:40.677+0000 E - [main] exiting with code 1
mongo1 exited with code 1
mongo3 exited with code 1
mongo2 exited with code 1

ReplicaSet connection failed

Hi,

I have deployed successfully the 3 nodes,

[root@localhost devcontainers-mongodb-replica-set-with-docker]# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                    PORTS                      NAMES
65282c4114d5        mongo:4.2           "docker-entrypoint.s…"   About an hour ago   Up 15 minutes             0.0.0.0:30002->27017/tcp   mongo2
f245429d27dc        mongo:4.2           "docker-entrypoint.s…"   About an hour ago   Up 12 minutes             0.0.0.0:30003->27017/tcp   mongo3
97f391716591        mongo:4.2           "docker-entrypoint.s…"   About an hour ago   Up 12 minutes (healthy)   0.0.0.0:30001->27017/tcp   mongo1

However, if you try to connect using the replicaSet name you cant connect.

This works:
mongo mongodb://pruebauser:prueba123@localhost:30001,localhost:30002,localhost:30003/prueba

And this doesnt work (you specify the replicaSet on connection ?replicaSet=rs0):

[root@localhost devcontainers-mongodb-replica-set-with-docker]# mongo mongodb://pruebauser:prueba123@localhost:30001,localhost:30002,localhost:30003/prueba?replicaSet=rs0
MongoDB shell version v4.2.0
connecting to: mongodb://localhost:30001,localhost:30002,localhost:30003/prueba?compressors=disabled&gssapiServiceName=mongodb&replicaSet=rs0

{"t":{"$date":"2021-03-18T10:18:11.751Z"},"s":"I",  "c":"NETWORK",  "id":4333208, "ctx":"ReplicaSetMonitor-TaskExecutor","msg":"RSM host selection timeout","attr":{"replicaSet":"rs0","error":"FailedToSatisfyReadPreference: Could not find host matching read preference { mode: \"nearest\" } for set rs0"}}
Error: connect failed to replica set rs0/localhost:30001,localhost:30002,localhost:30003 :
connect@src/mongo/shell/mongo.js:374:17
@(connect):2:6
exception: connect failed
exiting with code 1

The env file is the next below:

[root@localhost devcontainers-mongodb-replica-set-with-docker]# cat .env
# MongoDB
MONGO_URL=mongodb://mongodb:27017
MONGO_INITDB_ROOT_USERNAME=admin
MONGO_INITDB_ROOT_PASSWORD=admin123
MONGO_INITDB_DATABASE=prueba
MONGO_INITDB_USERNAME=pruebauser
MONGO_INITDB_PASSWORD=prueba123
MONGO_REPLICA_SET_NAME=rs0