Coder Social home page Coder Social logo

mpihlak / mongoproxy Goto Github PK

View Code? Open in Web Editor NEW
42.0 2.0 8.0 1.77 MB

Lightweight proxy to collect MongoDb client metrics

License: MIT License

Rust 86.55% Python 9.23% Dockerfile 0.42% Shell 0.77% Go 3.04%
metrics sidecar mongodb tracing observability

mongoproxy's People

Contributors

alekseiplotnikov avatar dependabot[bot] avatar mpihlak avatar zekena2 avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

marinanekrassova morpheus-med chrisvfabio aswinkarthik harscoet bertbrouwer zekena2 alekseiplotnikov

mongoproxy's Issues

Question: is this proxy stable?

I need a mongo proxy that works with the 3.6+ wire protocol. My primary need is to intercept write requests and return success while letting read requests through.

Is this proxy a good place to start? If not, do you know of others?

Support outbound TLS encryption.

Since your proxy inspects the traffic, the traffic obviously cannot be encrypted. That is fine for me if I deploy the proxy as a sidecar.

However, my application must be HIPAA compliant. So, I need to encrypt the traffic when it leaves the proxy.

At present, it appears that the proxy does not support outbound TLS encryption. Any plans to support it?

Robo3T 1.4.1 sends extra data that prevents logging mongo messages

I'm interested in capturing logs of client activity, e.g. what queries they send. To do this I look for MsgOpMsg output from the client tracker. Unfortunately Robo3t 1.4.1 sends extra data that causes the rdr.read*() calls that use await? to exit with an io::ErrorKind::UnexpectedEof, even if a document has been successfully read.

Robo3t 1.1.1 seems to work fine.

Reproducing the issue:

  1. Start mongoproxy: RUST_LOG=info target/release/mongoproxy --log-mongo-messages --proxy 27018:localhost:27017 | grep "client tracker"
  2. Connect to the proxy using Robo3T 1.4.1
  3. Query a collection.
  4. Observe the following errors, and with no query logging (no { find: ...}):
Oct 22 09:07:49.577  INFO handle_connection{client_addr="127.0.0.1:51290" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_QUERY BSON: { isMaster: 1, client: { application: { name: "robo3t-1.4.1" }, driver: { name: "MongoDB Internal Client", version: "4.2.6-17-g6bce88c" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "20.04" } } }
Oct 22 09:07:49.578  INFO handle_connection{client_addr="127.0.0.1:51290" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { listDatabases: 1, nameOnly: true, $readPreference: { mode: "secondaryPreferred" }, $db: "admin" }
Oct 22 09:07:49.578 ERROR handle_connection{client_addr="127.0.0.1:51290" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: Failed to parse MongoDb 2013 message: unexpected end of file
Oct 22 09:07:49.631  INFO handle_connection{client_addr="127.0.0.1:51296" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_QUERY BSON: { isMaster: 1, client: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "4.2.6-17-g6bce88c" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "20.04" } } }
Oct 22 09:07:49.632  INFO handle_connection{client_addr="127.0.0.1:51296" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { whatsmyuri: 1, $db: "admin" }
Oct 22 09:07:49.632 ERROR handle_connection{client_addr="127.0.0.1:51296" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: Failed to parse MongoDb 2013 message: unexpected end of file
Oct 22 09:07:56.704  INFO handle_connection{client_addr="127.0.0.1:51312" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_QUERY BSON: { isMaster: 1, client: { application: { name: "robo3t-1.4.1" }, driver: { name: "MongoDB Internal Client", version: "4.2.6-17-g6bce88c" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "20.04" } } }
Oct 22 09:07:56.706  INFO handle_connection{client_addr="127.0.0.1:51312" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { ping: 1, $db: "admin" }
Oct 22 09:07:56.706 ERROR handle_connection{client_addr="127.0.0.1:51312" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: Failed to parse MongoDb 2013 message: unexpected end of file

Expected behaviour (from a PR I will open soon):

Oct 22 09:10:14.693  INFO handle_connection{client_addr="127.0.0.1:51408" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_QUERY BSON: { isMaster: 1, client: { application: { name: "robo3t-1.4.1" }, driver: { name: "MongoDB Internal Client", version: "4.2.6-17-g6bce88c" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "20.04" } } }
Oct 22 09:10:14.694  INFO handle_connection{client_addr="127.0.0.1:51408" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { listDatabases: 1, nameOnly: true, $readPreference: { mode: "secondaryPreferred" }, $db: "admin" }
Oct 22 09:10:14.743  INFO handle_connection{client_addr="127.0.0.1:51414" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_QUERY BSON: { isMaster: 1, client: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "4.2.6-17-g6bce88c" }, os: { type: "Linux", name: "Ubuntu", architecture: "x86_64", version: "20.04" } } }
Oct 22 09:10:14.744  INFO handle_connection{client_addr="127.0.0.1:51414" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { whatsmyuri: 1, $db: "admin" }
Oct 22 09:10:14.747  INFO handle_connection{client_addr="127.0.0.1:51414" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { buildinfo: 1, $db: "admin" }
Oct 22 09:10:14.755  INFO handle_connection{client_addr="127.0.0.1:51408" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { serverStatus: "1", $db: "db" }
Oct 22 09:10:14.755  WARN handle_connection{client_addr="127.0.0.1:51408" server_addr="localhost:27017"}:client tracker: mongoproxy::tracker: unsupported op: serverStatus
Oct 22 09:10:14.756  INFO handle_connection{client_addr="127.0.0.1:51408" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { buildInfo: "1", $db: "db" }
Oct 22 09:10:14.756  INFO handle_connection{client_addr="127.0.0.1:51408" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { buildInfo: "1", $db: "db" }
Oct 22 09:10:14.760  INFO handle_connection{client_addr="127.0.0.1:51414" server_addr="localhost:27017"}:client tracker: mongoproxy::mongodb: OP_MSG BSON: { find: "user", filter: {}, lsid: { id: BinData(0x4, NNUSAJRbSh+lVvRyOsMazg==) }, $clusterTime: { clusterTime: Timestamp(1603379410, 4), signature: { hash: BinData(0x0, AAAAAAAAAAAAAAAAAAAAAAAAAAA=), keyId: 0 } }, $readPreference: { mode: "secondaryPreferred" }, $db: "test" }

Service mesh integration

I use linkerd and it uses Sidecars with iptables. I want to use your proxy in that environment. I can figure it out myself, but if you already have a config that avoids conflicts I would appreciate it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.