grpc_proxy imposter load failed when different proto files about mountebank-grpc

[cbrz]

Hey @mukrisp , had some time to take a look.

The problem is the JSON body of the HTTP request. The services key needs to match the <package>.<servicename> format.

Since the .proto contains package 'reverse' and service 'ReverseService', changing the services key to 'reverse.ReverseService' will work:

{
	"protocol": "grpc",
	"host": "0.0.0.0",
	"port": {{imposter-port}},
	"loglevel": "debug",
	"recordRequests": true,
	"services": {
		"reverse.ReverseService": {
			"file": "reverse.proto"
		}
	},
	"options": {
		"protobufjs": {
			"includeDirs": ["/etc/mountebank/mountebank-grpc/src/protos"]
		}
	},
	"stubs": [
		{
			"responses": [
				{
					"proxy": {
						"to": "135.249.22.155:50051",
						"mode": "proxyAlways",
						"predicateGenerators": [{ "matches": {"path": true} }]
					}
				}
			]
		}
	]
}

[mukrisp]

Hi,

Thanks a lot for looking into this.
With the change able to load the imposter.

I have a grpc_client which is going to initiate a message to grpc_server.
Trying to proxy it via MB.
Message is reaching the MB but looks like there is some issue post processing.
Please find below the console log.

MB_CONSOLE_LOG:

[ root@0aeda8e99f51:/usr/lib/node_modules/mountebank/mountebank-grpc {master } ]$ mb start --protofile protocols.json --loglevel debug --debug true
info: [mb:2525] Loaded custom protocol grpc
info: [mb:2525] mountebank v2.1.2 now taking orders - point your browser to http://localhost:2525/ for help
debug: [mb:2525] config: {"options":{"protofile":"protocols.json","loglevel":"debug","debug":true,"port":2525,"noParse":false,"no-parse":false,"pidfile":"mb.pid","nologfile":false,"logfile":"mb.log","allowInjection":false,"allow-injection":false,"localOnly":false,"local-only":false,"ipWhitelist":[""],"ip-whitelist":"*","mock":false,"heroku":false},"process":{"nodeVersion":"v13.7.0","architecture":"x64","platform":"linux"}}
info: [mb:2525] POST /imposters
debug: [mb:2525] ::ffff:127.0.0.1:51840 => {"protocol":"grpc","host":"0.0.0.0","port":"4546","loglevel":"debug","recordRequests":true,"services":{"reverse.ReverseService":{"file":"reverse.proto"}},"options":{"protobufjs":{"includeDirs":["/usr/lib/node_modules/mountebank/mountebank-grpc/src/protos"]}},"stubs":[{"responses":[{"proxy":{"to":"135.249.22.155:50051","mode":"proxyAlways","predicateGenerators":[{"matches":{"value":true}}]}}]}]}
info: [grpc:4546] Open for business...
info: [grpc:4546] [2020-02-13T13:52:01.163Z] server started on port '4546'
debug: [grpc:4546] [2020-02-13T13:52:14.406Z] sending unary-unary rpc
debug: [grpc:4546] [2020-02-13T13:52:14.408Z] send request to mountebank
debug: [grpc:4546] [2020-02-13T13:52:14.408Z] url='http://localhost:2525/imposters/4546/_requests', data='{"request":{"peer":"ipv4:127.0.0.1:40564","path":"/reverse.ReverseService/ReverseString","value":{"data":"Hello, world"},"metadata":{"initial":{"user-agent":"grpc-go/1.27.0-dev"}}}}'
error: [grpc:4546] (node:5234) UnhandledPromiseRejectionWarning: TypeError: Converting circular structure to JSON
--> starting at object with constructor 'ClientRequest'
| property 'socket' -> object with constructor 'Socket'
--- property '_httpMessage' closes the circle
at JSON.stringify ()
at Object.sendRequest (/usr/lib/node_modules/mountebank/mountebank-grpc/src/mountebank/request.js:16:36)
at processTicksAndRejections (internal/process/task_queues.js:97:5)

error: [grpc:4546] (node:5234) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:5234) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

MB_IMPOSTER_CONFIG:

[ root@0aeda8e99f51:/usr/lib/node_modules/mountebank/mountebank-grpc {master *} ]$ cat grpc_proxy.ejs
{
"protocol": "grpc",
"host": "0.0.0.0",
"port": "4546",
"loglevel": "debug",
"recordRequests": true,
"services": {
"reverse.ReverseService": {
"file": "reverse.proto"
}
},
"options": {
"protobufjs": {
"includeDirs": ["/usr/lib/node_modules/mountebank/mountebank-grpc/src/protos"]
}
},
"stubs": [
{
"responses": [
{
"proxy": {
"to": "135.249.22.155:50051",
"mode": "proxyAlways",
"predicateGenerators": [{ "matches": {"value": true} }]
}
}
]
}
]
}

Thanks and Regards,
Muthukrishnan.P

[cbrz]

Hey @mukrisp, it's going to take me a little bit to get to this... but can you try setting the loglevel to 'info' and make sure you don't see debug logs from the grpc service? I suspect this is coming from the JSON.stringify function in one of the logging calls. Thanks!

[mukrisp]

Hi,

Okay. I will try to debug this.

Analysis:

Yes, error was from JSON.stringify , but changing log level did not help. After directly printing the error. Got additional info in log.

Response var is empty.

error: [grpc:4546] [2020-02-14T08:29:30.487Z] error=Error: Request failed with status code 504
error: [grpc:4546] (node:6504) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'response' of undefined
at /usr/lib/node_modules/mountebank/mountebank-grpc/src/mock.js:70:39
at processTicksAndRejections (internal/process/task_queues.js:97:5)

MB_CONSOLE_LOG:

[ root@0aeda8e99f51:/usr/lib/node_modules/mountebank/mountebank-grpc {master } ]$ mb start --protofile protocols.json --loglevel debug --debug true
info: [mb:2525] Loaded custom protocol grpc
info: [mb:2525] mountebank v2.1.2 now taking orders - point your browser to http://localhost:2525/ for help
debug: [mb:2525] config: {"options":{"protofile":"protocols.json","loglevel":"debug","debug":true,"port":2525,"noParse":false,"no-parse":false,"pidfile":"mb.pid","nologfile":false,"logfile":"mb.log","allowInjection":false,"allow-injection":false,"localOnly":false,"local-only":false,"ipWhitelist":[""],"ip-whitelist":"*","mock":false,"heroku":false},"process":{"nodeVersion":"v13.7.0","architecture":"x64","platform":"linux"}}
info: [mb:2525] POST /imposters
debug: [mb:2525] ::ffff:127.0.0.1:33064 => {"protocol":"grpc","host":"0.0.0.0","port":"4546","loglevel":"debug","recordRequests":true,"services":{"reverse.ReverseService":{"file":"reverse.proto"}},"options":{"protobufjs":{"includeDirs":["/usr/lib/node_modules/mountebank/mountebank-grpc/src/protos"]}},"stubs":[{"responses":[{"proxy":{"to":"135.249.22.155:50051","mode":"proxyAlways","predicateGenerators":[{"matches":{"value":true}}]}}]}]}
info: [grpc:4546] Open for business...
info: [grpc:4546] [2020-02-14T08:29:27.757Z] server started on port '4546'
debug: [grpc:4546] [2020-02-14T08:29:30.161Z] sending unary-unary rpc
debug: [grpc:4546] [2020-02-14T08:29:30.162Z] send request to mountebank
debug: [grpc:4546] [2020-02-14T08:29:30.162Z] url='http://localhost:2525/imposters/4546/_requests', data='{"request":{"peer":"ipv4:127.0.0.1:50020","path":"/reverse.ReverseService/ReverseString","value":{"data":"Hello, world"},"metadata":{"initial":{"user-agent":"grpc-go/1.27.0-dev"}}}}'
error: [grpc:4546] [2020-02-14T08:29:30.487Z] error=Error: Request failed with status code 504
error: [grpc:4546] (node:6504) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'response' of undefined
at /usr/lib/node_modules/mountebank/mountebank-grpc/src/mock.js:70:39
at processTicksAndRejections (internal/process/task_queues.js:97:5)

error: [grpc:4546] (node:6504) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:6504) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Thanks and Regards,
Muthukrishnan.P

[cbrz]

Hey @mukrisp ,
I tried to replicate your issue using python grpc and was not able to. Even the logs are printing correctly.

Here's the debug logs I see when proxying a request:

info: [mb:2525] Loaded custom protocol grpc
info: [mb:2525] mountebank v2.2.0 now taking orders - point your browser to http://localhost:2525/ for help
debug: [mb:2525] config: {"options":{"protofile":"protocols.json","loglevel":"debug","debug":true,"port":2525,"noParse":false,"no-parse":false,"pidfile":"mb.pid","nologfile":false,"logfile":"mb.log","allowInjectio
n":false,"allow-injection":false,"localOnly":false,"local-only":false,"ipWhitelist":["*"],"ip-whitelist":"*","mock":false,"heroku":false},"process":{"nodeVersion":"v10.15.3","architecture":"x64","platform":"linux"
}}
info: [mb:2525] POST /imposters
debug: [mb:2525] ::1:51884 => {"protocol":"grpc","host":"0.0.0.0","port":4545,"loglevel":"debug","recordRequests":true,"services":{"data.Data":{"file":"data.proto"}},"options":{"protobufjs":{"includeDirs":["/home/
cbrz/dev/git/grpc"]}},"stubs":[{"responses":[{"proxy":{"to":"127.0.0.1:50051","mode":"proxyAlways","predicateGenerators":[{"matches":{"path":true}}]}}]}]}
info: [grpc:4545] Open for business...
info: [grpc:4545] [2020-02-22T07:39:53.747Z] server started on port '4545'
debug: [grpc:4545] [2020-02-22T07:40:02.230Z] sending unary-unary rpc
debug: [grpc:4545] [2020-02-22T07:40:02.231Z] send request to mountebank
debug: [grpc:4545] [2020-02-22T07:40:02.231Z] url='http://localhost:2525/imposters/4545/_requests', data='{"request":{"peer":"ipv4:127.0.0.1:57246","path":"/data.Data/Reverse","value":{"datum":"Hello, world!"},"me
tadata":{"initial":{"1":"2","key-request":"value-request","user-agent":"grpc-python/1.27.2 grpc-c/9.0.0 (manylinux; chttp2; guantao)"}}}}'
debug: [mb:2525] POST /imposters/4545/_requests
debug: [grpc:4545] using predicate match: {}
debug: [grpc:4545] generating response from {"proxy":{"to":"127.0.0.1:50051","mode":"proxyAlways","predicateGenerators":[{"matches":{"path":true}}]}}
debug: [grpc:4545] [2020-02-22T07:40:02.245Z] response.data="{"proxy":{"to":"127.0.0.1:50051","mode":"proxyAlways","predicateGenerators":[{"matches":{"path":true}}]},"request":{"peer":"ipv4:127.0.0.1:57246","path"
:"/data.Data/Reverse","value":{"datum":"Hello, world!"},"metadata":{"initial":{"1":"2","key-request":"value-request","user-agent":"grpc-python/1.27.2 grpc-c/9.0.0 (manylinux; chttp2; guantao)"}}},"callbackURL":"ht
tp://localhost:2525/imposters/4545/_requests/0"}"
debug: [grpc:4545] [2020-02-22T07:40:02.250Z] metadata='{"_internal_repr":{},"flags":0}'
debug: [grpc:4545] [2020-02-22T07:40:02.250Z] data="{"datum":"!dlrow ,olleH"}"
debug: [grpc:4545] [2020-02-22T07:40:02.251Z] status='{"code":0,"details":"","metadata":{"_internal_repr":{"1":["2"],"key-response":["value-response"]},"flags":0}}'
debug: [grpc:4545] [2020-02-22T07:40:02.251Z] proxy_response='{"value":{"datum":"!dlrow ,olleH"},"metadata":{"initial":{},"trailing":{"1":"2","key-response":"value-response"}}}'
debug: [grpc:4545] [2020-02-22T07:40:02.251Z] send request to mountebank
debug: [grpc:4545] [2020-02-22T07:40:02.251Z] url='http://localhost:2525/imposters/4545/_requests/0', data='{"proxyResponse":{"value":{"datum":"!dlrow ,olleH"},"metadata":{"initial":{},"trailing":{"1":"2","key-res
ponse":"value-response"}}}}'
debug: [mb:2525] POST /imposters/4545/_requests/0
debug: [grpc:4545] [2020-02-22T07:40:02.255Z] response.data="{"value":{"datum":"!dlrow ,olleH"},"metadata":{"initial":{},"trailing":{"1":"2","key-response":"value-response"}},"_proxyResponseTime":11}"
^Cinfo: [mb:2525] Adios - see you soon?

Is there anyway you can zip or tar the golang client/server and proto files you're using? I'd like to try to replicate this a bit more closely to what you're doing.

Thanks!

[mukrisp]

Hi,

Thanks for checking.
Please find the go source/proto in this link.
https://github.com/santiaago/grpc.demo

If MB proxy works for client/server in python, i ll take it for evaluation.

I have some further questions. Need your comments for the same.

1. Instead of grpc, if we used standard tcp, but data generation to be done with .proto files, what changes in MB is needed.
2. Once imposters are created from proxy, If we want to customize imposters by adding predicates to match data in a more granular way by doing data lookup with .proto fields (Example: If there are param1,value1 param2,value2 param3,value3 in the protobuff data if we want to add predicate to match param2 with value XYZ by parsing data with .proto files, can this be done in MB predicate matching .?
3. After matching protobuff data with predicates, to send response, a data generated from different proto file, Is this possible in MB ? On high level what are the changes required to achieve this in MB.

Thanks and Regards,
Muthukrishnan.P

[cbrz]

Hey @mukrisp,
I tried out the demo from the link you posted, and it worked for me. The imposter after running looked like this:

{
    "protocol": "grpc",
    "port": 50050,
    "numberOfRequests": 1,
    "recordRequests": true,
    "services": {
        "reverse.ReverseService": {
            "file": "/home/cbrz/go/src/github.com/santiaago/grpc.demo/proto/reverse.proto"
        }
    },
    "requests": [
        {
            "peer": "ipv6:[::1]:55830",
            "path": "/reverse.ReverseService/ReverseString",
            "value": {
                "data": "Hello, world"
            },
            "metadata": {
                "initial": {
                    "user-agent": "grpc-go/1.28.0-dev"
                }
            },
            "timestamp": "2020-03-01T03:25:16.566Z"
        }
    ],
    "stubs": [
        {
            "responses": [
                {
                    "proxy": {
                        "to": "localhost:50051",
                        "mode": "proxyAlways",
                        "predicateGenerators": [
                            {
                                "matches": {
                                    "path": true
                                }
                            }
                        ]
                    }
                }
            ],
            "_links": {
                "self": {
                    "href": "http://localhost:2525/imposters/50050/stubs/0"
                }
            }
        },
        {
            "predicates": [
                {
                    "deepEquals": {
                        "path": "/reverse.ReverseService/ReverseString"
                    }
                }
            ],
            "responses": [
                {
                    "is": {
                        "value": {
                            "reversed": "dlrow ,olleH"
                        },
                        "metadata": {
                            "initial": {},
                            "trailing": {}
                        },
                        "_proxyResponseTime": 13
                    }
                }
            ],
            "_links": {
                "self": {
                    "href": "http://localhost:2525/imposters/50050/stubs/1"
                }
            }
        }
    ],
    "_links": {
        "self": {
            "href": "http://localhost:2525/imposters/50050"
        },
        "stubs": {
            "href": "http://localhost:2525/imposters/50050/stubs"
        }
    }
}

Are you sure there weren't any changes from trying to debug the protocol?

As for your questions:
Question 1:
It sounds like you want to use protobuf messaging, but not gRPC. In that case, I think you'd need a new custom server/protocol.

Question 2:
You should be able to use mountebank stub predicates to customize the mock response.

The Mountebank protocol workflow involves converting the request to JSON, submitting that to Mountebank. Mountebank (depending on the stub definition) will send JSON back to the protocol which transforms that to a message which is sent to the client. So... anything Mountebank supports (for stubs) should work with this protocol (granted everything is coded correctly of course).

Question 3:
I'm not sure if I'm understanding this correctly... do you mean having a service return a different protobuf message than it was defined with? Or do you mean a different .proto file?

The first one can't be done, but the second one you can include multiple .proto files with the options.protobufjs.includeDirs setting. You'd still need the .proto that has your service to use the proper imports though.