threemammals / ocelot Goto Github PK
View Code? Open in Web Editor NEW.NET API Gateway
Home Page: https://www.nuget.org/packages/Ocelot
License: MIT License
.NET API Gateway
Home Page: https://www.nuget.org/packages/Ocelot
License: MIT License
This may be a little nit-picky, but why are you renaming ApiName (in identity server) to ScopeName (ocelot)? The same goes for AllowedScopes (identity server) to AdditionalScopes (ocelot)?
Are there any examples of using an identity server? How do you call the gateway? How does it interact with identity server? Do you interact with identity server directly and then send the bearer token to ocelot or do you hit the identity server through ocelot?
No idea how to do this but would be nice
Look at coveralls integration might be able to get code coverage stats
Hi,
I'm doing a project with Ocelot and IdentityServer. I dont understand the usability of some AuthenticationOptions properties.
"AuthenticationOptions": {
"Provider": "IdentityServer",
"ProviderRootUrl": "http://localhost:52888",
"ApiName": "api",
"AllowedScopes": [
"openid",
"offline_access"
],
"ApiSecret": "secret"
}
Why should you indicate ApiName, ApiSecret or AllowedScopes??? My project works whether I add those data or not add them. I dont understand the usefulness of these parameters since they do not seem to be used. What does Ocelot use for these three properties?
"AuthenticationOptions": {
"Provider": "IdentityServer",
"ProviderRootUrl": "http://localhost:52888",
"ApiName": "api",
"AllowedScopes": [
"openid",
"offline_access"
],
"ApiSecret": "secret",
"RouteClaimsRequirement": {
"MyClaim1": "bbb"
}
}
Thanks.
Set up CI here to build the Ocelot project and run tests
HttpClientHttpRequester need implements the Circuit Breaker Pattern,I using Polly from thepollyproject.org https://github.com/App-vNext/Polly/wiki/Transient-fault-handling%3A-why-Retry-and-Circuit-Breaker%3F
Hey @TomPallister,
Just been looking through the code trying to understand how it all works. I've got a few thoughts about the middleware implementation...
Is there any reason for ExceptionHandlingMiddleware
to not implement OcelotMiddleware
?
Any reason for not defining abstract Task Invoke(HttpContext context)
on OcelotMiddleware
? Adding this would give extra compile-time check that middleware implementors are on the correct path.
It took me a while to understand what parts of the code were middleware, and what parts of the code were infrastructure stuff. Given that the middleware are effectively the feature slices of the project, and are where all the magic happens, I feel it could do with being a bit more distinct from the rest of the supporting code. Any opinion on grouping all the middleware together as individual folders under the an Ocelot.Middleware
namespace? Eg,
+-Ocelot
+-Middleware
| +-Authentication
| | +- ...
| |
| +-Authorisation
| | +- ...
| |
| +-...
|
+-other stuff
This structure would also provide a path to providing a more modular package-based approach to middleware in the future, with a common naming convention: optional middleware could be published in packages; the common Ocelot.Middleware.Whatever
convention would make it easy to discover these packages; and say someone didn't want to use identityserver then they could choose to avoid pulling this package in.
If you think any of this is worth doing I'm happy to pick up the work on it.
The acceptance tests rely on TestConfiguration to get the path to the configuration file used by the tests. This causes a couple of problems...
We should be able to replace this with dynamic code that calculates the config file path dynamically. Surely we can do something like
using System.IO;
\\...
return Path.Combine(Directory.GetCurrentDirectory(), "configuration.json");
ResponderMiddleware.cs looks as if it can set a 500 to be returned, however, the implementation of ErrorsToHttpStatusCodeMapper creates a new OkResponse and does not pass the list of errors through.
As such, when ResponderMiddleware
calls IsError false
will always be returned.
I am 85% sure what I have said above is correct, as I am still learning the codebase, happy to PR though if I can be pointed in the correct direction.
I'm not really sure the logging strategy is any good so need to review it and have a think
At the moment we match the URLs using Regex built in FileOcelotConfigurationCreator.BuildUpstreamTemplatePattern. I noticed that if you had an upstream path of / it made a regex of (?i)/$ then if you tried to match something like /working/ it would match but you were expecting a different match for /working/ :(
In the regexmatcher tests this shows the problem
[Fact]
public void should_not_match()
{
this.Given(x => x.GivenIHaveAUpstreamPath("/working/"))
.And(x => x.GivenIHaveAnUpstreamUrlTemplatePattern("(?i)/$"))
.When(x => x.WhenIMatchThePaths())
.And(x => x.ThenTheResultIsFalse())
.BDDfy();
}
So i changed the FileOcelotConfigurationCreator.BuildUpstreamTemplatePattern to do
if (upstreamTemplate == "/")
{
return upstreamTemplate;
}
Then in the DownstreamRouteFinder.FindDownstreamRoute method i just do
if (upstreamUrlPath == reRoute.UpstreamTemplatePattern)
{
var templateVariableNameAndValues = _urlPathPlaceholderNameAndValueFinder.Find(upstreamUrlPath, reRoute.UpstreamPathTemplate.Value);
return new OkResponse<DownstreamRoute>(new DownstreamRoute(templateVariableNameAndValues.Data, reRoute));
}
Not sure this is ideal!
Nuget packages plus the description and URL information, facilitate project
Here we used Apache Bench performance in a simple test. We simulated 1000 users, 100 requests, the following are test results, configuration of the machine is CPU:Intel i5 2 nuclear, 4G memory test on the machine:
D:\Workshop\ab>ab.exe -n 1000 -c 100 http://localhost:5000/api/values/
This is ApacheBench, Version 2.3 <$Revision: 1757674 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Kestrel
Server Hostname: localhost
Server Port: 5000
Document Path: /api/values/
Document Length: 19 bytes
Concurrency Level: 100
Time taken for tests: 7.880 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 178000 bytes
HTML transferred: 19000 bytes
Requests per second: 126.90 [#/sec] (mean)
Time per request: 788.013 [ms] (mean)
Time per request: 7.880 [ms] (mean, across all concurrent requests)
Transfer rate: 22.06 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 2
Processing: 349 758 239.5 672 1800
Waiting: 349 758 239.4 672 1800
Total: 349 758 239.4 672 1801
Percentage of the requests served within a certain time (ms)
50% 672
66% 701
75% 738
80% 748
90% 1063
95% 1457
98% 1653
99% 1717
100% 1801 (longest request)
Average number of requests per second (throughput) to 126.90, judging by the number of requests from different distributions, response time is also more stable than direct access interface performance decreases sharply, a difference of 30 times.
ab.exe -n 1000 -c 100 http://localhost:9030/api/values/
This is ApacheBench, Version 2.3 <$Revision: 1757674 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Kestrel
Server Hostname: localhost
Server Port: 9030
Document Path: /api/values/
Document Length: 19 bytes
Concurrency Level: 100
Time taken for tests: 0.297 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 158000 bytes
HTML transferred: 19000 bytes
Requests per second: 3367.05 [#/sec] (mean)
Time per request: 29.700 [ms] (mean)
Time per request: 0.297 [ms] (mean, across all concurrent requests)
Transfer rate: 519.53 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 1
Processing: 0 28 18.7 18 75
Waiting: 0 25 18.8 15 65
Total: 0 28 18.7 18 75
Percentage of the requests served within a certain time (ms)
50% 18
66% 39
75% 47
80% 49
90% 54
95% 59
98% 64
99% 65
100% 75 (longest request)
I´m following the documentation to build an api gateway with Ocelot. I created a simple solution with two API, the APIGateway and the API1 (both with the Visual Studio Api template) with the target to try to redirect the requests. I have some questions:
services.AddOcelot(Configuration);
in ConfigureServices method from Startup class. This method AddOcelot is not available. I´m using:
services.AddOcelotOutputCaching(settings);
services.AddOcelotFileConfiguration(Configuration);
services.AddOcelot();
is this correct?
var host = builder.Build();
System.InvalidOperationException: 'A public method named 'ConfigureDevelopment' or 'Configure' could not be found in the 'OcelotTest.Startup' type.'
If I use the code by default, seems run OK.
var host = new WebHostBuilder()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseIISIntegration()
.UseStartup<Startup>()
.UseApplicationInsights()
.Build();
host.Run();
It´s possible use this code or will affect in something in the future?
Now I will try to use it with my IdentityServer, although I have to investigate a bit since I am very new to these two tools.
Thanks for your time.
in my configuration.json
"ReRoutes": [
{
"DownstreamPathTemplate": "/api/{customerid}/project/{id}",
"DownstreamScheme": "http",
"DownstreamPort": 5000,
"DownstreamHost": "localhost",
"UpstreamPathTemplate": "/{customerid}/project/{id}",
"UpstreamHttpMethod": "Delete",
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 3,
"DurationOfBreak": 10,
"TimeoutValue": 5000
}
}]
my request url
HTTP/1.1 DELETE http://apigatway/701/Project/1
my api server get request
HTTP/1.1 DELETE http://localhost:5000/api/701/project/%7Bid%7D
the second parameter {id} value was not find。
Hi Tom, I'm in the process of evaluating Ocelot, looks great, nice work. In my architecture I have several microservices each with their own REST API, some simple, some more complex. I looks like in Ocelot I have to define each HTTP request I want to map, which doesn't feel right and would add quite a lot of development overhead. Traefik for example supports a PathPrefix, can this kind of thing be done with Ocelot or have I misunderstood it's concept?
p.s
Just thinking to go with .net core for my next project and currently in research mode thinking about the architecture, distribution and hosting options, etc.
Hi, I was looking for about a Gateway for .NET for a long time. I´m very happy for found your project. I´m going to study the documentation and try to put in practice.
Do you have plans to continue this project?
Thanks for your time.
Vs2017 has released RC4 release Announcing .NET Core Tools Updates in VS 2017 RC ,NET core tooling also RC, we upgrade to the VS2017 project to develop it?
started working on this probably going to try and use raft consensus to keep cluster up to date.
Like the appveyor pipeline this would kick off on each commit. However, this would just run the Build task - ie, no publishing of packages. The authoritative build pipeline would still be the appveyor pipeline, and this is what would publish the nuget packages and do the releases.
Looks like the cake project tests their osx and linux builds Travis and Bitrise.
I deployed an instance of Ocelot under centos, starting with the following configuration:
{
"ReRoutes": [
{
"DownstreamPathTemplate": "/api/values",
"DownstreamScheme": "http",
"DownstreamHost": "10.125.32.121",
"DownstreamPort": 5002,
"UpstreamPathTemplate": "/api/values",
"UpstreamHttpMethod": "get",
"AdditionalRequestHeaderOptions": {
"EnableAdditionRequestHeader": true,
"RequestHeaders": [
{
"Key": "IP",
"Data": "127.0.0.1"
}
]
},
"AuthenticationOptions": {
"Provider": "IdentityServer",
"ProviderRootUrl": "http://10.125.32.121:5000",
"RequireHttps": false,
"ApiName": "api1",
"AllowedScopes": [
"openid",
"Profile",
"offline_access"
],
"ApiSecret": "myApiSecret"
}
}
],
"GlobalConfiguration": {
"RequestIdKey": "RequestId",
"AdministrationPath": "/admin",
"IdentityServerRootUrl": "http://10.125.32.121:5000"
}
}
Then I added some configuration as follows:
{
"ReRoutes": [
{
"DownstreamPathTemplate": "/api/values",
"DownstreamScheme": "http",
"DownstreamHost": "10.125.32.121",
"DownstreamPort": 5002,
"UpstreamPathTemplate": "/api/values",
"UpstreamHttpMethod": "get",
"AdditionalRequestHeaderOptions": {
"EnableAdditionRequestHeader": true,
"RequestHeaders": [
{
"Key": "IP",
"Data": "127.0.0.1"
}
]
},
"AuthenticationOptions": {
"Provider": "IdentityServer",
"ProviderRootUrl": "http://10.125.32.121:5000",
"RequireHttps": false,
"ApiName": "api1",
"AllowedScopes": [
"openid",
"Profile",
"offline_access"
],
"ApiSecret": "myApiSecret"
}
},
{
"DownstreamPathTemplate": "/api/FlowEngine/StartFlow",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/StartFlow",
"UpstreamHttpMethod": "post"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/SubmitTask",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/SubmitTask",
"UpstreamHttpMethod": "post"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/TerminateInstance",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/TerminateInstance",
"UpstreamHttpMethod": "post"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/DeliverToStaff",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/DeliverToStaff",
"UpstreamHttpMethod": "post"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/RollbackToActivity",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/RollbackToActivity",
"UpstreamHttpMethod": "post"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/GetTaskItems",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/GetTaskItems",
"UpstreamHttpMethod": "get"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/GetFinishItems",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/GetFinishItems",
"UpstreamHttpMethod": "get"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/GetProcessToken",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/GetProcessToken",
"UpstreamHttpMethod": "get"
},
{
"DownstreamPathTemplate": "/api/FlowEngine/GetInstanceById",
"DownstreamScheme": "http",
"DownstreamHost": "10.123.10.44",
"DownstreamPort": 21164,
"UpstreamPathTemplate": "/api/FlowEngine/GetInstanceById",
"UpstreamHttpMethod": "get"
}
],
"GlobalConfiguration": {
"RequestIdKey": "RequestId",
"AdministrationPath": "/admin",
"IdentityServerRootUrl": "http://10.125.32.121:5000"
}
}
Often a strange thing is that during the operation of Configuration.json into the previous file content
{"ReRoutes":[{"DownstreamPathTemplate":"/api/values","UpstreamPathTemplate":"/api/values","UpstreamHttpMethod":"get","AuthenticationOptions":{"Provider":"IdentityServer","ProviderRootUrl":"http://10.125.32.121:5000","ApiName":"api1","RequireHttps":false,"AllowedScopes":["openid","Profile","offline_access"],"ApiSecret":"myApiSecret"},"AddHeadersToRequest":{},"AddClaimsToRequest":{},"RouteClaimsRequirement":{},"AddQueriesToRequest":{},"RequestIdKey":"","FileCacheOptions":{"TtlSeconds":0},"ReRouteIsCaseSensitive":false,"ServiceName":"","DownstreamScheme":"http","DownstreamHost":"10.125.32.121","DownstreamPort":5002,"QoSOptions":{"ExceptionsAllowedBeforeBreaking":0,"DurationOfBreak":0,"TimeoutValue":0},"LoadBalancer":"","RateLimitOptions":{"ClientWhitelist":[],"EnableRateLimiting":false,"Period":"","PeriodTimespan":0.0,"Limit":0},"AdditionalRequestHeaderOptions":{"EnableAdditionRequestHeader":true,"RequestHeaders":[{"Key":"IP","Data":"127.0.0.1"}]}}],"GlobalConfiguration":{"RequestIdKey":"RequestId","ServiceDiscoveryProvider":{"Provider":"","Host":"","Port":0},"AdministrationPath":"/admin","IdentityServerRootUrl":"http://10.125.32.121:5000","RateLimitOptions":{"ClientIdHeader":"ClientId","QuotaExceededMessage":"","RateLimitCounterPrefix":"ocelot","DisableRateLimitHeaders":false,"HttpStatusCode":429}}}
This content should be FileConfigurationRepository written to disk
AppContext.BaseDirectory is not consistent in different systems,I found in the coreclr github several issues, the state is open :
https://github.com/dotnet/coreclr/issues/2128
There's a number of improvements we can make to the build process...
I have this kind of stuff working on another project, based on how the folks at BDDfy do it as documented at http://www.michael-whelan.net/continuous-delivery-github-cake-gittools-appveyor/ - shouldn't be too much effort to migrate the current bat scripts.
I need to add a combination of Consul (https://www.consul.io/)to Ocelot service discovery functions, Ocelot via the specific services that are currently available from Consul services and then call the service, I'm trying to implement this functionality, mainly DownstreamTemplate is dynamically modified, how the configuration item and better
Think the problem is that cake cannot reference variables declared at top of page in private methods on macos...no idea why it can on windows!
I have a code fork, watching an article, code, I did not continue to update the last month, is used in the project. NET core 1.0, many components already RTM, such as IdentityServer4.AccessTokenValidation, next step is what I expect to be with you contributions to this project
At the moment if you want to update ocelot configuration you need to edit configuration.json and restart the service. I guess if you are deploying Ocelot with docker containers updating one and throwing away the others is a viable strategy but where I work we would probably not do this.
I would like it so that you can update the configuration with no downtime. Probably going to add an API that be accessed over HTTP to do this.
I imagine the API would ultimately update configuration.json and reload the static config in memory. It will probably be fine to create the updated version in memory, lock the existing one, update it and release the lock. Should be pretty fast and not cause anyone problems.
Clients could be a CLI tool, web app etc
Concerns are mainly authentication / authorisation. Also user would need to specific some kind of api path that they would never map any routes to..in app.UseOcelot we could do a builder.Map("/someadminroute", etc) this means any requests that have the route in would go to the api.
There could be a section in configuration.json that says who can access endpoints and how they are authenticated. I really don't want to write my own authentication / authorisation code however would ideally like to do this with no dependencies which is going to make it hard. When I say dependencies I mean SQL Server for storing users etc.
Could potentially make Ocelot use IdenityServer and bearer tokens and make identity server look at configuration.json for users, claims, scopes etc. Even simpler when bootstrapping identity server you could use in memory implementation and load everything out of configuration.json. Problem then is if you expose endpoints to change authenticated users / authorization stuff how would you rebootstrap identity server.
After this is working for one deployed Ocelot instance the next step is some kind of clustering that keeps the config up to date among all Ocelots!
Hi,
Thank you for this great project. We started to look at .Net based API Getaway frameworks and it looks like yours is what we need.
According to API Getaway pattern purpose, there might be need to aggregate some downstreams into one upstream. For example, for mobile applications you can combine two downstreams like api.yourcompany.com/posts/{id}
and api.yourcompany.com/comments/{id}
under one upstream getaway.yourcompany.com/mobile/posts/{id}
.
What do you think about this feature? Are there any plans to implement it?
Hi Tom,
Congrats and thanks for your good initiative.
IMHO, As the intention is to provide a gateway for MicroServices we shall split the reroute configuration.
New MicroService level configuration or Downstream host level configuration, here we shall have the Authentication part. we use identity server to protect our client API, Instead of specifying this for each route we shall specify at top level.
A wild card mapping will be a lot interesting as well.
Waiting to hear from you, will send a pull request soon. Cheers
and maybe an acceptance test
Forwarding whether support of gRPC ?
Hi,
congratulations for this project!!
I did a little project with ocelot and two apis and all works fine in development, but when I publish the solution in production (with IIS) and I access to some reroute always return 404 not found. My api+ocelot has the default ValuesController (that creates Visual Studio by default) and works fine but the reroutes didn´t work in production.
It´s neccesary some special kind of configuration in the server?
Thanks.
AppMetrics to achieve performance data acquisition https://github.com/geffzhang/AppMetrics
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.