aseovic / helidon Goto Github PK
View Code? Open in Web Editor NEWThis project forked from helidon-io/helidon
Java libraries for writing microservices
Home Page: https://helidon.io
License: Apache License 2.0
This project forked from helidon-io/helidon
Java libraries for writing microservices
Home Page: https://helidon.io
License: Apache License 2.0
At the moment we only support check
method within our HealthService
implementation.
We need to add support for other methods inherited from HealthBase
, in order to implement full gRPC Health Service spec.
We need to be able to use gRPC interceptors in the CDI micro profile environment. The interceptors should be discoverable and applied to services or methods.
The current thinking is that we can use annotation binding for this in the same way that CDI interceptors work. A gRPC service class can be annotated with custom "interceptor binding" annotations. A gRPC interceptor is also annotated with the same annotation. When the server is bootstrapped and services are discovered the interceptors for the specific annotations should be discovered and applied to the service/method configuration.
There are standard annotations for micro profile metrics that the current Helidon CDI code creates interceptors for. This means that any gRPC service method annotated with one of these annotations will have metrics applied. The issue here is that using CDI interceptors to apply metrics does not really work for asynchronous methods such as gRPC methods (as an aside it would not work for async Jersey JAX-RS methods either) because the metrics will only wrap the actual method call and not the time between the start of the method and sending the async result.
We need to decide whether to do something different for gRPC. This might be difficult given how the rest of the Helidon MP code processes these annotations. We might have to use a different set of annotations for gRPC metrics.
The gRPC security examples use a custom CallCredentials
implementation BasicAuthCallCredentials
to configure basic auth for the client. As the comment in the PR to oracle/helidon says, this should br removed as it is not actually required. The clients should use a properly created and configured GrpcClientSecurity
instance.
We should modify the existing Tracing documentation to:
Tracer
creation independentlyTracer
created in 1 above is used to configure both WebServer and GrpcServer.The gRPC client module's ClientServiceDescriptor
and ClientMethodDescriptor
classes both have an inner interface called Config
. This inner interface's name needs to be changed to Rules
.
ClientServiceDescriptor.Config
---> ClientServiceDescriptor.Rules
ClientMethodDescriptor.Config
---> ClientMethodDescriptor.Rules
NOTE: For completeness (and tidiness and elegance) make sure that you change all of the relevant field and parameter names where these classes are used from config
to rules
.
This work should be done on a branch and merged into the grpc-client
branch.
We need to write gRPC Clients
doc section, with following sub-sections:
The current code base uses an interface to define a priority client or server interceptor. This should be changed so that priority is based on an int and priority of a specific interceptor can be defined by annotating the class with @Priority
or setting the priority when adding the interceptor.
In addition to the gRPC-generated clients that we currently use in examples (GreetClient, StringClient, HealthClient) we should implement the equivalents using our client framework.
We should also implement a client for the GreetServiceJava
, which would demonstrate how to use Java serialization between the client and the server.
Once the implementation is complete, we need to document the usage of the client-side Helidon MP gRPC framework, including:
Once #6 is completed, we should add support for service and method-level metrics configuration.
It should be possible to add custom annotations to server side service classes and methods and have those annotations processed to add functionality to the service. This is similar to how the Jersey "Feature" plugin mechanism works. For example a SecurityFeature
might process the @RolesAllowed
annotation to add security configuration to a service.
This issue is related to the MP gRPC server issue #27 and depends on the annotation support issue #52.
GrpcRouting
.GrpcRouting
regardless of which order services and features are registered.GrpcRouting
GrpcRouting
GrpcRouting
.The web server security examples include examples on how to use outbound security, the same features are required for outbound gRPC calls. Depends on #26 (allow configuration of client security)
The use case is that an authenticated request is made to a Helidon http endpoint and in the code that handles that endpoint another request is made to an external endpoint. This external endpoint requires authentication.
There are two scenarios:
Requirements:
Success Criteria:
We should provide a base class that makes it simpler to implement clients, along the lines of what we did in Synoki.
Tasks:
helidon-grpc-server
into helidon-grpc-client
renaming them appropriately.ServiceDescriptor
--> ClientServiceDescriptor
MethodDescriptor
--> ClientMethodDescriptor
In both cases above change where these classes use io.grpc.ServerInterceptor
to use io.grpc.ClientInterceptor
This will then give us the ability to define/build service definitions on the client.
io.synoki.grpc.client.AbstractGrpcClient
from Synoki gRPC client module https://gitlab-odx.oracledx.com/synoki/synoki-grpc
.Helidon gRPC has no concept of Serilizer
so this should be removed from io.synoki.grpc.client.AbstractGrpcClient
The m_methods
field can be replaced with a ClientServiceDescriptor
field as that is what we will use in the Helidon client to describe a service. This field can be set from the constructor, i.e. the new AbstractGrpcClient
should take a ClientServiceDescriptor
as its constructor parameter.
When complete we should be able to create an instance of the new AbstractGrpcClient
for a given ClientServiceDescriptor
and then use methods such as AbstractGrpcClient.blockingUnaryCall
to call methods. We will likely need to add a method name parameter to methods like AbstractGrpcClient.blockingUnaryCall
so that we can locate the relevant io.grpc.MethodDescriptor
from the ClientServiceDescriptor
field.
Allow users to configure security and pass necessary authN/authZ credentials to the server.
When #20 is complete we will have ClientSeviceDefinition
and ClientMethodDefinition
classes. We can add methods to these classes to set the CallCredentials
to use at the service or method level which will be applied in addition to any CallCredentials
set at the Channel
level. The CallCredentials
should be applied in the order channel followed by service followed by method so that each level can add to or override previously applied credentials.
CallCredentials
set at either the service or method level should apply the CallCredentials
set for the Channel
.CallCredentials
set at the service level should apply the service's CallCredentials
in addition to any CallCredentials
set for the Channel
.CallCredentials
set at the service level should apply the service's CallCredentials
and be able to override any CallCredentials
set for the Channel
. For example if the Channel
sets a basic auth header it should be possible for the services CallCredentials
to override that vale with a different basic auth header.CallCredentials
set at the method level should apply the method's CallCredentials
in addition to the service's CallCredentials
and in addition to any CallCredentials
set for the Channel
.CallCredentials
set at the method level should apply the method's CallCredentials
and override the service's and channel's CallCredentials
. For example if the service sets a basic auth header it should be possible for the channel's CallCredentials
to override that vale with a different basic auth header.The Helidon web server supports ABAC (attribute based access control) but the gRPC server does not currently supply enough information in the security interceptor to make this work.
A lot of the examples of ABAC work using Jersey and annotations but the gRPC server cannot yet support creating a service from annotated classes and methods.
ABAC can also be configured in configuration so this should be supported by the gRPC server so that ABAC rules can be configured globally, for services and, for methods.
See the web server examples under examples/grpc/security-abac
Success Criteria:
The MP version of the gRPC server should support discovery and configuration of gRPC health checks.
A discoverable health check would be one of:
HealthCheck
@Health
that returns a HealthCheckResponse
.These changes should go into the ServiceModeller
class that builds a ServiceDescriptor
from an annotated class. In both of the above cases the instance of the service to use to call the health check method will be provided by the instance Supplier
obtains from the ServiceModeller.instanceSupplier()
method. Some sort of wrapper health check implementation will be required:
public class WrapperHealthCheck
implements HealthCheck {
private final Supplier<?> supplier;
private final Method method;
... constructor ...
public HealthCheckResponse call() {
Object instance = supplier.get();
if (instance instances HealthCheck) {
return ((HealthCheck) instance).call():
}
return (HelathCheckResponse) method.invoke(supplier.get());
}
}
Completion Criteria:
@ ApplicationScoped
and implements HealthCheck
so that when the ServiceDescriptor
created by the ServiceModeller
has the service instance as the health check:@ApplicationScoped
public class MyService
implements HealthCheck {
}
@ ApplicationScoped
and has a method annotated with @Health
that returns a HealthCheckResponse
so that when the ServiceDescriptor
created by the ServiceModeller
uses a call to that method as the health check:@ApplicationScoped
public class MyService {
@ Health
public HealthCheckResponse health() {
... do check ...
}
}
We need to change documentation to make it clear that the gRPC server doesn't have to be started in order to get all service health checks from it and add them to HealthSupport
.
The current documentation example and comments imply that it does.
We should allow users to create a client-side proxy for a gRPC service for a given channel and client-side interface.
The users should be able to inject client-side proxies for a gRPC service using named channel and the service interface declared at the injection point:
@Inject
@Channel("myGrpcServer")
private MyServiceClient client;
The above should end up injecting the result of
GrpcClientHelper.proxy(myGrpcServerChannel, MyServiceClient.class)
Allow tracing to be configured on the client and propagate tracing span information from the client to the server.
Document outbound security: (depends on #29)
Currently docs/src/main/docs/security/03_containers-integration.adoc
only shows outbound security for a web request method calling out to another http endpoint. This documentation needs to be enhanced to show gRPC outbound examples:
We need to look deeper into a number of areas before we can implement Helidon MP, such as:
Once this exploratory work is completed, we will be able to better define the remaining work.
Document how to create secure gRPC services.
Include the following:
Implement GrpcClientConfiguration
, which will allow us to configure the client the same way we configure the server using GrpcServerConfiguration
.
Add the ability to create a server side gRPC service from annotated classes and interfaces in the same way that Jersey does for web services.
This work relates to the MP server issue #27
The gRPC server should be configurable to be secured with SSL.
The Helidon web server SocketConfiguration
contains code to configure SSL so this code can copied to configured SSL in GrpcServerConfiguration
the same way and then using the web server SSLContextBuilder
to build an SSLContext
from the config. This can be used using pretty much the same code that the web server uses to create a Netty JdkSslContext
(see the Helidon NettyWebServer
class) to pass to the NettyServerBuilder
when starting the gRPC server.
See the web server SslTest
class for examples of how to create a SSLContext
for testing.
GrpcServerConfiguration
and configure SSL. For example:SSLContext sslContext = ...
GrpcServerConfiguration.build()
.ssl(sslContext)
.build();
conf-ssl.conf
might look like this:grpcserver {
ssl: {
private-key: {
keystore-resource-path: "certificate.p12"
keystore-passphrase: "helidon"
}
}
then create the configuration like this:
Config config = Config.builder().sources(ConfigSources.classpath("config1.conf")).build();
GrpcServerConfiguration serverConfig = config.get("grpcserver").as(GrpcServerConfiguration::create).get();
For examples of how to test this the Synoki gRPC project has SSL tests that create an SSL enabled client in io.synoki.grpc.server.TlsIT
We need to refactor builder API to use ServiceDescriptor
and MethodDescriptor
, which should capture all relevant information necessary to fully configure each service method.
ServiceDescriptor
MethodDescriptor
The Webserver examples modules have examples of how to perform secure outbound calls to other http servers. We need to add similar examples for gRPC.
Note: This depends on #20 so that we can use the client API in the examples.
There are four scenarios: (see the io.helidon.security.integration.grpc.OutboundSecurityIT
functional test for code that sets up the outbound security).
The request and response types for a method are optional and should not be in the ClientMethodDescriptor
constructor. They need to be moved to the builder in the same way that the server descriptor was refactored.
The gRPC client module should not have a dependency on metrics (metrics will be in a new stand-alone module).
Once this is complete it should be possible to remove this dependency from helidon-grpc-client:
<dependency>
<groupId>io.helidon.metrics</groupId>
<artifactId>helidon-metrics</artifactId>
<version>${project.version}</version>
</dependency>
Remove the metricType field from both the ClientServiceDescriptor
and ClientMethodDescriptor
classes as well as corresponding builders and tests.
NOTE: this work needs to be done on a branch and merged into the grpc-client
branch.
The gRPC MP server supports creating and configuring services from annotated classes. We need to be able to support using annotations to configure security in the same way that the Helidon web server does, for example @RolesAllowed
, @Authenticated
, etc...
The existing web server code already does a lot of annotation work. We need to refactor this code to produce a new security/annotation-support
module that contains the common code shared between the web server and the gRPC server.
The Helidon team want to use yaml as the format for showing examples of configuration in documentation files.
Go through all gRPC documentation under /docs
and README.md
files in the various modules (including examples) and make sure configuration is in the correct format.
This work needs to be done on a branch that will be merged into master
so that it updated the initial pull request to Oracle/Helidon.
Security in gRPC cannot be created from configuration - this was initially removed when porting over code from the web server and now needs to be put back.
Success Criteria:
GrpcSecurity
interceptor to a GrpcRouting
by creating an instance just from configuration, i.e. GrpcSecurity.create(config.get("security"))
security {
grpc-server: {
# Configuration of integration with grpc server
# The default configuration to apply to all services not explicitly configured below
defaults {
authenticate = true
}
}
}
security {
grpc-server: {
services: [
{
name: "FooService"
defaults {
roles-allowed = ["admin"]
}
},
{
name: "BarService"
defaults {
roles-allowed = ["admin"]
}
}
]
}
}
It should be possible to define method level security in configuration, for example:
security {
grpc-server: {
services: [
{
name: "FooService"
defaults {
roles-allowed = ["admin"]
}
methods: [
{
name: "DoSomething"
roles-allowed = ["admin"]
},
{
name: "DoSomethingElse"
roles-allowed = ["user"]
}
]
}
]
}
}
We need to remove @author
tags from the Javadoc comments throughout the codebase, in order to pass the latest chekstyle rules and be able to merge into oracle/helidon
.
The gRPC server module's ServiceDescriptor
and MethodDescriptor
classes both have an inner interface called Config
. This inner interface's name needs to be changed to Rules
.
ServiceDescriptor.Config
---> ServiceDescriptor.Rules
MethodDescriptor.Config
---> MethodDescriptor.Rules
NOTE: For completeness (and tidiness and elegance) make sure that you change all of the relevant field and parameter names where these classes are used from config
to rules
.
This work needs to be done on a branch that will be merged into master so that it updated the initial pull request to Oracle/Helidon.
We should clean up and refactor GrpcServerConfiguration
to support same usage patterns (builder, etc.) that the existing web server ServerConfiguration
class uses.
In the gRPC code the priority value for both server and client interceptors is an enum. It was commented in the oracle/helidon PR that an enum is too restrictive. This had actually already been realised by us and changed in our grpc-mp
branch. This change needs to be ported over to the grpc-server
branch so that it can be applied to the PR. This change should also then be applied to the grpc-client
branch.
We need to design and implement support for the Helidon MP gRPC server bootstrapping, which will allow us to:
Capture client-side metrics at the service or method level, and make them available for collection/scraping.
We should be able to expose them locally via a co-located WebServer, when the client is a long-running process within the same network as the server.
However, we should also consider remote and/or short-running clients, which may not be accessible for scraping by Prometheus, so we should come up with a solution that pushes metrics from the clients to the gRPC server they are connected to, which can then expose them locally along with its own metrics and make them available for scraping.
Once the implementation is complete, we need to document the usage of the server-side Helidon MP gRPC framework, including:
We need to be able to configure who can execute particular method.
We need to add Reactive gRPC Server
section under the existing Reactive Webserver
section in the Helidon documentation, which will have the following sub-sections:
The metrics support in Helidon gRPC should be separated out of the helidon-grpc-server
module and into a new helidon-grpc-metrics
module.
NOTE: This work needs to be done on a branch that will be merged into master so that it updated the initial pull request to Oracle/Helidon.
Once this is complete it should be possible to remove this dependency from helidon-grpc-server
:
<dependency>
<groupId>io.helidon.metrics</groupId>
<artifactId>helidon-metrics</artifactId>
<version>${project.version}</version>
</dependency>
Currently it is possible to configure metrics for a service like this:
GrpcRouting routing = GrpcRouting.builder()
.register(new EchoService(), rules -> rules.timed())
.build();
After this change metrics for a service will be configured by adding the metrics interceptor:
GrpcRouting routing = GrpcRouting.builder()
.register(new EchoService(), rules -> rules.intercept(GrpcMetrics.timed()))
.build();
The same is true for method level configuration, for example inside a GrpcService.update()
method. For example this:
@Override
public void update(ServiceDescriptor.Config config) {
config.proto(Greet.getDescriptor())
.unary("Greet", this::greet, cfg -> cfg.timed()));
}
will become this:
@Override
public void update(ServiceDescriptor.Config config) {
config.proto(Greet.getDescriptor())
.unary("Greet", this::greet, cfg -> cfg.intercept(GrpcMetrics.timed()));
}
helidon-grpc-metrics
under grpc/metrics
.<dependency>
<groupId>io.helidon.grpc</groupId>
<artifactId>helidon-grpc-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.helidon.metrics</groupId>
<artifactId>helidon-metrics</artifactId>
<version>${project.version}</version>
</dependency>
as well as required test dependencies. It may also have test dependencies on the gRPC server for integration tests:
<dependency>
<groupId>io.helidon.grpc</groupId>
<artifactId>helidon-grpc-server</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
metricType
field from both the ServiceDescriptor
and MethodDescriptor
classes as well as corresponding builders.GrpcMetrics
class and the GrpcMetricsInterceptorIT
and MetricsIT
test classes to the new helidon-grpc-metrics
module.GrpcMetrics
interceptor.docs/grpc
to show the correct way to configure metrics.A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.