brightercommand / darker

The query-side counterpart of Brighter

Darker

The query-side counterpart of Brighter.

Usage with ASP.NET Core

In your ConfigureServices method, use AddDarker to add Darker to the container. ASP.NET Core integration is provided by the Paramore.Darker.AspNetCore package.

// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
    // Add Darker and some extensions.
    services.AddDarker()
        .AddHandlersFromAssemblies(typeof(GetPeopleQueryHandler).Assembly)
        .AddJsonQueryLogging()
        .AddDefaultPolicies();

    // Add framework services.
    services.AddMvc();
}

WARNING if you are using EFCore the DBContext DI Lifetime is scoped, for Darker to play nicely with EFCore and DI the QueryProcessor must also be registration as Scoped

 services.AddDarker(options =>
                {
                    //EFCore by default registers Context as scoped, which forces the QueryProcessorLifetime to also be scoped
                    options.QueryProcessorLifetime = ServiceLifetime.Scoped;
                })

This example uses the request logging integration provided by Paramore.Darker.QueryLogging and policy integration provided by Paramore.Darker.Policies. Have a look at the Startup.ConfigureServices method in the SampleApi project for more examples on how to use the integrations.

Inject IQueryProcessor and call Execute or ExecuteAsync to dispatch your query to the registered query handler.

using Paramore.Darker;
using Microsoft.AspNetCore.Mvc;
using System.Threading;
using System.Threading.Tasks;

public class FooController : ControllerBase
{
    private readonly IQueryProcessor _queryProcessor;

    public FooController(IQueryProcessor queryProcessor)
    {
        _queryProcessor = queryProcessor;
    }

    public async Task<IActionResult> Get(CancellationToken cancellationToken = default(CancellationToken))
    {
        var query = new GetFoo(42);
        var result = await _queryProcessor.ExecuteAsync(query, cancellationToken);
        return Ok(result);
    }
}

using Paramore.Darker;

public sealed class GetFoo : IQuery<string>
{
    public int Number { get; }

    public GetFoo(int number)
    {
        Number = number;
    }
}

Implement either QueryHandler<,> or QueryHandlerAsync<,> depending on whether you wish to execute your queries synchronously or asynchronously. For most control, you can also implement IQueryHandler<,> directly.

using Paramore.Darker;
using Paramore.Darker.Attributes;
using Paramore.Darker.Policies;
using Paramore.Darker.QueryLogging;
using System.Threading;
using System.Threading.Tasks;

public sealed class GetFooHandler : QueryHandlerAsync<GetFoo, string>
{
    [QueryLogging(1)]
    [FallbackPolicy(2)]
    [RetryableQuery(3)]
    public override async Task<string> ExecuteAsync(GetFoo query, CancellationToken cancellationToken = default(CancellationToken))
    {
        return await FetchFooForNumber(query.Number, cancellationToken);
    }
}

Usage without ASP.NET

Register your queries and handlers with QueryHandlerRegistry and use QueryProcessorBuilder to configure and build a IQueryProcessor.

var registry = new QueryHandlerRegistry();
registry.Register<GetFoo, string, GetFooHandler>();

IQueryProcessor queryProcessor = QueryProcessorBuilder.With()
    .Handlers(registry, Activator.CreateInstance, t => {}, Activator.CreateInstance)
    .InMemoryQueryContextFactory()
    .Build();

Instead of Activator.CreateInstance, you can pass any factory Func<Type, object> to constuct handlers and decorator. Integrations with some DI frameworks are available, for example SimpleInjector, as provided by the Paramore.Darker.SimpleInjector package:

var container = new Container();

var queryProcessor = QueryProcessorBuilder.With()
    .SimpleInjectoHandlers(container, opts =>
        opts.WithQueriesAndHandlersFromAssembly(typeof(GetPeopleQuery).Assembly))
    .InMemoryQueryContextFactory()
    .Build();

container.Register<IQueryProcessor>(queryProcessor);

In this case you don't need to manually register queries or handlers as the integration allows scanning assemblies for matching types.

darker's Issues

Memory/EF issues with the way queries are created

We've been getting an issue where EF Core logs the following: More than twenty 'IServiceProvider' instances have been created for internal use by Entity Framework. This is commonly caused by injection of a new singleton service instance into every DbContext instance. . We also noticed that after running for a short period of time the process was taking 6+Gb of RAM on our server.

We think we tracked it down to the fix for issue #9, i.e. this commit: 2a07b68

Basically, it seems that creating all those service providers is causing EF Core issues.

Our fix works for us, although it may not work in every scenario.

Initially, what we did was in the ServiceCollectionExtensions.AddDarker method we changed the services.AddSingleton<IQueryProcessor> line to:

services.AddSingleton<IQueryProcessor>(sp =>
{
  factory.SetProvider(sp);
  var queryProcessor = builder.Build();
  return queryProcessor;
});

And added a SetProvider() method to the AspNetHandlerFactory class, which we had to extract and rename because it was internal.

public sealed class AcHandlerFactory : IQueryHandlerFactory, IQueryHandlerDecoratorFactory
{
  private IServiceProvider _services;
  public void SetProvider(IServiceProvider serviceProvider)
  {
    if (_services != null)
      throw new InvalidOperationException($"A service provider is already assigned to the {nameof(AcHandlerFactory)}.");
    _services = serviceProvider ?? throw new ArgumentNullException(nameof(serviceProvider));
  }

  IQueryHandler IQueryHandlerFactory.Create(Type handlerType)
  {
    if (_services == null)
      throw new InvalidOperationException($"A service provider must be assigned to the {nameof(AcHandlerFactory)} in advance of its use.");

    var result = (IQueryHandler)_services.GetService(handlerType);
    return result;
  }
// ... other things in the class
}

That fixed the initial issue... But since the service provider passed to the Singleton's new factory function isn't scoped, anything created that is scoped, say, something working inside a web request, causes the DI framework to break when calling GetService() on the IServiceProvider.

So.... The next step was to ensure those didn't break either. So, I now create a new scope for every query (we've done this kind of thing in Brighter before, and works well in back end applications such as windows services where there is no natural scope for a single piece of work)

That changes the handler factory to this:

   public sealed class AcHandlerFactory : IQueryHandlerFactory, IQueryHandlerDecoratorFactory
    {
        private IServiceProvider _services;
        private readonly Dictionary<IQueryHandler, IServiceScope> _scopes = new Dictionary<IQueryHandler, IServiceScope>();

         public AcHandlerFactory()
        {
        }

         public void SetProvider(IServiceProvider serviceProvider)
        {
            if (_services != null)
                throw new InvalidOperationException($"A service provider is already assigned to the {nameof(AcHandlerFactory)}.");
            _services = serviceProvider ?? throw new ArgumentNullException(nameof(serviceProvider));
        }

         IQueryHandler IQueryHandlerFactory.Create(Type handlerType)
        {
            if (_services == null)
                throw new InvalidOperationException($"A service provider must be assigned to the {nameof(AcHandlerFactory)} in advance of its use.");

             var scope = _services.CreateScope();
            var result = (IQueryHandler)scope.ServiceProvider.GetService(handlerType);
            _scopes.Add(result, scope);
            return result;
        }

         void IQueryHandlerFactory.Release(IQueryHandler handler)
        {
            if (_scopes.TryGetValue(handler, out var scope))
            {
                scope.Dispose();
                _scopes.Remove(handler);
            }
        }

         T IQueryHandlerDecoratorFactory.Create<T>(Type decoratorType)
        {
            return (T)_services.GetService(decoratorType);
        }

         void IQueryHandlerDecoratorFactory.Release<T>(T handler)
        {
            // no op
        }
    }

The decorator part is not scoped here, but it may be useful to do that too.

I think the handler factory here needs to be extended to provide for a wider array of use cases (or have a way to pick a factory based on a specific use case). Instantiating a new service provider on each query appears to be hard on memory, at least when coupled with EF Core.

Paramore.Darker.SimpleInjector plugin

Followed description:

var container = new Container();

var queryProcessor = QueryProcessorBuilder.With()
    .SimpleInjectoHandlers(container, opts =>
        opts.WithQueriesAndHandlersFromAssembly(typeof(GetPeopleQuery).Assembly))
    .InMemoryQueryContextFactory()
    .Build();

container.Register<IQueryProcessor>(queryProcessor);

Expected result: Working
Actual result: Argument 1: cannot convert from 'Paramore.Darker.IQueryProcessor' to 'SimpleInjector.Lifestyle'

Pls, advice!

Scoped dependencies not being disposed when Query Handlers are created via the Query Processor

I have an ASP.NET Core application which is using Darker to run database queries in conjunction with EF Core. To do this, I am injecting a scoped service into my Query Handlers. When my handlers are created via the Query Processor, I have noticed that the Dispose() method on the scoped service is not called after the handler has completed its work. Also, the scoped service instance that is injected into the Query Handlers is not up-to-date, causing my queries to return incorrect results.

These problems do not occur if I inject the Query Handler instances directly into my controller and bypass the Query Processor. When I do this, my scoped services are disposed of correctly and my handler instances receive an up-to-date reference to the scoped service.

This is how I am configuring my container (taken from Startup.cs)

public void ConfigureServices(IServiceCollection services)
        {
            services.AddDbContext<UserManagementContext>(options =>
                options.UseSqlServer(Configuration.GetConnectionString("DefaultConnection")));

            services.AddScoped<IDataSession, EntityDataSession>();
            services.AddScoped<DbContext>(provider => provider.GetService<UserManagementContext>());

            services.AddDarker()
                .AddHandlersFromAssemblies(typeof(GetUsersQuery).Assembly);

            services.AddBrighter()
                    .AsyncHandlersFromAssemblies(typeof(SaveUserCommand).Assembly);

            services.AddMvc();
        }

My Query Handlers depend upon IDataSession, which in turn depends upon DbContext.

Please can you advise how I can use scoped dependencies in Query Handlers that are created via the Query Processor?

Documentation for Darker is sorely lacking (in contrast to Brighter, pun intended)

Although Darker is significantly simpler that Brighter some of its concepts are not that obvious or self-explanatory.

I have troubles especially with the policies. Could you at least document the default ones?

When I create a query and handler and register Darker exactly like you in the README, the handler never gets called. I can remedy that by removing [RetryableQuery(3)] from the ExecuteAsync method. I can't tell why it is not working or if I lose something by commenting it out (given that I use the default policies).

Objects injected into Query Handlers are not disposed at the end of the query lifecycle

Request
The IQueryHandlerFactory has no similar mechanism to Brighter to dispose of resources that may have been created during the Create method.

Precedent
Brighter's IAmAHandlerFactoryAsync has a Create() and Release() method which allows resources to be disposed at the end of the command processing that may have been created to support the command handler.

Context
In the context of, say, an MVC application this may not be important as many IoC containers will provide their own mechanism to have object lifecylces mirror the request/response lifecycle and will dispose of anything used by a command/query handler used within that context, other applications may not have this mechanism. In those cases it would be up to the the handler factory to dispose-of/release the object at the end of the query lifecycle.

NullReference in PipelineBuilder.Dispose

Hi,
We are seeing this but I notice you have recently fixed this in commit ea40ba7.
Do you have an ETA on a release to nuget?

Many thanks
Cheers
David

Use the IPolicyRegistry from Polly instead of Darkers homegrown one

Now that Polly has a policy registry built-in, it would make sense to use this rather than the homegrown one Darker currently uses.
I'm happy to do this if wanted as I'm looking in to transitioning a project from a homegrown CQRS implementation to at least darker on the query side.

Query classes should not have to be public

When I modify the sample code in the readme to remove the public qualifier on the GetFoo query, I get the following error:

Unhandled Exception: System.AggregateException: One or more errors occurred. ('object' does not contain a definition for 'ExecuteAsync') ---> Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: 'object' does not contain a definition for 'ExecuteAsync'
   at CallSite.Target(Closure , CallSite , Object , Object , CancellationToken )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet](CallSite site, T0 arg0, T1 arg1, T2 arg2)
   at Paramore.Darker.PipelineBuilder`1.<BuildAsync>b__10_0(IQuery`1 r, CancellationToken ct)
   at Paramore.Darker.QueryProcessor.<ExecuteAsync>d__8`1.MoveNext()

However, in brighter, there is no requirement that commands are public.

IAsyncEnumerable support

Are there any plans to add support for IAsyncEnumerable (and is that even necessarily sensible to use with this library)?

Fallback Policy Decorator not registered after upgrade to 2.0.63

After upgrading to 2.0.63, the FallbackPolicyDecorator does not seem to be registered automatically. We now need to register it manually by adding it via assembly and explicitly registering the decorator type in Startup.cs, like this:

// ...
services.AddDarker()
    .AddHandlersFromAssemblies(typeof(Paramore.Darker.Decorators.FallbackPolicyDecorator<,>).Assembly)
    .RegisterDecorator(typeof(Paramore.Darker.Decorators.FallbackPolicyDecorator<,>))
// ...

It looks like the changes were mentioned here in issue #65.
Is this going to be necessary going forward or is it a bug that will be fixed?
Thanks.

Consider merging policy and logging into main assembly

The problem with the separation of policy and logging is that most of our examples show Darker being used successfully with logging and policy attributes, but because these are not brought into scope unless we add these packages separately it can be confusing to the new user how to achieve the basic scenario.

Packaging principles would say that things that 'the unit of re-use is the unit of release', and 'keep things that change together, together'. I don't see real advantage or likelihood of these packages being released on a different schedule, so separate packages actually make this harder.

Brighter uses a single package for all these requirements, so the difference is a little confusing.

In addition ASP.NET Core has moved away from this kind of 'fine grained' package model as well.

Support for remote queries running on AWS Lambda or Azure Functions

This is pretty much just in idea/draft/gathering requirements stage.

It would be cool if we could configure queries such that when they are executed, they actually call an Azure or a Lambda function. This would basically be RPC over HTTP, but I think that's quite reasonable for the Darker (= query side) use case.

I'll just go ahead and sketch some stuff out. Any input is appreciated!

Is IQueryHandlerFactory.Release really needed?

Given all implementations of IQueryHandlerFactory.Release are no-ops, is this method really needed?
If it isn't, it would simplify the PipelineBuilder class as it wouldn't need to implement IDisposable.

Darker is nearly unusable for regular .NET Framework code

For a simple assembly (.NET 4.6.1) defining some domain contracts I'm using Paramore.Brighter. Building from a clean start this leads to just 4 assemblies, my own, brighter and the two depents libraries Polly and NewtonSoft.Json

Just by adding Darker 1.0.1. the number of assemblies increases to 101 (one-hundred-and-one). And although I probably will never need to manage any of these by hand, it creates HUGE amounts of binding redirects in the config files.

:(

Review logging for 'noise'

Darker may be a little noisy when logging. Brighter can be, but we tend to review and @holytshirt and I fight about what should be logged at a lower or higher level until we get some sort of consensus.

The trade-off is between diagnostically useful, and noise.

We should log something as WARN because it could become or trigger a fault
We should log something as ERROR because it is a fault, although we may take action in response.
We should log something as FATAL because something bad happened and we intend to fail fast. We rarely do this, because we don't control lifetime. Generally we ought to raise error and assume the host application decides our fault was fatal

We should INFO carefully. We used it a lot in the past for tracing but it made us noisy. We should be using it to help developers trace execution, indicating beginning and end of pipelines etc. . This level should help help folks see flow, but without the additional debug information If we have detail we should probably be DEBUG.

System.ValueTuple Causing Issues in .NET v4.61 and above

This is likely an issue with System.ValueTuple and/or its implementation with .NET Standard v2, but its causing issues when using Darker. Would you consider removing the dependency on System.ValueTuple in Paramore.Darker.PipelineBuilder.ResolveHandler or at least until the System.ValueTuple issue gets resolved?

I'd be happy to put in a pull request for this if it would be considered.

How to do paging in Darker?

Hello,
Could you please provide some information about how to do paging using Darker?
I have been using Brighter for commands and Darker for queries for some years now, and now I would like to add paging support to my application for queries,

Return only the last item from the pipeline

In the current implementation of PipelineBuilder, the Build and BuildAsync methods return the full pipeline. Is this really necessary, given the query processor then invokes the pipeline.Last method?

Could we get away with returning on the last func?

Add a decorator to allow the use of Pollies Cache policy

Given that Polly now has other available policies (cache for example) would it be desirable to add an equivalent to the brighter UsePolicy attribute and just use that rather than having Retry and an attribute for each policy type?