farfetch / rules-framework

A generic framework that allows defining and evaluating rules for complex business scenarios.

Rules Framework

Rules.Framework is a generic framework that allows defining and evaluating rules for complex business scenarios.

What is a rule

A rule is a data structure limited in time (date begin and date end), whose content is categorized by a content type. Its applicability is constrained by conditions, and a priority value is used as untie criteria when there are multiple rules applicable.

Why use rules

By using rules, we're able to abstract a multiplicity of business scenarios through rules configurations, instead of heavy code developments. Rules enable a fast response to change and a better control of the business logic by the product owners.

Packages

Name	nuget.org	downloads	fuget.org
Rules.Framework
Rules.Framework.Providers.MongoDB
Rules.Framework.WebUI

Rules.Framework

The Rules.Framework package contains the core of the rules engine. It includes an in-memory provider for the rules data source.

Basic usage

To set up a RulesEngine, define the content types and condition types to be used.

enum ContentType { FreeSample = 1, ShippingCost = 2 }
enum ConditionType { ClientType = 1, Country = 2 }

Build the engine with the RulesEngineBuilder.

var rulesEngine = RulesEngineBuilder.CreateRulesEngine()
    .WithContentType<ContentType>()
    .WithConditionType<ConditionType>()
    .SetInMemoryDataSource()
    .Configure(c => c.PriorityCriteria = PriorityCriterias.TopmostRuleWins)
    .Build();

Use the RuleBuilder to assemble a rule.

var ruleForPremiumFreeSample = RuleBuilder
    .NewRule<ContentType, ConditionType>()
    .WithName("Rule for perfume sample for premium clients.")
    .WithContent(ContentType.FreeSample, "SmallPerfumeSample")
    .WithCondition(ConditionType.ClientType, Operators.Equal, "Premium")
    .WithDateBegin(new DateTime(2020, 01, 01))
    .Build();

Add a rule to the engine with the AddRuleAsync().

rulesEngine.AddRuleAsync(ruleForPremiumFreeSample.Rule, RuleAddPriorityOption.ByPriorityNumber(1));

Get a matching rule by using the MatchOneAsync() and passing a date and conditions.

var matchingRule = rulesEngine.MatchOneAsync(
        ContentType.FreeSample, 
        new DateTime(2021, 12, 25), 
        new[]
        {
            new Condition<ConditionType>(ConditionType.ClientType, "Premium")
        });

Complex scenarios

For a more thorough explanation of the Rules.Framework and all it enables, check the Wiki.

Check also the test scenarios and samples available within the source-code, to see more elaborated examples of its application.

Rules.Framework.Providers.MongoDB

To keep rules persisted in a MongoDB database, use the extension method in the Providers.MongoDB package to pass your MongoClient and MongoDbProviderSettings to the RulesEngineBuilder.

var rulesEngine = RulesEngineBuilder.CreateRulesEngine()
    .SetInMongoDBDataSource(mongoClient, mongoDbProviderSettings)

Rules.Framework.WebUI

The WebUI package offers a way of visualizing the rules in your web service. To configure the UI, pass the rules engine as generic to the IApplicationBuilder extension method provided.

app.UseRulesFrameworkWebUI(rulesEngine.CreateGenericEngine());

Access is done via the endpoint {host}/rules/index.html.

Features

The following list presents features already available and others planned:

• Rules evaluation (match one)
• Rules evaluation (match many)
• Rules content serialization
• Rules management (Create, Read, Update)
• In-memory data source support
• MongoDB data source support
• SQL Server data source support
• Rules data source caching
• Rules data source compilation
• WebUI for rules visualization

Documentation

See the Wiki for full documentation, short examples and other information.

Contributing

Contributions are more than welcome! Submit comments, issues or pull requests, we promise to keep an eye on them :)

Head over to CONTRIBUTING for further details.

License

MIT License

rules-framework's Issues

CSV file storage

Review throughput

Describe rules match/search execution plan

Purpose

Explain a rule match/search execution plan, as a way for users to troubleshoot defined rules.

Main functionalities

• Describe match one rule execution plan
• Describe match many rules execution plan
• Describe search rules execution plan
• Format execution plan as a text string

Proposal

Create new a entity on Rules.Framework named ExecutionPlan, including information:

• Input - content type, match date/match interval, conditions
• Evaluation options - match mode and if rules without search conditions are excluded
• Rules loaded from rules data source
• For each rule evaluated, on each condition node evaluated on tree:
  • Original condition node
  • Left hand operator value (and if it comes from input conditions or from default value)
  • Condition node w/ left hand operator replaced
  • Condition node evaluation result: True / False
  • Rule included/not included on matched rules
• Matched rules
• Rule selected by priority (when on match one scenario)

Have new methods with similar functionality as match one, match many and search:

• DescribeMatchOneAsync(...)
• DescribeMatchManyAsync(...)
• DescribeSearchAsync(...)

The best design would be to probably have original methods with this functionality added, but would probably mean a breaking change, so we either proceed with this approach and re-design the Rules Engine API on next semantic major version (where we could do a breaking change), or we can do it right away and move immediately to next semantic major version - open for debate.

NOTE: to add actual implementation plan later here.

Review time efficiency

Commit tag action is failing on builds

The build process is using the action anothrNick/github-tag-action, which is failing with following error:

fatal: unsafe repository ('/github/workspace' is owned by someone else)
To add an exception for this directory, call:
git config --global --add safe.directory /github/workspace

Check build link for details: https://github.com/Farfetch/rules-framework/runs/6791672988?check_suite_focus=true

Issue is already reported on anothrNick/github-tag-action#148 and up for resolution. In the meantime, we can use this workaround done on a fork of the repository - anothrNick/github-tag-action#148 (comment).

Broken support for data source providers on ValueConditionNode

Since version 1.0.121 a new implementation for valued condition nodes has been made available on core package, however, that implementation is not supported by provider packages, which tends to fail with a NotSupportedException.

We need to:

• Add support to both InMemory and MongoDb providers for new ValueConditionNode.
• Set minimum version for core Rules.Framework package on packages for InMemory and MongoDb providers as 1.0.121.
• Implement a test based solution to ensure that when changes are made to core Rules.Framework package, all dependent packages will support it - this allows to detect breaking changes and make necessary adjustments before releasing a new core Rules.Framework package.

Support new "contains" operator for string operands

Add support for a new operator "contains":

• Only applies to strings
• With the meaning of input condition (of type string) contains the string specified on operand (e.g. "The quick brown fox jumps over the lazy dog" contains "fox").
• Should validate on rules builder and rule building when trying to apply contains operator to other types (not string).

Rules builder - set logical operator with specific fluent methods

When adding composed nodes to rules, the fluent interface available on rules builder is:

(...)
return b.AsComposed()
    .WithLogicalOperator(LogicalOperators.Or)
    .AddCondition(cb =>
(...)

Which creates a experience issue given LogicalsOperators has literals And, Or and Eval. The literal Eval is not valid for composed nodes, but the following configuration is possible:

(...)
return b.AsComposed()
    .WithLogicalOperator(LogicalOperators.Eval)
    .AddCondition(cb =>
(...)

So in order to avoid framework consumers from using the framework this way, the proposal is to deprecate method WithLogicalOperator(LogicalOperators logicalOperator) and expose 2 new specific fluent methods on builder:

• WithAndLogicalOperator()
• WithOrLogicalOperator()

Deprecated method would be removed on next major release.

Have a solution for bootstraping rules engine using a hosted service

Registering a suggestion (thank you @fernando-a-marins), it would be nice to have a way to bootstrap rules creation on application startup.

Thinking about this for a bit, it seems possible to be done with IHostedService.

Leaving this registered to think about possible solution designs.

Add support for starts with and ends with

Description:

At this moment is not possible to use regular expressions like StartsWith and EndsWith .

For example, if we need to verify if an email is from an organization and if is from an external user.

[email protected]

We want to have the possibility to verify in conditions if StartsWith ext and if EndsWith xxxxx.com
using contains it's a problem because the ext can be in the middle of the email [email protected] and be a valid email and we don't want that.

InMemory providers always returns same instance of rules' object

RuleBuilder
    .NewRule<ContentTypes, ConditionTypes>()
    .WithName(RmaInvoiceDisclaimerForShippingCostV3)
    .WithContent(
        ContentTypes.RmaDisclaimerShippingCost,
        new DisclaimersValue(
        "Freight Cost of {Shipping.AmountInclTax} not to be included in grand total")
    )
    .WithDateBegin(DateTime.Parse("2019-01-01"))
    .Build(),
RuleAddPriorityOption.ByPriorityNumber(1) 

Take the above example, if the rule was configured on this away, whenever this rule matches the condition, the instance of the object DisclaimersValue returned is going to be always the same. It means that if the application changes the content of the object then it will be changed for the entire application.

Perhaps the behavior here should be the rules framework returning a new instance of the object for each time that the rules match the conditions.

Search rules by condition types in UI

Add the functionality to search rules filtering by condition types (one or more) on the search rules UI.

Add caching with in-memory support

Add a function to not evaluate the rules again with the same conditions and return the same result.

Review memory efficiency

Add sample project for SQL Server usage

Add sample project demonstrating:

• Creating rules engine.
• Setting up a SQL Server database data source.
• Building and adding rules.
• Updating existent rules.
• Evaluating rules (one and many matches).
• Searching rules.

Also update documentation to refer sample projects.

Rules framework not working with newer versions of FluentValidation

Recently when installing the Rules-Framework on services that are using recent versions of FluentValidation, we are having these kinds of exceptions

{"Method not found: 'System.Collections.Generic.IList'1<FluentValidation.Results.ValidationFailure> FluentValidation.Results.ValidationResult.get_Errors()'."}

In the case above, the version used is FluentValidation 10.3.6

Add license to README.md

SQLite file storage

Create PULL_REQUEST_TEMPLATE

Create extensible REST API library

The purpose is to create a new .NET class library (C#) that should be published as an additional Nuget package and should allow being added to an ASP.NET application HTTP pipeline, exposing new endpoints. The library should be able to:

• Be added as an additional configuration on the ASP.NET application startup file.
• Retrieve a RulesEngine instance from the service provider as the default behavior - application startup should not fail, but endpoints should fail when invoked.
• Allow defining, with custom logic, which RulesEngine instance is used by the REST API.
• Have an extensibility model that allows adding new endpoints - the library will have its' own endpoints, but library users should also be able to add their own endpoint implementations.
• Have the capacity to set which endpoints should be available - a default set of endpoints should be configured by the library, but it should be also possible to configure which endpoints will be available.

Geo-distribution support

InMemoryProviderRulesDataSource and InMemoryRulesStorage AsEnumerable()

• InMemoryProviderRulesDataSource.cs
• InMemoryRulesStorage

InMemoryProviderRulesDataSource methods are returning methods with ToList().AsReadOnly(), I would propose just use .AsEnumerable(). Since is already readonly.

InMemoryRulesStorage are return methods with .ToList().AsEnumerable(), I would propose just use .AsEnumerable().

That would slightly improve performance.

Wrong date interval evalution

It seems that we have a bug on data evaluation, if we define the dates interval like this example, the range between 2021-02-28 00:00:00.000 and 2021-02-28 23:59:59.999 have no match.

.WithDatesInterval(
  DateTime.Parse("2019-01-01"), 
  DateTime.ParseExact("2021-02-28 23:59:59.999", "yyyy-MM-dd HH:mm:ss.fff", CultureInfo.InvariantCulture)
) 

Priority option "AtPriorityNumber" throws InvalidOperationException if rules collection is empty

If one tries to add a first rule to RulesEngine using priority option AtPriorityNumber, operation will fail with following exception:

System.InvalidOperationException : Sequence contains no elements
at System.Linq.Enumerable.Min[TSource](IEnumerable1 source, Func2 selector)
at Rules.Framework.RulesEngine2.<AddRuleInternalAsync>d__9.MoveNext() at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
at Farfetch.Finance.Invoices.Data.Rules.Tests.ContentTypeBaseTests..ctor(IContentTypes contentTypes, ContentTypes contentType) in C:\Git\Projects\finance-invoices\src\tests\unit\Data.Rules.Tests\ContentTypeBaseTests.cs:line 27
at Farfetch.Finance.Invoices.Data.Rules.Tests.SalesInvoice.LabelsTax.LabelsTaxContentTypeTests..ctor() in C:\Git\Projects\finance-invoices\src\tests\unit\Data.Rules.Tests\SalesInvoice\LabelsTax\LabelsTaxContentTypeTests.cs:line 17

Update Fluent Assertions to latest version

Create CODE_OF_CONDUCT

JSON file storage

Allow default value (zero literal) when condition type is a enum

When framework consumer uses a enum type without specifying numeric values for each literal, the first literal will assume zero (default value) which is not accepted by rule builder validation.

The goal is to skip/disable this validation for enum typed condition types.

Replace SonarQube by Codacy for static analysis

Purpose

Replace SonarQube static analysis with Codacy static analysis. Reason: it is the Farfetch adopted solution for static analysis and this way static code analysis ownership lies within Farfetch's hands.

Main functionalities

• Remove SonarQube analysis steps
• Add project to Codacy (needs admin permissions) - https://app.codacy.com/organizations/gh/Farfetch/repositories
• Replace the breadcrumbs on the README file from SonarQube with the equivalent ones from Codacy.
• Delete old appveyor.yaml.old CI file

Update xUnit.net to latest version

Add sample project for MongoDB usage

Add sample project demonstrating:

• Creating rules engine.
• Setting up a MongoDB database data source.
• Building and adding rules.
• Updating existent rules.
• Evaluating rules (one and many matches).
• Searching rules.

Also update documentation to refer sample projects.

Allow pre-compilation of rules' conditions (only exact match modes)

Purpose

Create the capability for pre-compiling the rules' conditions evaluation tree to speed up rules evaluation.

Main functionalities

When rules are loaded from the rules data source, pre-compile ahead of evaluation time the rules' conditions evaluation tree so that evaluation is faster when matching/matching many.

Proposal

After rules load from the data source, process each rule loaded, pre-compiling the conditions and storing the pre-compiled conditions on the loaded rule. Make this an optional feature that can be enabled using the options when building a RulesEngine.

There are 2 options to implement pre-compilation (known to date - new alternatives are welcome):

• Use Roslyn's Scripting API to evaluate and generate a Func<IDictionary<TConditionType, object>>. This implies generating C# source code on a string from the conditions tree of the rule. See link for some details: https://www.strathweb.com/2018/01/easy-way-to-create-a-c-lambda-expression-from-a-string-with-roslyn/.
• Create lambda expressions dynamically using the Expressions API. See the following links for a great introduction to Expressions API:
  • Link 1: https://www.codemag.com/article/1607041
  • Link 2: https://www.codemag.com/article/1609041
  • Link 3: https://www.codemag.com/article/1611041

Then, to deal with pre-compiled conditions, change RulesEngine implementation to use pre-compiled conditions when available and also when using MatchMode.Exact. With these requirements, only MatchOneAsync(...) and MatchManyAsync(...) will benefit from pre-compilation optimizations.

NOTE: the proposal is open to debate and the actual implementation plan is to be added later here.

Inserting rules AtBottom causes exception when rules data source is empty for a ContentType

When creating a RulesEngine with an empty IRulesDataSource - for a ContentType at least - it does not allow to add new rule specifying priority RuleAddPriorityOption.AtBottom - fails with an InvalidOperationException.

Sample:

RulesEngine<AnimalCharacteristics, AnimalConditionTypes> rulesEngine = RulesEngineBuilder.CreateRulesEngine()
    .WithContentType<AnimalCharacteristics>()
    .WithConditionType<AnimalConditionTypes>()
    .SetInMemoryDataSource()
    .Build();

var ruleResult1 = RuleBuilder.NewRule<AnimalCharacteristics, AnimalConditionTypes>()
    .WithName("Animal sound for big dogs")
    .WithDateBegin(DateTime.Parse("2000-01-01"))
    .WithContent(AnimalCharacteristics.AnimalSounds, "Woof...")
    .WithCondition(cnb => cnb.AsComposed()
        .WithLogicalOperator(LogicalOperators.And)
        .AddCondition(x => x.AsValued(AnimalConditionTypes.AnimalClass)
            .OfDataType<string>()
            .WithComparisonOperator(Operators.Equal)
            .SetOperand("Mammal")
            .Build())
        .AddCondition(x => x.AsValued(AnimalConditionTypes.Animal)
            .OfDataType<string>()
            .WithComparisonOperator(Operators.Equal)
            .SetOperand("Dog")
            .Build())
        .Build())
    .Build();

Rule<AnimalCharacteristics, AnimalConditionTypes> rule1 = ruleResult1.Rule;

await rulesEngine.AddRuleAsync(rule1, RuleAddPriorityOption.AtBottom).ConfigureAwait(false); // Throws InvalidOperationException here

Please investigate further and fix the issue, so that insertion is allowed.

Rule definition UI

Is your request related to a problem you have?

Users should be able to create new rules via the Web UI.

Describe the solution you'd like

When using the Web UI for Rules.Framework (TBC)

Are you able to help bring it to life and contribute with a Pull Request?

Not yet

Additional context

No response

YAML file storage

Update of the System.Net.Http to 4.3.4

Isn't a direct dependency, but is a dependent library that represents a risk.

Create SECURITY.md

Create SQL Server rules data source provider

Design and implement new class library (dll) released as a Nuget package implementing a SQL Server rules data source provider, using Rules.Framework package existent extensibility.

Add sample project for in-memory usage

Add some sample projects demonstrating:

• Creating rules engine.
• Setting up an in-memory data source.
• Building and adding rules.
• Updating existent rules.
• Evaluating rules (one and many matches).
• Searching rules.

Also update documentation to refer sample projects.

Add caching capabilities with in-memory cache

Purpose

Create caching functionality for rules fetched from data source. Implement first cache provider in-memory.

Main functionalities

Extend Rules.Framework functionalities by adding following cache behavior:

• Cache response from rules data source.
• Cache response from specific operations - optionally activated if users desire so.
• Cache invalidation of specific sets of cached values.

Proposal

Perform changes to selected points:

• Cache response from rules data source - with cache key by content type and dates interval used to fetch from rules data source.
  • e.g. If one tries to match one rule for "ContenType1" on date 2021-11-28, RulesEngine will ask to IRulesDataSource the collection of rules active for "ContenType1" between 2021-11-28 (limit inclusive) and 2021-11-29 (limit exclusive). When this happens and caching is active, result will be stored on cache with key composed of "ContenType1", 2021-11-28 and 2021-11-29.
  • If a request is placed to rules data source that generates a given cache key, and that cache key has value on cache provider, stored value must be returned as response from rules data source.
• Cache response from specific operations of RulesEngine - MatchOneAsync(...), MatchManyAsync(...), SearchAsync(...) - considering conditions:
  • Generate a unique hash from operations parameter values.
  • Register, per each operation, the requests made with same hash for last X minutes.
    • Should be a configurable value?
  • Cache response of operation hashes with request counters superior to defined thresholds
    • Have a threshold based on absolute number of requests - configurable.
    • Have a threshold based on the percentage of requests (requests made for a specific operation hash vs. requests made for all operation hashes) - configurable.
  • Return cached response when criteria of thresholds is matched.
  • Implement mechanisms to ensure cache invalidation.
    • Always use a sliding window, where older requests tend to leave statistic on a cleanup activity and new requests are registered as they are done.
    • 2 alternatives to assure cache consistency: cached value max TTL or cache invalidation on operations that cause changes to rules data source.
  • Feature is optional, meaning, adding caching capabilities does not imply caching operation responses - needs to be explicitly configured to do so.
• Invalidate all cached values for any change done to a content type (add/update rules).
  • Consider invalidating only a subset if possible, meaning, if changed rule/rules only affect evaluation from date A to date B, only cached values between A and B need to be invalidated, remaining ones are still valid.

Create cache abstractions to allow:

• Setting a new cached value by cache key.
• Invalidate a cached value by cache key.
• Verify if a cache key has value.

And implement a cache specific implementation for in-memory, considering the abstractions above and grouping cached values by content type.

NOTE: proposal is open to debate and actual implementation plan is to be added later here.

Add caching sliding window TTL strategy

Add additional methods for building rules with composed conditions

It will be useful to add methods for building rules with conditions that:

• must match a value in a range of values
• must not match any value in a range of values
• (some other cases may be useful)

These methods should probably be added to the IComposedConditionNodeBuilder interface.

Such implementantions have already been done as extensions in some services that use the rules framework. One example can be found in RuleConditionExtension.cs in OTS.

Rules engine throws not supported exception when condition required by the rule was not supplied

Support new "in" operator for string, integer and decimal operands

Add support for a new operator "in":

• Applies to string, integer and decimal operands.
• Left hand operand (input) is a single value and right hand operand (condition node kept) is a enumeration of values (e.g. "allowed colors" -> "red" in ("red", "green", "blue", "yellow"))
• Should validate on rules builder and rule building when trying to apply in operator to other types (not string, integer or decimal).

Rules using Operators.In are not working properly with InMemory RulesFramework

Description:

When we try to create a rule using Operand.In the list sent in SetOperand is not being correctly evaluated. The problem seems to be in the Rules.Framework.Providers.InMemory.RuleFactory<TContentType, TConditionType>.CreateValueConditionNode method which, in case the DataType is DataTypes.String, is not considering the possibility that the conditionNodeDataModel.Operand can be an Enumerable:

        private static IConditionNode<TConditionType> CreateValueConditionNode(IConditionNodeBuilder<TConditionType> conditionNodeBuilder, ValueConditionNodeDataModel<TConditionType> conditionNodeDataModel)
        {
            return conditionNodeDataModel.DataType switch
            {
                DataTypes.Integer => conditionNodeBuilder.AsValued(conditionNodeDataModel.ConditionType)
                    .OfDataType<int>()
                    .WithComparisonOperator(conditionNodeDataModel.Operator)
                    .SetOperand(Convert.ToInt32(conditionNodeDataModel.Operand, CultureInfo.InvariantCulture))
                    .Build(),
                DataTypes.Decimal => conditionNodeBuilder.AsValued(conditionNodeDataModel.ConditionType)
                   .OfDataType<decimal>()
                   .WithComparisonOperator(conditionNodeDataModel.Operator)
                   .SetOperand(Convert.ToDecimal(conditionNodeDataModel.Operand, CultureInfo.InvariantCulture))
                   .Build(),
                DataTypes.String => conditionNodeBuilder.AsValued(conditionNodeDataModel.ConditionType)
                   .OfDataType<string>()
                   .WithComparisonOperator(conditionNodeDataModel.Operator)
                   .SetOperand(Convert.ToString(conditionNodeDataModel.Operand, CultureInfo.InvariantCulture))
                   .Build(),
                DataTypes.Boolean => conditionNodeBuilder.AsValued(conditionNodeDataModel.ConditionType)
                   .OfDataType<bool>()
                   .WithComparisonOperator(conditionNodeDataModel.Operator)
                   .SetOperand(Convert.ToBoolean(conditionNodeDataModel.Operand, CultureInfo.InvariantCulture))
                    .Build(),
                _ => throw new NotSupportedException($"Unsupported data type: {conditionNodeDataModel.DataType}."),
            };
        }

OS:

Windows 10

.Net SDK:

.Net Standard 2.1

Exception message and stack trace:

"System.NotSupportedException: The combination 'one-to-one-In' is not supported.
at Rules.Framework.Evaluation.ValueEvaluation.Dispatchers.ConditionEvalDispatchProvider.ThrowIfUnsupportedOperandsAndOperatorCombination(String combination) in C:\\_git\\rules-framework\\src\\Rules.Framework\\Evaluation\\ValueEvaluation\\Dispatchers\\ConditionEvalDispatchProvider.cs:line 71
at Rules.Framework.Evaluation.ValueEvaluation.Dispatchers.ConditionEvalDispatchProvider.GetEvalDispatcher(Object leftOperand, Operators operator, Object rightOperand) in C:\\_git\\rules-framework\\src\\Rules.Framework\\Evaluation\\ValueEvaluation\\Dispatchers\\ConditionEvalDispatchProvider.cs:line 55
at Rules.Framework.Evaluation.ValueEvaluation.DeferredEval.Eval[TConditionType](IEnumerable`1 conditions, IValueConditionNode`1 valueConditionNode, Object rightOperand, MatchModes matchMode) in C:\\_git\\rules-framework\\src\\Rules.Framework\\Evaluation\\ValueEvaluation\\DeferredEval.cs:line 64
at Rules.Framework.Evaluation.ValueEvaluation.DeferredEval.Eval[TConditionType](IEnumerable`1 conditions, ValueConditionNode`1 valueConditionNode, MatchModes matchMode) in C:\\_git\\rules-framework\\src\\Rules.Framework\\Evaluation\\ValueEvaluation\\DeferredEval.cs:line 42
at Rules.Framework.Evaluation.ValueEvaluation.DeferredEval.<>c__DisplayClass3_1`1.<GetDeferredEvalFor>b__4(IEnumerable`1 conditions) in C:\\_git\\rules-framework\\src\\Rules.Framework\\Evaluation\\ValueEvaluation\\DeferredEval.cs:line 30
at Rules.Framework.Evaluation.Specification.FuncSpecification`1.IsSatisfiedBy(T input) in C:\\_git\\rules-framework\\src\\Rules.Framework\\Evaluation\\Specification\\FuncSpecification.cs:line 14
at Rules.Framework.Evaluation.ConditionsEvalEngine`1.Eval(IConditionNode`1 conditionNode, IEnumerable`1 conditions, EvaluationOptions evaluationOptions) in C:\\_git\\rules-framework\\src\\Rules.Framework\\Evaluation\\ConditionsEvalEngine.cs:line 29
at Rules.Framework.RulesEngine`2.<>c__DisplayClass10_0.<MatchAsync>b__0(Rule`2 r) in C:\\_git\\rules-framework\\src\\Rules.Framework\\RulesEngine.cs:line 272
at System.Linq.Enumerable.WhereListIterator`1.ToList()
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Rules.Framework.RulesEngine`2.MatchAsync(TContentType contentType, DateTime matchDateBegin, DateTime matchDateEnd, IEnumerable`1 conditions, EvaluationOptions evaluationOptions) in C:\\_git\\rules-framework\\src\\Rules.Framework\\RulesEngine.cs:line 271
at Rules.Framework.RulesEngine`2.MatchOneAsync(TContentType contentType, DateTime matchDateTime, IEnumerable`1 conditions) in C:\\_git\\rules-framework\\src\\Rules.Framework\\RulesEngine.cs:line 109
[...]"

A repro of the issue:

this.Add(
    this.Add(
        RuleBuilder
            .NewRule<RuleContentType, ConditionType>()
            .WithName("Some Rule")
            .WithContent(
                RuleContentType.SomeContentType,
                  "SomeText"
            )
            .WithCondition(x => x
                .AsValued(ConditionType.SomeConditionType).OfDataType<string>()
                .WithComparisonOperator(Operators.In)
                .SetOperand(new[] { "11111", "22222", "33333", "44444", "55555" })
                .Build()
            )
            .WithDateBegin(DateTime.Parse("2022-01-01"))
            .Build(),
        RuleAddPriorityOption.ByPriorityNumber(1)
    );

Branch with the unit test reproducing the problem
https://github.com/Farfetch/rules-framework/tree/test/inmemory_problem

Add support for SQLServer provider

Purpose

Have support to manage rules using a SQL Server repository to persist them.

Main functionalities

• Add rules on SQL Server repository
• Get rules from SQL Server repository
• Update rules on SQL Server repository
• Configure optionally a schema - default is dbo when omitted.

Proposal

Development can be partitioned into 3 parts:

1. Repository + Data Model implementation
2. Schema creation scripts
3. Automatic schema creation on application startup (using scripts) --> not mandatory, but useful.

Proposed schema:

NOTE: refer to KafkaFlow Retry extension which might help to create a solution to automate schema creation.

Record and report rules usage statistics

Purpose

Record usage of each rule and generate statistics, to help users assess the impact of specific rules on their usage scenarios.

Main functionalities

• Record rules evaluation counters - per rule.
• Record rules match counters - per rule, separate counters for match one and match many.
• Generate parameterized statistics in tabular-like format.

Assume default time bucket for counters as 1 hour - but make it configurable. Make sure each time bucket records its' start date and time.

Proposal

Assume the record of evaluation and match counters to be created/updated on each evaluation done to RulesEngine. A internal Rules.Framework component for acting as listener should be designed and implemented to achieve this - design open for suggestions, leaving this without proposal for now.

Create a new method on RulesEngine to generate statistics in a tabular-like format - Statistics GenerateRulesUsageStatistics(GenerateRulesUsageStatisticsArgs generateStatisticsArgs):

• Receive as parameter new value object GenerateRulesUsageStatisticsArgs:
  • Allow defining a time interval - DateBegin and DateEnd
    • Should we make time interval mandatory? If not mandatory, should we have a hard default defined (not configurable)?
    • Should we limit time interval range?
  • Allow filtering by content type - ContentType
  • Allow filtering for rule names set - RuleNames - an array of strings
  • Must define scope of statistics generation - Scope - an enum parameter:
    • Evaluation - generate statistics for rules evaluation
    • MatchOne - generate statistics for match one result rules
    • MatchMany - generate statistics for match many result rules
    • Match - generate statistics for all match types result rules
• Return result in new object Statistics:
  • Must have a collection of StatisticsBucket:
    • Each StatisticsBucket includes:
      • Name to describe, in this case, which rule name the bucket refers to
      • A Timestamp to describe in time to when the bucket is reporting
      • And also a numeric Value to refer the bucket value - in this case it will be an integer, but it would be appropriate to use a decimal or a floating point type for future usage on other statistics use cases.
  • Should we group StatisticsBucket per name or per timestamp?
• Must have a time interval - DateBegin and DateEnd

NOTE: proposal is open to debate and actual implementation plan is to be added later here.

Add support for searching rules with text ignoring case sensitive

Description:
At this moment we are supporting rules search by string using StartsWith and EndsWith functions but this search is case-sensitive and culture-sensitive.

In the future, it may be useful to make these 2 kinds of searches not take into consideration the case-sensitive.
For that we can add 2 similar operators:

• InsensitiveStartsWith or IStartsWith
• InsensitiveEndsWith or IEndsWith

As an example, we may want to search the word dog inside a text that we don't have sure how it is defined like the quick brown fox jumps over the lazy Dog. With this configuration, the actual operator EndsWith will not work.

Please see this Microsoft info about the functions override (equivalent for String.StartsWith):

• using EndsWith(String, StringComparison)
• using EndsWith(String, Boolean, CultureInfo)

Get the unique condition types associated with rules of a specific content type

The purpose is to be able to fetch the unique condition types associated with the rules persisted on data source for a specific content type.

i.e. content type "ABC" has condition types 1 and 2 for rules between dates A and B, and this functionality would let us know that we would need to fill conditions 1 and 2 for rules evaluation on date interval between A and B.

This may have applications such as:

• Implementing automated application mechanisms that fill the needed conditions to ensure a consistent rule evaluation without falling back to data type defaults .
• Having a dynamic UI which presents the conditions needed to be passed to evaluate a content type rules at a specific date.

Of course there can be other applications, but those are the ones I can remember for now.

Geo-replication support