Coder Social home page Coder Social logo

kros.korm's Introduction

Kros.KORM Build Status

Kros.KORM is simple, fast and easy to use micro-ORM framework for .NETStandard created by Kros a.s. from Slovakia.

Why to use Kros.KORM

  • You can easily create query builder for creating queries returning IEnumerable of your POCO objects
  • Linq support
  • Saving changes to your data (Insert / Update / Delete)
  • Kros.KORM supports bulk operations for fast inserting and updating large amounts of data (BulkInsert, BulkDelete)

Documentation

For configuration, general information and examples see the documentation.

Download

Kros.KORM is available from:

Contributing Guide

To contribute with new topics/information or make changes, see contributing for instructions and guidelines.

This topic contains following sections

Query

You can use Kros.KORM for creating queries and their materialization. Kros.KORM helps you put together desired query, that can return instances of objects populated from database by using foreach or linq.

Query for obtaining data

var people = database.Query<Person>()
    .Select("p.Id", "FirstName", "LastName", "PostCode")
    .From("Person JOIN Address ON (Person.AddressId = Address.Id)")
    .Where("Age > @1", 18);

foreach (var person in people)
{
    Console.WriteLine(person.FirstName);
}

For more information take a look at definition of IQuery.

Linq to Kros.KORM

Kros.KORM allows you to use Linq for creating queries. Basic queries are translated to SQL language.

Example

var people = database.Query<Person>()
    .From("Person JOIN Address ON (Person.AddressId = Address.Id)")
    .Where(p => p.LastName.EndsWith("ová"))
    .OrderByDescending(p => p.Id)
    .Take(5);

foreach (var person in people)
{
    Console.WriteLine(person.FirstName);
}

Supported Linq methods are Where, FirstOrDefault, Take, Sum, Max, Min, OrderBy, OrderByDescending, ThenBy, ThenByDescending, Count, Any, Skip.

Other methods, such as Select, GroupBy, Join are not supported at this moment because of their complexity.

You can use also some string functions in Linq queries:

String function Example Translation to T-SQL
StartWith Where(p => p.FirstName.StartWith("Mi")) WHERE (FirstName LIKE @1 + '%')
EndWith Where(p => p.LastName.EndWith("ová")) WHERE (LastName LIKE '%' + @1)
Contains Where(p => p.LastName.Contains("ia")) WHERE (LastName LIKE '%' + @1 + '%')
IsNullOrEmpty Where(p => String.IsNullOrEmpty(p.LastName)) WHERE (LastName IS NULL OR LastName = '')
ToUpper Where(p => p.LastName.ToUpper() == "Smith") WHERE (UPPER(LastName) = @1)
ToLower Where(p => p.LastName.ToLower() == "Smith") WHERE (LOWER(LastName) = @1)
Replace Where(p => p.FirstName.Replace("hn", "zo") == "Jozo") WHERE (REPLACE(FirstName, @1, @2) = @3)
Substring Where(p => p.FirstName.Substring(1, 2) == "oh") WHERE (SUBSTRING(FirstName, @1 + 1, @2) = @3)
Trim Where(p => p.FirstName.Trim() == "John") WHERE (RTRIM(LTRIM(FirstName)) = @1)
Compare Where(p => string.Compare(p.FirstName, "John") == 0) WHERE (CASE WHEN FirstName = @1 THEN 0 WHEN FirstName < @2 THEN -1 ELSE 1 END = @3)

Translation is provided by implementation of ISqlExpressionVisitor.

DataAnnotation attributes

Properties (not readonly or writeonly properties) are implicitly mapped to database fields with same name. When you want to map property to database field with different name use AliasAttribute. The same works for mapping POCO classes with database tables.

[Alias("Workers")]
private class Staff
{
    [Alias("PK")]
    public int Id { get; set; }

    [Alias("Name")]
    public string FirstName { get; set; }

    [Alias("SecondName")]
    public string LastName { get; set; }
}

private void StaffExample()
{
    using (var database = new Database(_connection))
    {
        _command.CommandText = "SELECT PK, Name, SecondName from Workers";

        using (var reader = _command.ExecuteReader())
        {
            var staff = database.ModelBuilder.Materialize<Staff>(reader);
        }
    }
}

When you need to have read-write properties independent of the database use NoMapAttribute.

[NoMap]
public int Computed { get; set; }

Convention model mapper

If you have different conventions for naming properties in POCO classes and fields in database, you can redefine behaviour of ModelMapper, which serves mapping POCO classes to database tables and vice versa.

Redefining mapping conventions example

Database.DefaultModelMapper.MapColumnName = (colInfo, modelType) =>
{
    return string.Format("COL_{0}", colInfo.PropertyInfo.Name.ToUpper());
};

Database.DefaultModelMapper.MapTableName = (tInfo, type) =>
{
    return string.Format("TABLE_{0}", type.Name.ToUpper());
};

using (var database = new Database(_connection))
{

    _command.CommandText = "SELECT COL_ID, COL_FIRSTNAME from TABLE_WORKERS";

    using (var reader = _command.ExecuteReader())
    {
        var people = database.ModelBuilder.Materialize<Person>(reader);

        foreach (var person in people)
        {
            Console.WriteLine(person.FirstName);
        }
    }
}

Alternatively you can write your own implementation of IModelMapper.

Custom model mapper
Database.DefaultModelMapper = new CustomModelMapper();

If your POCO class is defined in external library, you can redefine mapper, so it can map properties of the model to desired database names.

External class mapping example
var externalPersonMap = new Dictionary<string, string>() {
    { nameOf(ExternalPerson.oId), "Id" },
    { nameOf(ExternalPerson.Name), "FirstName" },
    { nameOf(ExternalPerson.SecondName), "LastName" }
};

Database.DefaultModelMapper.MapColumnName = (colInfo, modelType) =>
{
    if (modelType == typeof(ExternalPerson))
    {
        return externalPersonMap[colInfo.PropertyInfo.Name];
    }
    else
    {
        return colInfo.PropertyInfo.Name;
    }
};

using (var database = new Database(_connection))
{
    var people = database.Query<ExternalPerson>();

    foreach (var person in people)
    {
        Console.WriteLine($"{person.oId} : {person.Name}-{person.SecondName}");
    }
}

For dynamic mapping you can use method SetColumnName<TModel, TValue>

Database.DefaultModelMapper.SetColumnName<Person, string>(p => p.Name, "FirstName");

Configure model mapping by fluent api

Configuration by data annotation attributes is OK in many scenarios. However, there are scenarios where we want to have a model definition and mapping it to a database separate. For example, if you want to have entities in domain layer and mapping in infrastructure layer.

For these scenarios you can derive database configuration from DatabaseConfigurationBase.

public class User
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string FullName => FirstName + " " + LastName;
    public Address Address { get; set; }
    public IEmailService EmailService { get; set; }
}

public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Entity<User>()
            .HasTableName("Users")
            .HasPrimaryKey(entity => entity.Id).AutoIncrement(AutoIncrementMethodType.Custom)
            .UseConverterForProperties<string, NullToEmptyStringConverter>()
            .Property(entity => entity.Title).IgnoreConverter()
            .Property(entity => entity.FirstName).HasColumnName("Name")
            .Property(entity => entity.FullName).NoMap()
            .Property(entity => entity.Addresses).UseConverter<AddressConverter>()
            .Property(entity => entity.EmailService).InjectValue(() => new EmailService())
            .Property(entity => entity.IsGenerated).UseValueGeneratorOnInsert<RandomGenerator>();
    }
}

And use IDatabaseBuilder for creating KORM instance.

var database = Database
    .Builder
    .UseConnection(connection)
    .UseDatabaseConfiguration<DatabaseConfiguration>()
    .Build();

If converter is used for property type (UseConverterForProperties) and also for specific property of that type (UseConverter), the latter one has precedence.

Global query filter

In many cases, we want to define a global filter to apply to each query. For example: ParentId = 1, UserId = ActiveUser.Id, etc.

You can configurate query filter in DatabaseConfiguration class.

public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Table("Document")
            .UseQueryFilter<Document>(entity => entity.UserId == ActiveUser.Id && entity.ParentId == 1);
    }
}

KORM will automatically add a condition ((UserId = @__Dqf1) AND (ParentId = @__Dqf2)) when calling any query using Query<Document>().

Except for direct sql calls like _database.Query<Document>().Sql("SELECT * FROM DOCUMENT").

⚠️ Configuration modelBuilder.Table("Documents") is applied for all entities mapped to table Documents (for example Document, DocumentDto, DocumentInfo, ...).

Ignoring global filters

If I need to call a query without these conditions, I must explicitly say:

_database.Query<Document>()
    .IgnoreQueryFilters()
    .ToList();

Reserved words as names for columns or table

If you need to name your column or table with a name that the server has among the reserved words (for example, Order), you must ensure that these names are quoted in queries. If the queries are generated by KORM, then you must explicitly specify delimiters in the DatabaseConfiguration.

public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.UseIdentifierDelimiters(Delimiters.SquareBrackets);
    }
}

Converters

Data type of column in database and data type of property in your POCO class may differ. Some of these differences are automatically solved by Kros.KORM, for example double in database is converted to int in your model, same as int in database to enum in model, etc.

For more complicated conversion Kros.KORM offers possibility similar to data binding in WPF, where IValueConverter is used.

Imagine you store a list of addresses separated by some special character (for example #) in one long text column, but the property in your POCO class is list of strings.

Let's define a converter that can convert string to list of strings.

public class AddressesConverter : IConverter
{
    public object Convert(object value)
    {
        var ret = new List<string>();
        if (value != null)
        {
            var address = value.ToString();
            var addresses = address.Split('#');

            ret.AddRange(addresses);
        }

        return ret;
    }

    public object ConvertBack(object value)
    {
        return  string.Join("#", (value as List<string>));
    }
}

And now you can set this converter for your property using attribute or fluent configuration.

[Converter(typeof(AddressesConverter))]
public List<string> Addresses { get; set; }

Value generators

Value generators are used to generate column values. KORM contains some predefined generators but you can create your own.

For this purpose exists IValueGenerator interface which your class must implement.

public interface IValueGenerator
{
    object GetValue();
}

Here is an example of custom value generator:

private class AutoIncrementValueGenerator : IValueGenerator
{
      public object GetValue() => 123;
}

For using value generators you can use these three methods in DatabaseConfiguration:

  • .UseValueGeneratorOnInsert<YourGenerator>() - values will be generated on insert to the database.

  • .UseValueGeneratorOnUpdate<YourGenerator>() - values will be generated on update to the database.

  • .UseValueGeneratorOnInsertOrUpdate<YourGenerator>() - values will be generated on insert and update to the database.

Currently predefined value generators:

  • CurrentTimeValueGenerator - Generator generates date and time that are set to the current Coordinated Universal Time (UTC).

Ignoring value generators

If you need to perfrom a database operation without the use of value generators you can do so by passing parameter ignoreValueGenerators in either of the commit methods. This will ensure that all operations within commited transaction will be completed without supplementing values from value generators.

_dbSet.CommitChanges(ignoreValueGenerators: true);

OnAfterMaterialize

If you want to do some special action right after materialisation is done (for example to do some calculations) or you want to get some other values from source reader, that can not by processed automatically, your class should implement interface IMaterialize.

You can do whatever you need in method OnAfterMaterialize.

For example, if you have three int columns for date in database (Year, Month and Day) but in your POCO class you have only one date property, you can solve it as follows:

[NoMap]
public DateTime Date { get; set; }

public void OnAfterMaterialize(IDataRecord source)
{
    var year = source.GetInt32(source.GetOrdinal("Year"));
    var month = source.GetInt32(source.GetOrdinal("Month"));
    var day = source.GetInt32(source.GetOrdinal("Day"));

    this.Date = new DateTime(year, month, day);
}

Property injection

Sometimes you might need to inject some service to your model, for example calculator or logger. For these purposes Kros.KORM offers IInjectionConfigurator, that can help you with injection configuration.

Let's have properties in model

[NoMap]
public ICalculationService CalculationService { get; set; }

[NoMap]
public ILogger Logger { get; set; }

And that is how you can configure them.

Database.DefaultModelMapper
    .InjectionConfigurator<Person>()
        .FillProperty(p => p.CalculationService, () => new CalculationService())
        .FillProperty(p => p.Logger, () => ServiceContainer.Instance.Resolve<ILogger>());

Model builder

For materialisation Kros.KORM uses IModelFactory, that creates factory for creating and filling your POCO objects.

By default DynamicMethodModelFactory is implemented, which uses dynamic method for creating delegates.

If you want to try some other implementation (for example based on reflexion) you can redefine property Database.DefaultModelFactory.

Database.DefaultModelFactory = new ReflectionModelfactory();

Committing of changes

You can use Kros.KORM also for editing, adding or deleting records from database. IdDbSet is designed for that.

Records to edit or delete are identified by the primary key. You can set primary key to your POCO class by using Key attribute.

[Key()]
public int Id { get; set; }

public string FirstName { get; set; }

public string LastName { get; set; }

Inserting records to database

public void Insert()
{
    using (var database = new Database(_connection))
    {
        var people = database.Query<Person>().AsDbSet();

        people.Add(new Person() { Id = 1, FirstName = "Jean Claude", LastName = "Van Damme" });
        people.Add(new Person() { Id = 2, FirstName = "Sylvester", LastName = "Stallone" });

        people.CommitChanges();
    }
}

Kros.KORM supports bulk inserting, which is one of its best features. You add records to DbSet standardly by method Add, but for committing to database use method BulkInsert instead of CommitChanges.

var people = database.Query<Person>().AsDbSet();

foreach (var person in dataForImport)
{
    people.Add(person);
}

people.BulkInsert();

Kros.KORM supports also bulk update of records, you can use BulkUpdate method.

var people = database.Query<Person>().AsDbSet();

foreach (var person in dataForUpdate)
{
    people.Edit(person);
}

people.BulkUpdate();

This bulk way of inserting or updating data is several times faster than standard inserts or updates.

For both of bulk operations you can provide data as an argument of method. The advantage is that if you have a specific enumerator, you do not need to spill data into memory.

Primary key generating

Kros.KORM supports generating of primary keys for inserted records.

Support two types of generating:

  1. Custom

KORM supports 'Int32' and 'Int64' generators. Primary key property in POCO class must be decorated by Key attribute and its property AutoIncrementMethodType must be set to Custom.

public class User
{
    [Key(autoIncrementMethodType: AutoIncrementMethodType.Custom)]
    public int Id { get; set; }
}

Kros.KORM generates primary key for every inserted record, that does not have value for primary key property. For generating primary keys implementations of IIdGenerator are used.

The names of internal generators are the same as table names, for which the values are generated. But this can be explicitly set to some other name. It can be used for example when one generated sequence of numbers need to be used in two tables.

public class User
{
    [Key(autoIncrementMethodType: AutoIncrementMethodType.Custom, generatorName: "CustomGeneratorName")]
    public int Id { get; set; }
}

// Or using fluent database configuration.

public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Entity<User>()
            .HasPrimaryKey(entity => entity.Id).AutoIncrement(AutoIncrementMethodType.Custom, "CustomGeneratorName");
    }
}
  1. Identity

When you set AutoIncrementMethodType to Identity, Kros.KORM use MsSql Identity for generating primary key and fill generated keys into entity.

CREATE TABLE [dbo].[Users](
    [Id] [bigint] IDENTITY(1,1) NOT NULL,
    [FIrstName] [nvarchar](50) NULL,
CONSTRAINT [PK_Users] PRIMARY KEY CLUSTERED
(
    [Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
[Key(autoIncrementMethodType: AutoIncrementMethodType.Identity)]
public long Id { get; set; }

When you call dbSet.CommitChanges(), Kros.KORM fill generated keys into entity. Unfortunately, doesn't know it when you call a method dbSet.BulkInsert().

Editing records in database

public void Edit()
{
    using (var database = new Database(_connection))
    {
        var people = database.Query<Person>().AsDbSet();

        foreach (var person in people)
        {
            person.LastName += "ová";
            people.Edit(person);
        }

        people.CommitChanges();
    }
}

Deleting records from database

public void Delete()
{
    using (var database = new Database(_connection))
    {
        var people = database.Query<Person>().AsDbSet();

        people.Delete(people.FirstOrDefault(x => x.Id == 1));
        people.Delete(people.FirstOrDefault(x => x.Id == 2));

        people.CommitChanges();
    }
}
Deleting records by Ids or condition
public void Delete()
{
    using (var database = new Database(_connection))
    {
        var people = database.Query<Person>().AsDbSet();

        people.Delete(1);
        people.Delete(p => p.ParentId == 10);

        people.CommitChangesAsync();
    }
}

Upsert record by it's primary key

Kros.KORM supports upserting records based on primary key match. This can be handy when you know the primary key (or composite primary key) of the record but you can not be sure if it already exists in database.

Given (CompanyId, UserId) is composite primary key for UserRole table:

var admin = new UserRole { CompanyId = 1, UserId = 11, Role = "Admin" };
var owner = new UserRole { CompanyId = 1, UserId = 11, Role = "Owner" };
var user = new UserRole { CompanyId = 2, UserId = 22, Role = "User" };

using (var database = new Database(_connection))
{
    var userRoles = database.Query<UserRole>().AsDbSet();

    userRoles.Add(admin);
    userRoles.CommitChanges();

    var userRoles = database.Query<UserRole>().AsDbSet();

    userRoles.Upsert(owner); // this will update admins UserRole to owner
    userRoles.Upsert(user); // this will insert user
    userRoles.CommitChanges();
}

Upsert record by custom columns match

It is possible to upsert records by match of non PK columns. !!! Use this with caution. This updates all records with matching provided columns !!!

var admin1 = new UserRole { Id = 1, InternalUserNo = 11, Role = "Admin" };
var admin2 = new UserRole { Id = 2, InternalUserNo = 12, Role = "Admin" };
var owner1 = new UserRole { Id = 3, InternalUserNo = 11, Role = "Owner" };

using (var database = new Database(_connection))
{
    var userRoles = database.Query<UserRole>().AsDbSet();

    userRoles.Add(admin1);
    userRoles.CommitChanges();

    var userRoles = database.Query<UserRole>().AsDbSet()
        .WithCustomUpsertConditionColumns(nameof(UserRole.InternalUserNo));

    userRoles.Upsert(admin2); // this will insert new admin with internalUserNo = 12
    userRoles.Upsert(owner1); // this will update user with internalUserNo = 11 to Owner
    userRoles.CommitChanges();
}

Explicit transactions

By default, changes of a DbSet are committed to database in a transaction. If committing of one record fails, rollback of transaction is executed.

Sometimes you might come to situation, when such implicit transaction would not meet your requirements. For example you need to commit changes to two tables as an atomic operation. When saving changes to first of tables is not successful, you want to discard changes to the other table. Solution of that task is easy with explicit transactions supported by Kros.KORM. See the documentation of BeginTransaction.

using (var transaction = database.BeginTransaction())
{
    var invoicesDbSet = database.Query<Invoice>().AsDbSet();
    var itemsDbSet = database.Query<Item>().AsDbSet();

    try
    {
        invoicesDbSet.Add(invoices);
        invoicesDbSet.CommitChanges();

        itemsDbSet.Add(items);
        itemsDbSet.CommitChanges();

        transaction.Commit();
    }
    catch
    {
        transaction.Rollback();
    }
}

Simplify Adding, Deleting and Editing records

For simplifying calling methods (Add, Edit, Delete) use extension methods from IDatabaseExtensions class.

await database.AddAsync(person);
await database.AddAsync(people);
await database.BulkAddAsync(people);
await database.DeleteAsync(person);
await database.DeleteAsync(people);
await database.DeleteAsync<Person>(2);
await database.DeleteAsync<Person>(p => p.Id == 2);
await database.DeleteAsync<Person>("Id = @1", 2);
await database.EditAsync(person);
await database.EditAsync(person, "Id", "Age");
await database.EditAsync(people);
await database.BulkEditAsync(people);
await database.UpsertAsync(person);
await database.UpsertAsync(people);

Execute with temp table

Kros.KORM offers special execute commands for SQL databases, that inserts provided simple data into temp table and then executes some specified action using those temporary data. You can find these extension methods in IDatabaseExtensions class.

database.ExecuteWithTempTable<TValue>(IEnumerable<TValue> values, action);
await database.ExecuteWithTempTableAsync<TValue>(IEnumerable<TValue> values, function);

database.ExecuteWithTempTable<TKey, TValue>(IDictionary<TKey, TValue> values, action);
await database.ExecuteWithTempTable<TKey, TValue>(IDictionary<TKey, TValue> values, function);

T database.ExecuteWithTempTable<T, TValue> (IEnumerable<TValue> values, action);
await T database.ExecuteWithTempTable<T, TValue> (IEnumerable<TValue> values, function);

T database.ExecuteWithTempTable<T, TKey, TValue> (IDictionary<TKey, TValue> values, action);
await T database.ExecuteWithTempTable<T,TKey, TValue> (IDictionary<TKey, TValue> values, function);

public class Person
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

var ids = new List<int>(){ 0, 1, 2, 3 };

_database.ExecuteWithTempTable(ids, (database, tableName)
    => database.Query<Person>()
        .From($"PERSON AS P INNER JOIN {tableName} AS T ON (P.Id = T.Value)")
        .ToList());
        
public class IdDto
{
    public IdDto(int id)
    {
        Id = id;
    }
    
    public int Id { get; set; }
}

var ids = new List<IdDto>(){ new IdDto(0), new IdDto(1), new IdDto(2), new IdDto(3) };

_database.ExecuteWithTempTable(ids, (database, tableName)
    => database.Query<Person>()
        .Select("P.*")
        .From($"PERSON AS P INNER JOIN {tableName} AS T ON (P.Id = T.Id)")
        .ToList());

SQL commands executing

Kros.KORM supports SQL commands execution. There are three types of commands:

  • ExecuteNonQuery for commands that do not return value (DELETE, UPDATE, ...)

    private Database _database = new Database(new SqlConnection("connection string"));
    
    // to work with command parameters you can use CommandParameterCollection
    var parameters = new CommandParameterCollection();
    parameters.Add("@value", "value");
    parameters.Add("@id", 10);
    parameters.Add("@type", "DateTime");
    
    _database.ExecuteNonQuery("UPDATE Column = @value WHERE Id = @id AND Type = @type", parameters);
    
    // or you can send them directly via params array
    _database.ExecuteNonQuery("UPDATE Column = @value WHERE Id = @id AND Type = @type", "value", 10, "DateTime");
  • ExecuteScalar for commands that return only one value (SELECT)

  • ExecuteStoredProcedure for executing of stored procedures. Stored procedure may return scalar value or list of values or it can return data in output parameters.

Execution of stored procedure example

public class Person
{
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public DateTime BDay { get; set; }
}

private Database _database = new Database(new SqlConnection("connection string"));

// Stored procedure returns a scalar value.
int intResult = _database.ExecuteStoredProcedure<int>("ProcedureName");
DateTime dateResult = _database.ExecuteStoredProcedure<DateTime>("ProcedureName");

// Stored procedure sets the value of output parameter.
var parameters = new CommandParameterCollection();
parameters.Add("@param1", 10);
parameters.Add("@param2", DateTime.Now);
parameters.Add("@outputParam", null, DbType.String, ParameterDirection.Output);

_database.ExecuteStoredProcedure<string>("ProcedureName", parameters);

Console.WriteLine(parameters["@outputParam"].Value);


// Stored procedure returns complex object.
Person person = _database.ExecuteStoredProcedure<Person>("ProcedureName");


// Stored procedure returns list of complex objects.
IEnumerable<Person> persons = _database.ExecuteStoredProcedure<IEnumerable<Person>>("ProcedureName");

CommandTimeout support

If you want to execute time-consuming command, you will definitely appreciate CommandTimeout property of transaction. See the documentation of BeginTransaction.

Warning: You can set CommandTimeout only for main transaction, not for nested transactions. In that case CommandTimout of main transaction will be used.

IEnumerable<Person> persons = null;

using (var transaction = database.BeginTransaction(IsolationLevel.Chaos))
{
    transaction.CommandTimeout = 150;

    try
    {
        persons = database.ExecuteStoredProcedure<IEnumerable<Person>>("LongRunningProcedure_GetPersons");
        transaction.Commit();
    }
    catch
    {
        transaction.Rollback();
    }
}

Record types

KORM supports a new record type for model definition.

public record Person(int Id, string FirstName, string LastName);

using var database = new Database(new SqlConnection("connection string"));

foreach (Person person = database.Query<Person>())
{
    Console.WriteLine($"{person.Id}: {person.FirstName} - {person.LastName}");
}

The same features as for "standard" class-es are supported. Converters, name mapping and value injection. It is possible to use fluent notation, but also using attributes.

To use attribute notation, you must use syntax with property: keyword.

public record Person(int Id, [property: Alias("FirstName")]string Name);

Materializing record types is a bit faster than with property-filled classes.

1000 rows of InMemoryDbReader:

Method Mean Error StdDev
RecordTypes 301.50 μs 5.07 μs 7.11 μs
ClassTypes 458.10 μs 7.13 μs 6.68 μs

Logging

Kros.KORM offers the ability to log each generated and executed query. All you have to do is add this line to your source code.

Database.Log = Console.WriteLine;

Supported database types

Kros.KORM uses its own QueryProvider to execute query in a database. ISqlExpressionVisitor transforms IQuery to SELECT command specific for each supported database engine.

MsAccess is suported from version 2.4 in Kros.KORM.MsAccess library. If you need to work with MS Access database, you have to refer this library in your project and register MsAccessQueryProviderFactory.

MsAccessQueryProviderFactory.Register();

Current version of Kros.KORM suports databases MS ACCESS and MS SQL.

If you want to support a different database engine, you can implement your own IQueryProvider. And register it in QueryProviderFactories.

public class CustomQueryProvider : QueryProvider
{
    public CustomQueryProvider(ConnectionStringSettings connectionString,
       ISqlExpressionVisitor sqlGenerator,
       IModelBuilder modelBuilder,
       ILogger logger)
        : base(connectionString, sqlGenerator, modelBuilder, logger)
    {
    }

    public CustomQueryProvider(DbConnection connection,
        ISqlExpressionVisitor sqlGenerator,
        IModelBuilder modelBuilder,
        ILogger logger)
            : base(connection, sqlGenerator, modelBuilder, logger)
    {
    }

    public override DbProviderFactory DbProviderFactory => CustomDbProviderFactory.Instance;

    public override IBulkInsert CreateBulkInsert()
    {
        if (IsExternalConnection)
        {
            return new CustomBulkInsert(Connection as CustomConnection);
        }
        else
        {
            return new CustomBulkInsert(ConnectionString);
        }
    }

    public override IBulkUpdate CreateBulkUpdate()
    {
        if (IsExternalConnection)
        {
            return new CustomBulkUpdate(Connection as CustomConnection);
        }
        else
        {
            return new CustomBulkUpdate(ConnectionString);
        }
    }

    protected override IDatabaseSchemaLoader GetSchemaLoader()
    {
        throw new NotImplementedException();
    }
}

public class CustomQuerySqlGenerator : DefaultQuerySqlGenerator
{
    public CustomQuerySqlGenerator(IDatabaseMapper databaseMapper)
        : base(databaseMapper)
    { }
}


public class CustomQueryProviderFactory : IQueryProviderFactory
{
    public Query.IQueryProvider Create(DbConnection connection, IModelBuilder modelBuilder, IDatabaseMapper databaseMapper) =>
        new CustomQueryProvider(connection, new CustomQuerySqlGenerator(databaseMapper), modelBuilder, new Logger());

    public Query.IQueryProvider Create(ConnectionStringSettings connectionString, IModelBuilder modelBuilder, IDatabaseMapper databaseMapper) =>
        new CustomQueryProvider(connectionString, new CustomQuerySqlGenerator(databaseMapper), modelBuilder, new Logger());

    public static void Register()
    {
        QueryProviderFactories.Register<CustomConnection>("System.Data.CustomDb", new CustomQueryProviderFactory());
    }
}

ASP.NET Core extensions

For simple integration into ASP.NET Core projects, the Kros.KORM.Extensions.Asp package was created.

You can use the AddKorm extension method to register IDatabase to the DI container.

public void ConfigureServices(IServiceCollection services)
{
    services.AddKorm(Configuration);
}

The configuration file (typically appsettings.json) must contain a section ConnectionStrings.

  "ConnectionStrings": {
    "DefaultConnection": "Server=servername\\instancename;Initial Catalog=database;Persist Security Info=False;"
  }

If you need to initialize the database for IIdGenerator then you can call InitDatabaseForIdGenerator.

public void ConfigureServices(IServiceCollection services)
{
    services.AddKorm(Configuration)
        .InitDatabaseForIdGenerator();
}

Unit and performance tests

Kros.KORM unit test coverage is more than 87%. There are also some performance test written for Kros.KORM. Here you can see some of their results:

  • Reading of 150 000 records with 25 columns (long strings and guids) from DataTable is finished in about 410 ms.
  • Reading of 1 500 records with 25 columns (long strings and guids) from DataTable is finished in about 7 ms.

kros.korm's People

Contributors

burgyn avatar janoslivka avatar kubinko avatar lukassefcik avatar marianmatula avatar michalau avatar misho048 avatar ml13 avatar ondrix avatar peto-210 avatar petriq avatar satano avatar tereziavrabkova avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

kros.korm's Issues

Modify DbSet to not use CommandGenerator directly

  • Modify DbSet to not use CommandGenerator directly, but only IQueryProvider.
  • CommandGenerator is class for creating commands for DbSet. But in case of database connection, it relies on IQueryProvider. IQueryProvider creates instance of DbCommand, because it knows database connection. Because of that, there is useless method GetCommandForCurrentTransaction.
  • It would be great if the DbSet will be dependent only on IQueryProvider and IQueryProvider will use CommandGenerator inside. Consider deleting CommandGenerator and moving code to IQueryProvider.

NotSupportedException when deleting using LINQ

Library name and version

  • Kros.KORM 4.0-alpha10

Description

Delete throws NotSupportedException.

Steps To Reproduce

public async Task DeleteDocuments(long c)
{
 IDBSet<Document> dbSet = _database.Query<Document>().AsDbSet();
 dbSet.Delete(x => x.CompanyId == c); //Throws exception. Message: The member 'c' is not supported.
 //dbSet.Delete(x => x.CompanyId == 5); //Works fine.
 await dbSet.CommitChangesAsync();
}

Do SQL syntax conversion for ExecuteNonQuery

If you write all your SQL commands in MS Access syntax, you can create your own sql syntax convertor by inheriting SqlServerQuerySqlGenerator class and overriding GenerateSql method.
This method is called every time you use IDatabase.Query or IDatabase.ExecuteScalar, but is not called in case of IDatabase.ExecuteNonQuery.

Master-Detail

  • Modify KORM, so it can read Master-detail.

Something like korm.Query<Person>().Include(p=>p.Addresses)

Flattening of complex (object) properties of an entity

This is detailed info about Support for complex entities, whose properties are another objects from issue #8.

The goal

We want to load (and save) from database entities with complex properties. Complex property is a property which is another object. In its simplest form, say we have this entity Invoice:

public class Invoice
{
    int Id { get; set; }
    string InvoiceNumber { get; set; }
    Address Supplier{ get; } = new Address();
    Address Purchaser { get; } = new Address();
}

public class Address
{
    public string Street { get; set; }
    public string City { get; set; }
    public string Country{ get; set; }
}

And we want to automatically map it to table with these columns:

  • Id
  • InvoiceNumber
  • SupplierStreet
  • SupplierCity
  • SupplierCountry
  • PurchaserStreet
  • PurchaserCity
  • PurchaserCountry

What/how should work

  • Complex properties with injector set must not be flattened.
  • Column name will be defined as concatenated names of properties:
    Supplier.Street -> SupplierStreet
  • Deep flattening of properties:
    Invoice.Supplier.Address.Street -> SupplierAddressStreet
  • Setting specific column name for nested property:
    .Property(p => p.Supplier.Address.Street).HasColumnName("SupplierStreet")
  • Complex properties can be readonly ({ get; }) or read/write ({ get; set; }).
    • If the property is null, instance should be created automatically if it is possible (property is r/w and property's type has parameterless constructor).
  • Can set nested properties as not mapped:
    .Property(p => p.Supplier.Country).NoMap()
  • Can use injectors for nested properties:
    .Property(p => p.Supplier.Country).InjectValue(...)
  • Update readme.

Other considerations

  • Should flattening work automatically and be turned off if not wanted, or vice versa?
  • We will support only fluent configurations for this (no attributes).
  • Can we make whole attribute configuration obsolete?

Add option to increase timeout for sql migrations

Is your feature request related to a problem? Please describe

When migration in SQL script takes longer than default timeout specified in connection string, migration fails and service won't start.

Describe the solution you'd like

Abillity to manually set timeout for migration script to run in config, code or in migration scrip.

Global Query Filter

In many cases, we want to define a global filter to apply to each query. For example: IsDeleted = false, ParentId = 1, UserId = ActiveUser.Id, etc.

It would be great to be able to define it in one place and KORM would automatically add this condition to every query.

public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Entity<Document>()
            .UseQueryFilter(entity => entity.IsDeleted == false && entity.ParentId == 1);
    }
}

KORM will automatically add a condition ((IsDeleted = 0) AND (ParentId = 1) when calling any query using Query<Document>().

It should be possible to do this in all scenarios except for direct sql calls _database.Query<Document>().Sql("SELECT * FROM DOCUMENT");

Ignoring global filters

If I need to call a query without these conditions, I must explicitly say:

_database.Query<Document>()
    .IgnoreQueryFilters()
    .ToList();

Things to think about

  • We can have multiple classes to retrieve data from the same table. For example: DocumentHeader, DocumentVerySpecialDto, etc. It would be nice to define this filter for all these classes.

  • Create prototype

  • Allow append where to existing where

  • Configuration

  • Add necessery information into query provider

  • Use defined filter

  • Ignore query filter

  • Unit and integration tests

  • Xml comments

  • Information to readme file.

  • Add info into KORM.MsAccess. (Constructor of QueryProvider was changed).

KORM does not call converter if property value is null

Library name and version

  • Kros.KORM 4.4.1

Description

I used converters for string properties but they are not called if value of the property is null. Similar bug (#19) which should already be fixed exists.

Steps To Reproduce

To simulate bug you can use project https://github.com/marek011/KormBugSimulationProject (connection string in appsettings.json has to be specified). In DatabaseConfiguration two ways of configuring converter are used (none is working for null values).

modelBuilder.Entity<WeatherForecast>()
   .HasTableName(WeatherForecastsTableName)
   .UseConverterForProperties<string>(NullAndTrimStringConverter.ConvertNullAndTrimString)
   .Property(u => u.Temperatures).UseConverter<JsonToListConverter<string>>();

You can check that strings are trimmed but null values are not converted to empty string. JsonToListConverter.Convert is not called for null values.

Expected behavior

Converters should be called for null values.

Actual behavior

Converters are not called for null values.

Nullable nvarchar/varchar column with null value mapping to string record

Library name and version

  • Kros.KORM 4.1.1

Description

Mapping null value to record string property is not working. Mapping to class is working as expected.

Steps To Reproduce

  1. Create table with NULLable string column (for example nvarchar)
  2. Insert testing data where column value is NULL
  3. Create query with KORM and map as record.

you can check my tests to reproduce issue:
https://github.com/rudolfficek/Korm.Query.Tests/blob/16c61b02ca72d82ae7f3f113a4eacb1eb140f962/Korm.Query.Tests/QuerieTests/BuiltQueryWithNullableShould.cs#L78

Expected behavior

Null value should be correcly mapped.

Actual behavior

exception thrown. System.Data.SqlTypes.SqlNullValueException : Data is Null. This method or property cannot be called on Null values.

use OUTPUT INTO on SqlServer for returning columns

When inserting an entity, query was using SQL Server's OUTPUT clause to return values for columns with primary key (usually Id).
Query preview:

INSERT INTO table (columns...) OUTPUT INSERTED.Id VALUES (values...);

This breaks when the target table has a trigger defined, with the following error:

The target table '...' of the DML statement cannot have any enabled triggers
if the statement contains an OUTPUT clause without INTO clause.

The correct query is:

DECLARE @OutputTable TABLE (columns...);
INSERT INTO table (columns...) OUTPUT INSERTED.Id, ... INTO @OutputTable VALUES (values...);
SELECT * FROM @OutputTable;

Sum throws error when there are no data to sumarize

Library name and version

  • Kros.KORM 3.7.2

Description

Calling _database.Query().Where(person => person.TermId == termId).Sum(person => person.Count); throws SqlNullValueException: Data is Null when there are no data with required conditions.

Expected behavior

Sum function should return 0

Actual behavior

System.Reflection.TargetInvocationException: 'Exception has been thrown by the target of an invocation.'
SqlNullValueException: Data is Null. This method or property cannot be called on Null values.

Workaround:
_database.Query().Where(person => person.TermId == termId).AsEnumerable().Sum(person => person.Count);

Order by nullable column

Library name and version

  • Kros.KORM 3.0

Description

Ordering by nullable datetime column caused exception, because query was incorrectly translate into SQL query.
IIF(($it.DueDate == null), null, $it.DueDate) was transled as (DueDate IS NULL)NULLDueData in function Kros.KORM.Query.Sql.DefaultQuerySqlGenerator.VisitOrderBy
MicrosoftTeams-image (7)

Steps To Reproduce

  1. Set up one column as DateTime?
  2. Order by this column

Expected behavior

IIF(($it.DueDate == null), null, $it.DueDate) will be transled as `IIF((DueDate IS NULL),NULL,DueData)

Actual behavior

Trown exception:
Microsoft.Data.SqlClient.SqlException: 'Incorrect syntax near the keyword 'IS'.'

Generate queries with columns in parentheses

  • Consider entity with property Order and also the same column in the database. KORM will generate query like : SELECT Col1, Col2, Order, Col3 FROM Table. This query will not be executed, because Order is reserved keyword.
  • All columns in query need to be closed in parentheses. If someone writes whole query on its own, query shoud be write correctly (with parentheses).

Implement converter for strings to convert null to empty string

The converter is useful for string values, which are NOT NULL in database. Now the users have two options how to handle this:

  • Implement setters for such properties with logic to convert null to string.Empty. This is the safest option, but we cannot use auto properties and the converting logic is just a smell. So DTOs are not as clean as they could be.
  • Explicitly set values of such properties to empty string when used. This can be annoying if I do not want to set real value and leave it to be default.

The real power of this will be if used together with #21. Just one attribute on class, no logic in property setters and no setting empty strings when using DTOs.

The converter itself could be a little smarter and have some settings (if convert just null or even whitespace string and trim value).

KORM - ideas and suggestions for new features

General info

Do not close this issue. This is a "meta issue". It is just for writing down any ideas and suggestions for KORM. Ideas here, once consulted, will be moved to separate issues.

Simpler Delete of entities

Implemented in #40.

Now, if we want to delete an entity, we can use direct SQL statement (ExecuteNonQuery), or DbSet. The disadvantage of ExecuteNonQuery is, that it does not have ane knowledge about entity type, so the SQL must be hand written. Disadvantage of DbSet is that we need to create an instance of the entity, just to have its ID. I'd like to have something which can be used without creating an entity instance:

_database.Delete<EntityType>(123)

We could support delete with condition (instead of just primary key):

_database.Delete<EntityType>(item => item.ParentId == 123);

Things to think about:

  • Where to implement this (IDbSet, IDatabase...). And it should at least support int and long primary keys (generic of <TEntity, TKey>?), but better any keys (what about composite ones?).
  • What kind of expressions to support when deleting with condition and throw some exception if we cannot translate it into SQL.

Value generators for columns

Implemented in #41.

Now we have some support for generated primary keys. But we only support int (no long). It is achieved by Key attribute. It would be better to have something like ValueGenerator attribute, which will generate value for column on INSERT. And it could be used for any column, not just primary key.

Timestamp column

This would be special column (for start marked with some attribute) of DateTime/DateTimeOffset type. Value for this column would be automatically set to current date and time on every INSERT or UPDATE.

Soft delete

It is common to use so-called soft delete instead of directly deleting a database record. The record is not deleted, it is only marked with IsDeleted flag.

It would be nice to support it directly in KORM.

I suggest something like this:

public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Entity<Document>()
            .UseSoftDelete(entity => entity.IsDeleted);
    }
}

After calling dbSet.CommitChanges(); KORM will call update IsDeleted = true.

Global Query Filters

Implemented in #47

Discused in #42.

In many cases, we want to define a global filter to apply to each query. For example: IsDeleted = false, ParentId = 1, UserId = ActiveUser.Id, etc.

It would be great to be able to define it in one place and KORM would automatically add this condition to every query.

public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Entity<Document>()
            .UseSoftDelete(entity => entity.IsDeleted)
            .HasQueryFilter(entity => entity.IsDeleted == false && entity.ParentId == 1);
    }
}

KORM will automatically add a condition ((IsDeleted = 0) AND (ParentId = 1) when calling any query using Query<Document>().

It should be possible to do this in all scenarios except for direct sql calls _database.Query<Document>().Sql("SELECT * FROM DOCUMENT");

Ignoring global filters

If I need to call a query without these conditions, I must explicitly say:

_database.Query<Document>()
    .IgnoreQueryFilters()
    .ToList();

Things to think about

  • We can have multiple classes to retrieve data from the same table. For example: DocumentHeader, DocumentVerySpecialDto, etc. It would be nice to define this filter for all these classes.
  • We need overload with string condition HasQueryFilter(string whereCondition, params object[] args);

Use ConnectionStrings section with named connection strings

Implemented in Kros.KORM.Extensions.Asp repository.

Filed in Kros.KORM.Extensions.Asp repository.

We use our own setup of connection string in setting. We require ConnectionString section with ProviderName and ConnectionString subkeys. We could use default connection strings settings (as Entity Framework does it). The section name is ConnectionStrings and subkeys are name of connection strings. Provider can be integrated in connection string itself. When creating IDatabase, connection string name will be specified. If the name is not specified, default name DefaultConnection will be used.

Use different kind of configuration than attributes

Implemented in #18

Now some configuration is allowed only using attributes (Key, Alias). Something can be changed using custom ModelMapper. Maybe we can think about different kind of configuration instead of attributes.

Use new feature of C# 8.0 - Async Streams for async loading data

Use new C# 8.0 feature Async Streams for async obtaining data by KORM.

IEnumerable<Person> people = await _database.Query<Person>().AsAsync();

Support for LongCount method

Is your feature request related to a problem? Please describe

When using lib Microsoft.AspNetCore.OData and requesting $count of elements in entity set, the lib requests LongCount on IQueryable. Without LongCount method support the query fails on System.NotSupportedException: The method 'LongCount' is not supported.

MicrosoftTeams-image (1)

Describe the solution you'd like

Add support for LongCount in DefaultQuerySqlGenerator class.

Support query timeout

Add possibility to set query timeout. Currently is impossible make long time running query (over 30s) to database because of Execution Timeout expired exception.

Bug: Cannot insert explicit value for identity column in table 'TableName' when IDENTITY_INSERT is set to OFF

Library name and version

  • Kros.KORM 4.1.0

Description

It is not enough to define in the DatabaseConfiguration for the entity:
.AutoIncrement (autoIncrementType: AutoIncrementMethodType.Identity).
When adding an item. It falls: Cannot insert explicit value for identity column in table 'TableName' when IDENTITY_INSERT is set to OFF

Steps To Reproduce

  1. Initialization script:
IF NOT EXISTS (SELECT 1 FROM sysobjects WHERE NAME='Users' and xtype='U')
BEGIN
    CREATE TABLE [dbo].[Users](
        [Id] [bigint] IDENTITY(1,1) NOT NULL,
        [Email] [nvarchar](255) NOT NULL,

        CONSTRAINT [PK_Users] PRIMARY KEY CLUSTERED ([Id] ASC)
        WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
    ) ON [PRIMARY]
END
  1. DatabaseConfiguration:
public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
{
     modelBuilder.Entity<User>()
     .HasTableName(UsersTableName)
     .HasPrimaryKey(f => f.Id)
     .AutoIncrement(autoIncrementType:AutoIncrementMethodType.Identity);
}
  1. Entity:
[Alias("Users")]
    public class User
    {
        /// <summary>
        /// Id.
        /// </summary>
        public long Id { get; set; }

        /// <summary>
        /// Email.
        /// </summary>
        public string Email { get; set; }
    }

4.Used:

var users = _database.Query<User>().AsDbSet();
users.Add( new User() { Email = "[email protected]" });
users.CommitChanges();

Actual behavior:

When I add an annotation to an id, it adds an item.

[Alias("Users")]
public class User
{
    /// <summary>
    /// Id.
    /// </summary>
   [Key(autoIncrementMethodType: AutoIncrementMethodType.Identity)]
    public long Id { get; set; }

    /// <summary>
    /// Email.
    /// </summary>
    public string Email { get; set; }
}

Use different kind of configuration than attributes

Starting discussion about Use different kind of configuration than attributes from #8.

Current state

Now some configuration is allowed only using attributes (Key, Alias). Something can be changed using custom ModelMapper.

[Alias("Foo")]
public class Foo
{
    [Key("PK_Foo", AutoIncrementMethodType.Identity)]
    public int Id { get; set; }

    [Alias("Name")]
    public string FirstName { get; set; }

    [NoMap]
    public string LastName { get; set; }

    [Converter(typeof(AddressConverter))]
    public IEnumerable<string> Addresses { get; set; }

    [NoMap]
    public IEmailService EmailService { get; set; }
}

Available attributes:

  • Alias - Define name for database object (table, column).
  • Key - Define primary key for table.
  • NoMap - When property is mark with this attribute, then KORM ignore this column.
  • Converter - Define converter for converting values between database and CLR.

When want use property injection, we must use ModelMapper:

Database.DefaultModelMapper
    .InjectionConfigurator<Foo>()
        .FillProperty(p => p.EmailService, () => new EmailService());

In many scenarios it's OK. However, there are scenarios where we want to have a model definition and mapping it to a database separate. For example, if you want to have entities in domain layer and mapping in infrastructure layer.

Design

The proposal is based on how it is in EF. Have a configuration class where, we are fluent define model mapping to database.

public class DatabaseConfiguration: DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Entity<Foo>()
            .HasTableName("Foo")
            .HasPrimaryKey(f => f.Id)
                .HasAutoIncrementMethodType(AutoIncrementMethodType.Identity)
                .HasConstraintName("PK_Foo")
                .Entity
            .Property(f => f.FirstName)
                .HasColumnName("Name")
                .Entity
            .Property(f => f.LastName)
                .NoMap()
                .Entity
            .Property(f => f.Addresses)
                .HasConverter<AddressConverter>()
                .Entity
            .Property(f => f.EmailService)
                .FillProperty(() => new EmailService());
    }
}

Uses in ASP.NET Core applications:

public void ConfigureServices(IServiceCollection services)
{
  services.AddKorm(Configuration)
	.AddKormMigrations(Configuration)
        .UseDatabaseConfiguration<DatabaseCofiguration>()
	.Migrate();
}

Implementation

I tried to explore it.
It is possible implemented into KORM.

Add support for GUID primary key

Currently we support only integer primary keys (either SQL Server auto-increment, or our own sequences). We need to support GUID primary keys. The value of primary key can be set explicitly on entities. If it is not set, it will be generated before insert.

Query with ternary operator in LINQ Where clause fails

Library name and version

  • Kros.KORM 6.2.0

Description

When applying OData query options to IQueryable object, the OData self creates expressions for Where, OrderBy and other parts of a LINQ query. In some cases it produces Where expressions with ternary operator, which KORM fails to parse into SQL command.

Steps To Reproduce

var searchtext = "kro";
var test2 = _database.Query<MovementsOverviewSearchResult>()
    .Where(x => (x.PartnerName == null ? null : x.PartnerName.ToLower()) == null || 
        x.PartnerName.ToLower().Contains(searchtext))
    .OrderByDescending(x => x.MovementDate);

I know the Where condition in example is not 'optimal', but something like that OData produces for $filter=contains(tolower(PartnerName) 'kro') query string param.

Expected behavior

Parsed WHERE clause from example above should be like

WHERE ((CASE WHEN PartnerName IS NULL THEN NULL ELSE LOWER(PartnerName) END) IS NULL) OR (LOWER(PartnerName) LIKE '%' + @1 + '%')))

Also comparing something to NULL should be done by IS NULL, not = NULL. In TSQL comparing NULL = NULL produces FALSE. See separate issue #110 .

Actual behavior

The example provided above produces this SELECT statement:

SELECT ...  -- colums omitted   
FROM ...    -- table name omitted
WHERE ((((PartnerName = NULL)NULLLOWER(PartnerName) = NULL) OR (LOWER(PartnerName) LIKE '%' + @1 + '%'))) ORDER BY MovementDate DESC
  WITH PARAMETERS (kro)

Notice NULLLOWER in the WHERE clause. That is the place where ternary operator should be - well, or rather (CASE WHEN ... THEN ... ELSE ... END).

The query fails on SqlException: Incorrect syntax near 'NULLOWER'. Incorrect syntax near 'PartnerName'.

Create InMemoryProvider for unittesting

It would be nice to have some type of InMemoryProvider where I can pass data and I will be able to mock IDatabase.
Right now its almost impossible to mock it.

We created some quick solution, that should be refactored and maybe used.

It provide get methods. It is able to returns all data and for other (Any, Where, ...) it returns just first entity. Other methods are NotImplemented yet.

/// <summary>
/// Class for mocking KORM IDatabase
/// </summary>
public static class InMemoryDatabaseHelper
{
    /// <summary>
    /// Create simple in memory mock of KORM with given data.
    /// </summary>
    /// <typeparam name="TData">Type of data.</typeparam>
    /// <param name="data">Data to be mocked.</param>
    /// <returns></returns>
    public static IDatabase BuildInMemory<TData>(IEnumerable<TData> data)
        => Database.Builder.UseConnection("memory", "test").UseQueryProviderFactory(CreateProvider(data)).Build();

    private static InMemoryQueryProviderFactory<TData> CreateProvider<TData>(IEnumerable<TData> data)
        => new InMemoryQueryProviderFactory<TData>(data);
}

internal class InMemoryQueryProviderFactory<TData> : IQueryProviderFactory
{
    private readonly IEnumerable<TData> _data;

    public InMemoryQueryProviderFactory(IEnumerable<TData> data)
    {
        _data = data;
    }

    public KORM.Query.IQueryProvider Create(DbConnection connection, IModelBuilder modelBuilder, IDatabaseMapper databaseMapper)
    {
        return new InMemoryQueryProvider<TData>(_data);
    }

    public KORM.Query.IQueryProvider Create(ConnectionStringSettings connectionString, IModelBuilder modelBuilder, IDatabaseMapper databaseMapper)
    {
        return new InMemoryQueryProvider<TData>(_data);
    }
}

internal class InMemoryQueryProvider<TData> : KORM.Query.IQueryProvider
{
    private const string NotImplementedError = "This method was not implemented yet. If you need to mock it please implement it by yourself.";
    private readonly IEnumerable<TData> _data;

    public InMemoryQueryProvider(IEnumerable<TData> data)
    {
        _data = data;
    }

    public IEnumerable<T> Execute<T>(IQuery<T> query) => _data.Cast<T>();

    public IQueryable CreateQuery(Expression expression) => new InMemoryQuery<TData>(this, _data, expression);

    public IQueryable<TElement> CreateQuery<TElement>(Expression expression) => (IQueryable<TElement>)CreateQuery(expression);

    public object Execute(Expression expression) => _data.FirstOrDefault();

    public TResult Execute<TResult>(Expression expression) => (TResult)Execute(expression);

    #region NotImplemented
    public DbProviderFactory DbProviderFactory => throw new NotImplementedException(NotImplementedError);

    public ITransaction BeginTransaction(IsolationLevel isolationLevel) => throw new NotImplementedException(NotImplementedError);

    public IBulkInsert CreateBulkInsert() => throw new NotImplementedException(NotImplementedError);

    public IBulkUpdate CreateBulkUpdate() => throw new NotImplementedException(NotImplementedError);

    public IIdGenerator CreateIdGenerator(string tableName, int batchSize) => throw new NotImplementedException(NotImplementedError);

    public void Dispose() => throw new NotImplementedException(NotImplementedError);

    public Task ExecuteInTransactionAsync(Func<Task> action)
        => throw new NotImplementedException(NotImplementedError);

    public int ExecuteNonQuery(string query)
        => throw new NotImplementedException(NotImplementedError);

    public int ExecuteNonQuery(string query, CommandParameterCollection parameters)
        => throw new NotImplementedException(NotImplementedError);

    public Task<int> ExecuteNonQueryAsync(string query)
        => throw new NotImplementedException(NotImplementedError);

    public Task<int> ExecuteNonQueryAsync(string query, params object[] paramValues)
        => throw new NotImplementedException(NotImplementedError);

    public Task<int> ExecuteNonQueryAsync(string query, CommandParameterCollection parameters)
        => throw new NotImplementedException(NotImplementedError);

    public int ExecuteNonQueryCommand(IDbCommand command)
        => throw new NotImplementedException(NotImplementedError);

    public Task<int> ExecuteNonQueryCommandAsync(DbCommand command)
        => throw new NotImplementedException(NotImplementedError);

    public object ExecuteScalar<T>(IQuery<T> query)
        => throw new NotImplementedException(NotImplementedError);

    public object ExecuteScalarCommand(IDbCommand command)
        => throw new NotImplementedException(NotImplementedError);

    public Task<object> ExecuteScalarCommandAsync(DbCommand command)
        => throw new NotImplementedException(NotImplementedError);

    public TResult ExecuteStoredProcedure<TResult>(string storedProcedureName)
        => throw new NotImplementedException(NotImplementedError);

    public TResult ExecuteStoredProcedure<TResult>(string storedProcedureName, CommandParameterCollection parameters)
        => throw new NotImplementedException(NotImplementedError);

    public DbCommand GetCommandForCurrentTransaction() => throw new NotImplementedException(NotImplementedError);

    public void SetParameterDbType(DbParameter parameter, string tableName, string columnName)
        => throw new NotImplementedException(NotImplementedError);

    public bool SupportsIdentity() => throw new NotImplementedException(NotImplementedError);

    public bool SupportsPrepareCommand() => throw new NotImplementedException(NotImplementedError);
    #endregion
}

internal class InMemoryQuery<TData> : IQueryable<TData>
{
    private readonly IEnumerable<TData> _data;

    public Type ElementType => typeof(TData);

    public Expression Expression { get; private set; }

    public System.Linq.IQueryProvider Provider { get; private set; }

    public InMemoryQuery(InMemoryQueryProvider<TData> queryProvider, IEnumerable<TData> data)
    {
        Provider = queryProvider;
        Expression = data.AsQueryable().Expression;
        _data = data;
    }

    public InMemoryQuery(InMemoryQueryProvider<TData> queryProvider, IEnumerable<TData> data, Expression expression)
    {
        Provider = queryProvider;
        Expression = expression;
        _data = data;
    }

    public IEnumerator<TData> GetEnumerator() => _data.GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

Mock IDatabase in UnitTests:

var data = new List<Note>() { new Note() { Id = 1, Note = "Hello in memory KORM." }};
var database = InMemoryDatabaseHelper.BuildInMemory(data);

public class Note {
  public int Id { get; set; }
  public string Note { get; set; }
}

Simpler Delete of entities

Starting discussion about Simplier Delete of entities from #8.


Now, if we want to delete an entity, we can use direct SQL statement (ExecuteNonQuery), or DbSet. The disadvantage of ExecuteNonQuery is, that it does not have ane knowledge about entity type, so the SQL must be hand written. Disadvantage of DbSet is that we need to create an instance of the entity, just to have its ID. I'd like to have something which can be used without creating an entity instance:

_database.Delete<EntityType>(123)

We could support delete with condition (instead of just primary key):

_database.Delete<EntityType>(item => item.ParentId == 123);

Things to think about:

  • Where to implement this (IDbSet, IDatabase...). And it should at least support int and long primary keys (generic of <TEntity, TKey>?), but better any keys (what about composite ones?).
  • What kind of expressions to support when deleting with condition and throw some exception if we cannot translate it into SQL.

ToDo:

  • Delete by id
    • Check if id has the same type as primary key property
  • Integration test for Delete by id
  • Delete by condition
    • Prototype
    • Implementation
    • Unit and integration tests
    • xml comments
  • Extension for easy calling Delete and Add operation.
  • Info to readme file.

Creating an IDatabase from service provider throws exception if fluent configuration is used

Library name and version

  • Kros.KORM.Extensions.Asp 1.1.0-alfa

Description

If KORM is added with fluent database configuration, it throws an exception when trying to get IDatabase from service provider.

Steps To Reproduce

  1. Create a database configuration using fluent API:
public class DatabaseConfiguration : DatabaseConfigurationBase
{
    public override void OnModelCreating(ModelConfigurationBuilder modelBuilder)
    {
        modelBuilder.Entity<Invoice>()
            .HasTableName("Invoices")
            .UseConverterForProperties<string>(NullAndTrimStringConverter.ConvertNullAndTrimString)
    }
}
  1. Use this configuration when adding KORM to service provider:
public override void ConfigureServices(IServiceCollection services)
{
    services.AddKorm(configuration).UseDatabaseConfiguration<DatabaseConfiguration>();
}
  1. Try to use IDatabase (just inject it in some controller). Application throws when accessing that controller.

Additional info

The bug is in KormBuilder class. The problem is, that KORM is added as scoped to the container (which is good), but every time an instance is asked for, it is being built again. But Build() method is called on the same (global) instance of IDatabaseBuilder. This way, fluent configuration API is is called multiple times for the same ModelConfigurationBuilder instance. There are couple of checks in fluent config API, that prevents duplicate configuration and these checks throw the exception.

Generic query not working correctly in generic method with generic type constraint for interface with readonly fields and where clausule

Library name and version

  • Kros.KORM 3.0

Description

Query fails when using generic method with generic type constrait for interface like where T : IInterface with readonly members. Where this generic type is used as generic type of Database.Query<T> and Where() method is used.

Steps To Reproduce

  1. Create generic method with query in it and interface must containts field Id
  private IResource GetResourceById<TResource>(string tableName, int id) where TResource : IResource
           => _database.Query<TResource>()
                 .From(tableName)
                 .FirstOrDefault(x => (x.Id == id));
  1. Call this method, with correct generic type, implemeting mentioned interfaca
GetResourceById<MyResource>(mytable, id);
  1. Call fails on NullReferenceException.

Expected behavior

Correctly return selected item byt id, without throwing an exception.

Actual behavior

Cause of problem is readonly field Id on interface.
When KORM is creating TableInfo in ConventionModelMapper.CreateTableInfo() for where clausule. It is reading only writable properties from given type. Type in this case is interface from constraint, instead of correct generic type given for query.

KORM in this point should "know" it is creating only where clausule and properties doesn't have to be writable.

BulkUpdate on table with identity column as primary key updates wrong rows

Library name and version

  • Kros.KORM 4.0 - alfa 11

Description

BulkUpdate on table, which has primary key column as IDENTITY, updates wrong rows. It updates rows with id smaller or equal as is count of rows updated by bulk update.
It is due to fact, that temporary table for BulkInsert copies DENTITY settings on Id column too, so during BulkInsert are existing values of Id column replaced by new Id values generated by sql server. BulkUpdate then doesn't join rows in temporary table with correct changed rows in original table, but joint them with other rows and updates them. So existing rows stay untouched and it updates wrong rows based on new Ids in temporary table.

Steps To Reproduce

  1. create table with primary key column as identity
CREATE TABLE [dbo].[Foo](
[Id] [bigint] NOT NULL,
[Name] [nvarchar](100) NOT NULL,
[Description] [nvarchar](250) NOT NULL
CONSTRAINT [PK_Foo] PRIMARY KEY CLUSTERED ([Id] ASC)
    WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
  1. Insert some data
INSERT INTO FOO VALUES ('FooName1', 'FooDescription1');
INSERT INTO FOO VALUES ('FooName2', 'FooDescription2');
INSERT INTO FOO VALUES ('FooName3', 'FooDescription3');
INSERT INTO FOO VALUES ('FooName4', 'FooDescription4');

So In Foo table will be these rows

1  FooName1  FooDescription1
2  FooName2  FooDescription2
3  FooName3  FooDescription3
4  FooName4  FooDescription4
  1. Change some existing data and save them with BulkUpdate
Kros.KORM.Query.IDbSet<Foo> dbSet = _DBS.Query<Foo>().AsDbSet();
dbSet.Edit(new Foo() {
  Id = 3,
  Name = "NewFooName3",
  Description = "NewFooDescription3"
})
dbSet.BulkUpdate();
  1. Check your table and you will see
1  NewFooName3  NewFooDescription3
2  FooName2         FooDescription2
3  FooName3         FooDescription3
4  FooName4         FooDescription4

Foo with Id = 3 is untouched even when you specify Id = 3 in edit
Foo with Id = 1 is overwritten instead of Foo with Id = 3

Program fails at Microsoft.Data.SqlClient.SqlException: 'Must declare the scalar variable "@InvoiceTemplateId".

Library name and version

  • Kros.KORM 4.1.2

Description

Program fails at Microsoft.Data.SqlClient.SqlException: 'Must declare the scalar variable "@InvoiceTemplateId".
Statement(s) could not be prepared.' The exception throws if Select statement contains field with another database column name as defined by model.
Program not throws the exception without Select statement which specified concrete columns.

Steps To Reproduce

string[] columns = new[] { nameof(InvoiceTemplateSettings.DocumentId), nameof(InvoiceTemplateSettings.TemplateName) };
IDbSet<InvoiceTemplateSettings> dbSet = _database.Query<InvoiceTemplateSettings>().Select(columns).AsDbSet();
var documentFlat = new InvoiceTemplateSettings() {
    DocumentId = 7,
    TemplateName = "zmenená šablónka"
};
dbSet.Edit(documentFlat);
await dbSet.CommitChangesAsync();

And the console show

UPDATE [InvoiceTemplateSettings]
SET [TemplateName] = @TemplateName
WHERE ([InvoiceTemplateId] = @InvoiceTemplateId)
WITH PARAMETERS (zmenená šablónka)

Model is

public class InvoiceTemplateSettings
{
    /// <summary>
    /// Invoice template ID.
    /// </summary>
    public long DocumentId { get; set; }
    public string TemplateName { get; set; }
}

Model builder is

modelBuilder.Entity<Entities.InvoiceTemplateSettings>()
    .HasTableName(DatabaseTableNames.InvoiceTemplateSettingsTableName)
    .HasPrimaryKey(f => f.DocumentId)
    .Property(f => f.DocumentId).HasColumnName("InvoiceTemplateId");

SQL

CREATE TABLE [dbo].[InvoiceTemplateSettings] (
    [InvoiceTemplateId] [bigint] NOT NULL,
    [TemplateName] [nvarchar](250) NOT NULL,
    CONSTRAINT [UK_InvoiceTemplateSettings] UNIQUE NONCLUSTERED (
        [InvoiceTemplateId] ASC
    )
    WITH (
        PAD_INDEX = OFF,
        STATISTICS_NORECOMPUTE = OFF,
        IGNORE_DUP_KEY = OFF,
        ALLOW_ROW_LOCKS = ON,
        ALLOW_PAGE_LOCKS = ON,
        OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
    ) ON [PRIMARY]
)

If I replace the line with the query without .Select(columns) statement:

IDbSet<InvoiceTemplateSettings> dbSet = _database.Query<InvoiceTemplateSettings>().AsDbSet();

the code runs success without exception.

Expected behavior

I think this is a bug because I must use the name of database column, not column of model. I would expect to use model column name.

Use Dynamic methods for DbSet

KORM use reflection for generating insert/update commands and setting generated ids into entities. (starting method is CommitChangesAddedItemsAsync in DbSet)
We can use DynamicMethods as in DynamicMethodModelFactory. This can improve performance.

Create extension methods ExecuteWithTempTable(Async)

In many cases we need to execute some action over data specified by list of ids (or other simple values). It would be nice to have such extension methods for IDatabase in KORM, user just passes a list (or dictionary) of values, and action, that should be executed over data defined in input parameter.

This method should:

  • create temp table according to input values (with use of BulkInsert)
  • execute defined action (query, update command, ...)
  • return result of action, if needed
  • drop temp table
  • execute all this steps in transaction

Add DB extension methods for entities collection

KORM contains database extension methods AddAsync, EditAsync and DeleteAsync that simplify process of adding, editing and deleting of entity by encapsulating and hiding all necessary codebehind (such as manipulating IDbSet).

However, at this time, these methods accept only single entity. It would be nice to have overloads of these methods that would accept IEnumerable of entities. And also extension methods that would encapsulate bulk insert and bulk update in similar manner.

Comparing with NULL is always FALSE

Library name and version

  • Kros.KORM 6.2.0

Description

KORM fails to parse correct condition for NULL comparison.

Steps To Reproduce

Example:

var test = _database.Query<MovementsOverviewSearchResult>()
                .Where(x => x.PartnerName != null);

Expected behavior

LINQ query above should be parsed

SELECT ... FROM ... Where (PartnerName IS NOT NULL)

Actual behavior

KORM parses the query as

SELECT ... FROM ... Where (PartnerName <> NULL)

Comparing NULL with anything using operators =, !=, <> ALWAYS produces FALSE. Even NULL = NULL is FALSE. Therefore, IS and IS NOT keywords should be parsed instead when comparing with NULL.

QueryFilter - System.MissingMethodException

After I implemented QueryFilter I got this exception:

System.MissingMethodException: 'Method not found: 'Kros.KORM.Query.IProjectionQuery`1<System.__Canon> Kros.KORM.Query.IQuery`1.From(System.String)'.'

This is what i added to DatabaseConfiguration.cs file:

      modelBuilder.Table(PartnersTableName)
            .UseQueryFilter<Partner>(entity => !entity.IsDeleted);

Then I tried to use it in this query:

_database.Query<Infrastructure.Entities.Partner>()
                .FirstOrDefault(i => i.Id == request.Id)

Support converters for all properties of specified type

Now if I want to use the same converter for several properties, I have to annotate them all. It would be nice to have a converter attribute for table, which will be used for all properties of specified type.

For now, the only useful example I have is, that I want for all string properties to convert null values to empty string (because string columns in database are NOT NULL). And to have just one attribute on class is cleaner than to have attributed ever property.

We have to take care about:

  • If the property itself has Converter attribute it has precedence over attribute on class.
  • We need to say somehow that the property does not want to use converter from class (NoConverterAttribute?). For example if I have 10 string properties and I want to use converter for all of them except one.

Add option to specify columns that should not be updated

KORM now allows to specify which columns should be updated by Edit. It would be nice to have an option to set which columns should not be updated by Edit.

For example:
I have a class with a property that is managed separately from other class properties. I want to map this property to a database column (so NoMap is not an option for me), but I do not want to change value of this column in most of the updates, because the property value might not be assigned (yet), and therefore the Edit would assign an empty value to corresponding database column (effectively erasing the value).

I propose to add method EditExcept, that would act as Edit with a slight difference, that it would take as parameters a param array of column names, that should not be updated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.