tombatron / nrejson

A set of extensions for interacting with the ReJSON module for Redis.

NReJSON

Overview

NReJSON is a series of extension methods for the StackExchange.Redis library that will enable you to interact with the Redis module RedisJSON. This is made possible by the Execute and ExecuteAsync methods already present in the SE.Redis library.

The following blog post by Marc Gravell was the inspiration behind this: StackExchange.Redis and Redis 4.0 Modules. He even has an example of how to call a command from the RedisJSON module!

Installation

PM> Install-Package NReJSON -Version 4.0.0

Usage

I'm assuming that you already have the RedisJSON module installed on your Redis server.

You can verify that the module is installed by executing the following command:

MODULE LIST

If RedisJSON is installed you should see output similar to the following:

1) 1) "name"
   2) "ReJSON"
   3) "ver"
   4) (integer) 10001

(The version of the module installed on your server obviously may vary.)

Major Changes in Version 4.0

• Requires RedisJson 2.0.

• All deprecated RedisJson commands have been removed.

• Introduced PathedResult<TResult> in order to handle commands which can return multiple results in the format of a JSON array based on a provided JSONPath specification.

• BREAKING CHANGE : The generic overloads for JsonGet and JsonGetAsync now return an instance of PathedResult<TResult>.

• BREAKING CHANGE : JsonIncrementNumber and JsonIncrementNumberAsync now return an instance of PathedResult<double?>.

• BREAKING CHANGE : JsonMultiplyNumber and JsonMultiplyNumberAsync now return an instance of PathedResult<double?>.

• BREAKING CHANGE : JsonAppendJsonString and JsonAppendJsonStringAsync now return int?[] to support multiple JSONPath matches.

• BREAKING CHANGE : JsonStringLength and JsonStringLengthAsync now return int?[] to support multiple JSONPath matches.

• BREAKING CHANGE : JsonArrayAppend and JsonArrayAppendAsync now return int?[] to support multiple JSONPath matches.

• BREAKING CHANGE : JsonArrayIndexOf and JsonArrayIndexOfAsync now return int?[] to support multiple JSONPath matches.

• BREAKING CHANGE : JsonArrayInsert and JsonArrayInsertAsync now return int?[] to support multiple JSONPath matches.

• BREAKING CHANGE : JsonArrayLength and JsonArrayLengthAsync now return int?[] to support multiple JSONPath matches.

• BREAKING CHANGE : JsonArrayPop and JsonArrayPopAsync now return string?[] or TResult[] for the generic overloads to support multiple JSONPath matches.

• BREAKING CHANGE : JsonArrayTrim and JsonArrayTrimAsync now return int?[] to support multiple JSONPath matches.

• Removed ability to associate a JSON object with an index using the JsonSet and JsonSetAsync methods.

• Changed param array in JsonArrayAppend and JsonArrayAppendAsync to be an array of type object.

• Changed jsonScalar parameter in JsonArrayIndexOf and JsonArrayIndexOfAsync to be of type object.

• Changed param array in JsonArrayInsert and JsonArrayInsertAsync to be an array of type object.

Major Changes in Version 3.0

In version 3.0 support for serialization and deserialization was added in the form of new generic overloads for the following extension methods:

• JsonGet/JsonGetAsync
• JsonMultiGet/JsonMultiGetAsync
• JsonSet/JsonSetAsync
• JsonArrayPop/JsonArrayPopAsync
• JsonIndexGet/JsonIndexGetAsync

In order to leverage the serialization/deserialization support you must create an implementation of the ISerializerProxy interface. The following is a sample implementation taken from the integration tests:

public sealed class TestJsonSerializer : ISerializerProxy
{
    public TResult Deserialize<TResult>(RedisResult serializedValue) =>
        JsonSerializer.Deserialize<TResult>(serializedValue.ToString());

    public string Serialize<TObjectType>(TObjectType obj) =>
        JsonSerializer.Serialize(obj);
}

Once that is implemented it can be assigned to the static property SerializerProxy found in the static class NReJSONSerializer. The following is an example of how to do this:

NReJSONSerializer.SerializerProxy = new TestJsonSerializer();

If this isn't setup before leveraging the extension methods that make use of it, an NReJSONException will be thrown in order to remind you that it needs to be done.

The result type for the following methods has change to OperationResult:

• JsonSet/JsonSetAsync
• JsonIndexAdd/JsonIndexAddAsync
• JsonIndexDelete/JsonIndexDeleteAsync

The OperationResult is a struct that will return and contain whether or not the operation was successful, and will also contain the raw response from Redis. This type is implicitly convertable to bool so it can be used in operations like:

var result = await db.JsonSetAsync(key, obj);

if (result)
{
   // Do something if there was success...
} 
else
{
   // Do something if there wasn't success...
}

Last but not least, we have a new result type called IndexedCollection which is now returned by the overload which deserializes results on the following method:

• JsonIndexGet/JsonIndexGetAsync

This type is generic and allows for dealing with the result of the JsonIndexGet and JsonIndexGetAsync as a dictionary and as a collection.

Examples

Sam Dzirasa has authored a blog post full of practical examples of how to use NReJSON in an application:

Using RedisJson

Also, in this repository there are a suite of integration tests that should be sufficent to serve as examples on how to use all supported RedisJSON commands.

Integration Tests

nrejson's Issues

What is the License?

`JSON.STRAPPEND` isn't implemented.

I tried a few different things, but I'm just not sure how this command works so I wasn't able to add support for it.

Pathed Result does not support implicit conversion for IEnumerables

When calling the GetJson<T> method, if you set T to IEnumerable<TI> implicit conversion fails. This feels like incorrect behaviour.

The below exception is thrown during deserialisation:

System.Text.Json.JsonException
  HResult=0x80131500
  Message=The JSON value could not be converted to System.Collections.Generic.IEnumerable`1[System.Object]. Path: $[0] | LineNumber: 0 | BytePositionInLine: 8.
  Source=System.Text.Json
  StackTrace:
   at System.Text.Json.ThrowHelper.ThrowJsonException_DeserializeUnableToConvertValue(Type propertyType)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, TCollection& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo jsonTypeInfo, Nullable`1 actualByteCount)
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
   at Flow.Runtime.RedisInterface.DefaultSerializer.Deserialize[TResult](RedisResult serializedValue) in C:\projects\Flow Core\Runtime\Flow.Runtime.Redis\Serializer.cs:line 17
   at NReJSON.PathedResult`1.ParseRedisResult()
   at NReJSON.PathedResult`1.GetEnumerator()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Flow.Runtime.RedisInterface.Repository.RedisRepository.<GetJsonArray>d__8`1.MoveNext() in C:\projects\Flow Core\Runtime\Flow.Runtime.Redis\Repository\RedisRepository.cs:line 97

  This exception was originally thrown at this call stack:
    [External Code]
    Flow.Runtime.RedisInterface.DefaultSerializer.Deserialize<TResult>(StackExchange.Redis.RedisResult) in Serializer.cs
    [External Code]
    Flow.Runtime.RedisInterface.Repository.RedisRepository.GetJsonArray<T>(string, string) in RedisRepository.cs

This feels like incorrect behavior when getting json

Support Redis Transactions

To support transactions only a single modification is required which is

• DatabaseExtensionsAsync needs to extend IDatabaseAsync instead of IDatabase.

• We could use trans.AddCondition(Condition.KeyExists("key")) to watch key and check for it's existence.
• User could wait for any JSON async queued task, after calling trans.Execute()

---

OLD Thoughts:

I think the following points should be considered:

• DatabaseExtensionsAsync needs to extend IDatabaseAsync instead of IDatabase
• return types should be RedisResult or Task<bool> as waiting any queued operation will block the execution
• an extension method could be added for RedisResult for the deserialize operation

JsonIncrementNumberAsync

I don´t know the reason the following instruction works well:

db.JsonIncrementNumber(Key,"$.Day.Total",1);

but when I use a double instead of an integer:

db.JsonIncrementNumber(Key,"$.Day.Total",1.1);

always yields the exception StackExchange.Redis.RedisServerException: 'ERR trailing characters at line 1 column 2'

Support json.toggle and json.clear commands

Add support for the following commands:
json.toggle
json.clear

Need to update StackExchange.Redis nuget pack version to lates

<PackageReference Include="StackExchange.Redis" Version="2.5.61" /

JSON.GET Options

On the JSON.GET command, add support for:

INDENT
NEWLINE
SPACE

https://oss.redislabs.com/redisjson/commands/#jsonget

Add support for RedisJSON 2.0 full JSONPath syntax

Please notice all the commands can now get full jsonpath, i.e. dynamic paths that start with a $ sign.
As a result all the commands can now return multi parts of the JSON.
see: https://oss.redis.com/redisjson/commands/

Remove escape characters from JSON.GET

I have this code

        var connectionMultiplexer = await ConnectionMultiplexer.ConnectAsync("localhost");
        var redisDb =  connectionMultiplexer.GetDatabase(1);

        var setRedisResult = await redisDb
            .JsonSetAsync("test", "{ \"param1\": \"value1\", \"param2\": \"value2\", \"intValue\": 1974 }")
            .ConfigureAwait(false);

        var getRedisResult = await redisDb
            .JsonGetAsync("test", ".param1")
            .ConfigureAwait(false);

        var getResult = getRedisResult.ToString().Replace("\"", ""); 
        if (getResult == "value1")
        {
            Console.WriteLine("YES!!");
        }

        Console.WriteLine("nooooooooooooooooooooooooooooooooo");

What I would like to know is, why the JsonGetAsync doesn't return the string with the " in the beginning and end of the key and value?
I have to replace it in my code but this could be done internally and if not why?

I checked the tests and there is no test checking the content of what we get.

Best wishes
Paulo Aboim Pinto

Support Full Framework net461 and net472

Stackexchange Redis does build binaries for the full frameworks net461 and net472.
Is there a reason why NReJSON doesn't build for the same?

As far as I can tell, the only thing that would need to change is some default keywords being changed over to null. (language features)
Do you mind if I submit a PR for that change?
My scenario requires net461 to be used rather than .netstandard 2.0.

JsonGetAsync not working

I just installed v3 of Nrejson

The following code retuns:
{"type":4,"isNull":false}

RedisResult result = await dbLocal.JsonGetAsync("mykey");
return result;

---

The v2 code
var result = await dbLocal.JsonGetAsync("mykey");
return result;

worked

---

redis-cli returns the correct json

json.get mykey -- > Works

Is it possible to add the extension methods to the redis transaction interface (ITransaction)

StackExchange.Redis allows many async operations over a transaction as documented here (https://stackexchange.github.io/StackExchange.Redis/Transactions.html). However, JSON-specific functions are lacking in the transaction interface. Is it possible to also have extension methods on the transactions, so JSON operations can be used?

NullReferenceException when using JsonSetAsync with CommandFlags.FireAndForget

I apologize if I'm overlooking something obvious, but I'm new to RedisJSON and am running into an issue while trying to do a pretty simple task. I'm trying to generate some sample products and add them to Redis, but I get a NullReferenceException when executing the following code:

var multiplexer = await ConnectionMultiplexer.ConnectAsync("localhost");
var redis = multiplexer.GetDatabase();

var departments = new string[] { "Men", "Women", "Kids", "Travel", "Electronics" };
var brands = new string[] { "Apple", "Microsoft", "Samsung", "Sony", "Nike", "Adidas", "Reebok" };
var colors = new string[] { "Black", "Blue", "Green", "Grey", "Orange", "Pink", "Purple", "Red", "White" };
var random = new Random();

var count = 0;

while (count < 1000000)
{
    var tasks = new List<Task>();

    while (tasks.Count < 10000)
    {
        count++;

        var product = new
        {
            Id = $"product-{count}",
            Department = departments.GetValue(random.Next(0, departments.Length)),
            Brand = brands.GetValue(random.Next(0, brands.Length)),
            Color = colors.GetValue(random.Next(0, colors.Length)),
            Name = $"Product {count}",
            Description = "<p>Sed interdum aliquet augue eget placerat. Aliquam ac orci placerat, mattis arcu quis, finibus est. Aenean erat purus, dignissim in dapibus ut, eleifend nec nunc.</p>",
            Price = Math.Round((0.01 + random.NextDouble()) * 250, 2)
        };

        var task = redis.JsonSetAsync(product.Id, product, commandFlags: CommandFlags.FireAndForget);

        tasks.Add(task);
    }

    await Task.WhenAll(tasks);
}

The error is triggered by this line of code:

var task = redis.JsonSetAsync(product.Id, product, commandFlags: CommandFlags.FireAndForget);

I can eliminate the error by replacing this line with either of the following:

var task = redis.JsonSetAsync(product.Id, product);

or

var json = JsonSerializer.Serialize(product);
var task = redis.SetAddAsync(product.Id, json, CommandFlags.FireAndForget);

I want to keep CommandFlags.FireAndForget as it is much faster, but it seems that combining this with NReJSON doesn't work.

If it makes any difference, this is my SerializerProxy:

public sealed class RedisJsonSerializer : ISerializerProxy
{
    public TResult Deserialize<TResult>(RedisResult serializedValue)
    {
        return JsonSerializer.Deserialize<TResult>(serializedValue.ToString());
    }

    public string Serialize<TObjectType>(TObjectType obj)
    {
        return JsonSerializer.Serialize(obj);
    }
}

Any assistance would be greatly appreciated. Thanks!

Getting issue in the length property of JSONPATH

Below is my json string

[{"Id":1,"Name":"first value","Added":"2022-06-15T09:48:36.7132436Z"},{"Id":2,"Name":"second value","Added":"2022-06-15T09:48:36.7132479Z"},{"Id":3,"Name":"third value","Added":"2022-06-15T09:48:36.713248Z"}]

And the structure is:

public class Data
{
    public int Id { get; set; }
    public string Name { get; set; }
    public DateTime Added { get; set; }
}

In the redis, I have one key for which I am storing List.
In case to insert a single data record I was using length to store new data record and was using JSONPATH.

Below is the JSONPATH
var serializedRedisData = JsonConvert.SerializeObject(data);
_database.JsonSet(redisKey, serializedRedisData, ".[(@.length)]");

But getting below error:

StackExchange.Redis.RedisServerException: 'JSON Path error: path error: \n$.[(@.length)]\n^^^^^\n'

redisjson indexing

Add redisjson indexing to NReJSON. From https://github.com/RedisJSON/RedisJSON, support:

json.set user2 $ '{"last":"Joan", "first":"Mc"}' INDEX person"
json.index add person last $.last
JSON.QGET person Jo*

ERR Search path error at offset 2: an identifier can only begin with a letter, a dollar sign or an underscore - use bracket notation for anything

Hi,

Recently I came across NReJson and thought of using this. Right now I am working on a POC. I used the reference of Tom's article https://www.linkedin.com/pulse/introduction-redisjson-net-developers-thomas-hanks/ to start with the same.

In my API I have used 2 use cases which is to insert one object into list of objects and update an existing object in that list.

Below is my object:

{
  "id": 100,
  "name": "xyz"
}

To make it work I spun up redislabs/rejson image as a docker container and both the use cases are working fine.

Insert Use case:

var serializedRedisData = JsonConvert.SerializeObject(data);
_database.JsonArrayAppend(redisKey, ".", serializedRedisData);

Update Use case:

var serializedRedisData = JsonConvert.SerializeObject(data);
_database.JsonSet(redisKey, serializedRedisData, $".[?(@.Id=={data.Id})]");

In the docker environment everything is working fine.

But in the actual environment we are having an Ubuntu 16.04 machine where we have installed a Redis server.
And there I have referred this article to install RedisJson module in the redis.
https://gist.github.com/lmj0011/820eea392f6f43c755fadc2ba56b69e9

In this way Insert use case is working but the update is failing with the below error:

**ERR Search path error at offset 2: an identifier can only begin with a letter, a dollar sign or an underscore - use bracket notation for anything**

Is it the redisjson modules I have installed is the older one?

Need help for this error @tombatron