Support storing arbitrary dart types as JSONB BLOBs with configuration support within .drift files about drift

[simolus3]

You can already get pretty far with the JsonbConverter I've suggested here if you don't need web support.

I've tried approaching this a few times, but I always backed out due to the complexity - this feature requires us to rewrite queries based on the result type, so if you do selectOnly(table)..addColumns([table.jsonbColumn]), we'd have to recognize that and actually run SELECT json(jsonbColumn) FROM table. And for default tables, the SELECT * also wouldn't work anymore - we'd have to list the columns explicitly. It might not sound that hard, but there is nothing in the runtime currently ready for that, and so it requires a lot of refactoring to get right.
It's not closed, and I'll probably give this another go once I have a bit more time, but it's not on my near-term list.

[simolus3]

I agree that we should have JSONB support. Using a type for this is kind of tricky because it requires things type converters are not yet able to do - altering the generated query.

For instance, if a user writes INSERT INTO tbl (dependencies) VALUES (?) in a drift file, we need to turn that query into INSERT INTO tbl (dependencies) VALUES (jsonb(?)). That's not super hard to do with the things we already have in place in the generator, but we also need to do it at runtime: into(tbl).insert(TblCompanion.insert(dependencies: [1, 2, 3])) also needs to construct a jsonb invocation somewhere.
Similarly, when selecting from a JSONB column, we'd have to wrap the expression in json() to get a readable format.

(of course, we could cheat and open an in-memory database that just does json(?) and jsonb(?) synchronously in the converter. but that doesn't work with sqflite and requires even more setup on the web)

[dvoloshyn]

Agree, this does not look like a trivial problem. It likely requires architectural changes.
As such, it might make sense to start a discussion and collect all the known and expected problems around types to see a broader picture.
From your comment I already see two (generalized):

• support different types at SQL query parameter level (TEXT) and database storage level (BLOB)
• inject necessary functions to convert between those during read (json()) and write (jsonb())
• ...

[simolus3]

For those using NativeDatabase (meaning that the sqlite3 library is available via dart:ffi), a converter like this can support JSONB today:

import 'dart:convert';

import 'package:drift/drift.dart';
import 'package:sqlite3/sqlite3.dart' as sqlite3;

class JsonbConverter<T> implements TypeConverter<T, Uint8List> {
  final T Function(Object?) _fromJson;

  JsonbConverter(this._fromJson);

  final sqlite3.Database _database = sqlite3.sqlite3.openInMemory();
  late final sqlite3.PreparedStatement _toText =
      _database.prepare('select json(?)');
  late final sqlite3.PreparedStatement _toBinary =
      _database.prepare('select jsonb(?)');

  @override
  T fromSql(Uint8List fromDb) {
    final result = _toText.select([fromDb]).single;
    final asText = result.values[0] as String;

    return _fromJson(json.decode(asText));
  }

  @override
  Uint8List toSql(T value) {
    final asText = json.encode(value);
    final result = _toBinary.select([asText]).single;
    return result.values[0] as Uint8List;
  }
}

It can be used in a column definition like this:

  BlobColumn myJson => blob().map(JsonbConverter((json) => MyClass.fromJson(json)))();

[AlexandreAndrade00]

Hi @simolus3, is this feature in the roadmap or is hanged?

[AlexandreAndrade00]

Thank you so much for the fast update! 😀