ivoleitao / stash Goto Github PK

@ivoleitao, what do you think about adding a way to interact with records, ie filter, sort, select, etc?

A powerful expression evaluator / search library is already available for Collections, JMESPath.

Describe the bug
By extension Stash is unfortunately pretty much incompatible with most flutter projects

Because every version of flutter_test from sdk depends on async 2.6.1 and stash >=3.0.0-dev.2 depends on async ^2.7.0, flutter_test from sdk is incompatible with stash >=3.0.0-dev.2.
And because stash_dio >=3.0.0 depends on stash ^3.0.0, flutter_test from sdk is incompatible with stash_dio >=3.0.0.
So, because alltold depends on both stash_dio 3.0.0 and flutter_test any from sdk, version solving failed.

Is your feature request related to a problem? Please describe.
As sqlite3 is a synchronous C library, accessing the database from the main isolate can cause blocking IO operations

Describe the solution you'd like
drift can spawn a long-running isolate to run SQL statements using NativeDatabase.createInBackground instead of the default NativeDatabase constructor.

Describe alternatives you've considered
None

Additional context
https://drift.simonbinder.eu/docs/advanced-features/isolates/

Describe the bug
Caching crashes when responses do not have cache-control or expires headers set

To Reproduce
Make a request to any URL that does not return any of the cache-control or expires headers set.

Expected behavior
The response should be successfully cached.

Version
stash_dio 2.0.2

Additional context
I believe there is a typo here:

This line looks like it should be ?. rather than !.

This is a question.

How do I retrieve a stream of cached values that can be used in a reactive user interface. Hive itself offers an implementation of this.

Is your feature request related to a problem? Please describe.
The management of multiple repos is becoming a burden I would like to manage everything in only one place

Describe the solution you'd like
Move to a mono repo strategy, this approach used on mobx seem good

Describe alternatives you've considered
Maintaining the repos will become an even greater burden as soon objectbox and isar support are added

Additional context
I want to create additional packages initially supporting isar and objectbox

The backend implementations of stash should support running on the web if the associated package supports running on the web. For example, sembast is marked as being JS compatible on pub.dev, but stash_sembast is not.

This is a question
Hi, hive and objectbox get requests are not async but with stash, its synchronous. Are there any performance downsides to this?

• Map<K, V> getAll(Set<? extends K> keys);
• void putAll(java.util.Map<? extends K, ? extends V> map);
• void removeAll(Set<? extends K> keys);

In my app log, I receive the following warning numerous times when using moor stashes:

I/flutter (10409): WARNING (moor): It looks like you've created the database class CacheDatabase multiple times. When these two databases use the same QueryExecutor, race conditions will occur and might corrupt the database. 
I/flutter (10409): Try to follow the advice at https://moor.simonbinder.eu/faq/#using-the-database or, if you know what you're doing, set moorRuntimeOptions.dontWarnAboutMultipleDatabases = true
I/flutter (10409): Here is the stacktrace from when the database was opened a second time:
I/flutter (10409): #0      GeneratedDatabase._handleInstantiated (package:moor/src/runtime/api/db_base.dart:88:30)
I/flutter (10409): #1      new GeneratedDatabase (package:moor/src/runtime/api/db_base.dart:59:12)
I/flutter (10409): #2      new _$CacheDatabase (package:stash_sqlite/src/sqlite/cache_database.g.dart:480:38)
I/flutter (10409): #3      new CacheDatabase (package:stash_sqlite/src/sqlite/cache_database.dart:22:43)
I/flutter (10409): #4      new SqliteAdapter (package:stash_sqlite/src/sqlite/sqlite_adapter.dart:39:19)
I/flutter (10409): #5      new SqliteFileAdapter (package:stash_sqlite/src/sqlite/sqlite_adapter.dart:97:9)
I/flutter (10409): #6      newSqliteFileCache (package:stas

I am using several moor stashes in my app, but I believe each is correctly instantiated each only once. The building of the first cache does not trigger the warning, but all subsequent caches do.

I can build a min reproducible example or test case, but before I spent time on it I wanted to check if it was a known problem or perhaps, as the warning suggests, I should simply enable the dontWarnAboutMultipleDatabases flag.

Is your feature request related to a problem? Please describe.
Retrieving values from a store through async means makes everything in flutter 4 times more complicated

Describe the solution you'd like
Just like shared preferences and many other storage solutions, a Vault could memory cache its current value.
Because initialising a Vault already requires an async operation, this is the perfect place to get the first value.
After that, the Vault can update its value under the hood whenver an event arrives.

Describe alternatives you've considered
I can cook up my own solution that is sync

Additional context
The intialize async once, use sync mechanism could be a great improvement over the always async approach.
It would allow us to make Settings classes which need only be initialized once in main, then can be used fully synchronously for the rest of the runtime.

The cache method of the FileCacheStore object was removed between version 4.3.2 and version 4.3.3.
Also, the documentation and examples don't seem to reflect this change.

I expect changes to a module's interface such as this to require a change to the major portion of the version number.

This is from https://semver.org/

Given a version number MAJOR.MINOR.PATCH, increment the:

MAJOR version when you make incompatible API changes
MINOR version when you add functionality in a backwards compatible manner
PATCH version when you make backwards compatible bug fixes

Describe the bug

ExpiryPolicy.eternal sets a duration that overflows Javascript's number type. This is specific to compiling to web.

To Reproduce
Steps to reproduce the behavior:

Nothing special required. Create a cache with Lfu eviction policy, and add any item to it. It'll work fine when running on the Dart VM, but throw an error when running in a web browser.

Expected behavior
Consistent behaviour on every platform.

Version
Dart SDK version: 2.12.2 (stable) (Wed Mar 17 10:30:20 2021 +0100) on "linux_x64"

Additional context
The problem can be fixed by reducing the value of the eternal constant in api/expiry/expiry_policy.dart:

abstract class ExpiryPolicy {
/// The value used to represent a eternal duration
static const Duration eternal = Duration(days: 999999);

...

It's presumably due to the difference in numeric precision between the Dart VM and Javascript, although that's just a guess; I didn't spend much time tracking it down other than the bare minimum necessary to fix my problem.

Describe the bug
I need to add the following dependency overrides to my flutter projects using stash 4.3.0

dependency_overrides: 
  async: 2.8.2
  clock: 1.1.0
  meta: 1.7.0
  path: 1.8.1

To Reproduce
Steps to reproduce the behavior:

1. Create a new flutter project
2. Add stash and stash_objectbox
3. Run pub get
4. See error

Expected behavior
Works without adding dependency_overrides

Version
4.3.0

Describe the bug
So, this one took me quite a bit of digging. It turns out that some backends (stash_hive and stash_isar at least, possibly others) are unable to decode nested JSON structures that where written by a different VM instance (as in, the values have left memory).

It seems that this is due to objects' runtime type of Map<String, dynamic> is lost and instead you get Map<dynamic, dynamic>. The top level is fine because stash casts this, but nested objects are not touched.

To Reproduce
Here's a test scenario that can be used to recreate the behavior:

import 'package:json_annotation/json_annotation.dart';
import 'package:stash/stash_api.dart';
import 'package:stash_hive/stash_hive.dart';
import 'package:test/test.dart';

part 'events_cache_test.g.dart';

@JsonSerializable()
class User {
  User(this.id);

  factory User.fromJson(Map<String, dynamic> json) => _$UserFromJson(json);

  final String id;

  Map<String, dynamic> toJson() => _$UserToJson(this);
}

@JsonSerializable()
class Event {
  Event(this.id, this.owner);

  factory Event.fromJson(Map<String, dynamic> json) => _$EventFromJson(json);

  final String id;
  final User owner;

  Map<String, dynamic> toJson() => _$EventToJson(this);
}

Future<Cache<Event>> _cache() async {
  final store = await newHiveDefaultCacheStore();
  return store.cache<Event>(name: 'test', fromEncodable: Event.fromJson);
}

void main() {
  final event = Event('123', User('123'));

  test('Step 1, run this first, by itself', () async {
    final cache = await _cache();
    await cache.put('123', event);
    await cache.get('123');
  });

  test('Step 2, run this second, also separately', () async {
    final cache = await _cache();
    await cache.get('123');
  });
}

Expected behavior
After writing the entry in step 1, it should be deserializable in step 2, but instead it fails with: type '_Map<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>' in type cast

Version
4.5.2

Additional context
I have been able to work around the issue by changing PersistenceStore.decodeValue to this:

dynamic decodeValue(
      dynamic value, dynamic Function(Map<String, dynamic>)? fromEncodable,
      {dynamic Function(dynamic)? mapFn, dynamic Function()? defaultFn}) {
    if (value == null) {
      return null;
    }

    if (fromEncodable != null) {
      return fromEncodable(_toEncodable(mapFn != null ? mapFn(value) : value));
    } else if (defaultFn != null) {
      return defaultFn();
    }

    return value;
  }

  dynamic _toEncodable(dynamic source) {
    if(source is Map) {
      return source.cast<String, dynamic>().map((key, value) => MapEntry(key, _toEncodable(value)));
    }
    if(source is Iterable) {
      return source.map(_toEncodable).toList();
    }
    return source;
  }

I'd open a pull request but I've haven't been able to write a testcase which properly sets up the preconditions for the current code to fail. I also have some doubts about the efficiency of the code (Especially since Dart is not tail-recursive :D )

I also just realized that this would probably affect storing of plain maps (without a fromEncodable) as well, meaning that return source; above will return a Map<dynamic, dynamic> when reading a cached value after a restart. So _toEncodable may have to be run on that as well.

I'm new to this, so forgive the simplicity of the question. Once you create the memory cache in your main.dart, how do you import the same cache in other files to use/leverage the cache?

Describe the bug
The stash_isar package has a reported analysis issue on pub.dev: ERROR: Evaluation of this constant expression throws an exception.

https://pub.dev/packages/stash_isar/score

Describe the bug
If you store a String value into a hive vault in version 4.5.2 and want to retrieve it in version 5.1.0 a TypeError occurs when the value is decoded in file 'packages/stash/lib/src/api/store.dart'

To Reproduce

1. Setup a simple vault in version 4.5.2
2. Store a String value in that vault
3. Update stash to newest version 5.1.0 in pubspec.yaml
4. Run app again and try to retrieve value from vault

Here is a simple Example

Expected behaviour
Expected behaviour is that value is retrieved without errors.

Screenshots

Version
Dart SDK version: 3.3.0 (stable) (Tue Feb 13 10:25:19 2024 +0000) on "macos_arm64"

This is a question

Currently I have an app with multiple repositories

• each repository handles its local data source
• previously I had a custom HiveLocalDataSource base class that was implemented by all local data sources
• I'd like to replace this current system with stash

For example my new local data source for my account_repository will look like this.

final accountStore = await newObjectboxLocalCacheStore(
    fromEncodable: (json) => Account.fromJson(json),
  );

  // Creates a cache with a capacity of 10 from the previously created store
  final cache = await accountStore.cache<Account>(
    name: 'cache1',
    maxEntries: 10,
    eventListenerMode: EventListenerMode.synchronous,
  );

Is it safe to do this in all my repositories or is it more efficient to have one store shared through the entire app?

Describe the bug
Currently, when deleting a cache entry stash's GenericCache will try to read the entry as part of the deletion, which will try to decode its contents. If the decoding fails, the deletion will fail.

An important aspect of a local cache is to be able to treat it as ephemeral data. Ie. if your schema evolves, you just delete invalid entries and repopulate them. Currently, this is not possible with stash when using custom decoders.

To Reproduce

1. Create a cache using stash_hive.
2. Add an entry to the cache that is not decodable from JSON.
3. Try to delete the entry.

Expected behavior
The entry is deleted

Version
stash 4.5.1

Is your feature request related to a problem? Please describe.
I would like to use one consistent tool for my vaults, caches and also FIFO and LIFO queues.

Describe the solution you'd like
FIFO and LIFO queues are quite different from the key, value construct of the vault and cache implemented by stash, but there is also a lot of functionality that stash currently implements that would be very useful in FIFO and LIFO queues, most importantly the ability to plugin the storage, file, memory, etc...

Describe alternatives you've considered
I am considering, flutter_persistent_queue, which has a MUCH lower pub.dev score than stash.

Additional context
I might have some time to help implement, but this is a significant addition and would not want to get involved without help.

Does this package support etag ?

I have been avoiding upgrading for flutter 2.0 because the one time I tried, it was a nightmare. So until I have time I have been sticking with 1.22.6

So I want to use Stash! When I add
stash_memory: ^1.0.4
when I do a pub get, it gives me error
Because XXX depends on stash_memory ^1.0.4 which doesn't match any versions, version solving failed.
pub get failed (1; Because XXX depends on stash_memory ^1.0.4 which doesn't match any versions, version solving failed.)
However I can download 1.0.4
Is there anyway to manually import it? Or some other solution other than spending a week upgrading to flutter 2.x

Hi this is not bug, i just confused about Eviction Policies 😅
Can you give me example for how to using Eviction Policies ?
I want to using FifoEvictionPolicy and then i have added FifoEvictionPolicy in evictionPolicy parameter, but is not work.
Thanks

I'm using stash as a non-volatile LRU store of records representing documents. I will cache these forever, but only cache a limited number. I would like to implement a simple text search (against titles, keywords, etc) of the stored documents.

I know the Moor stash uses an sqlite database via moor, so I wonder if you have any thoughts on how the underlying sqlite database (through sqlite, sqflite, or moor packages perhaps) could be accessed and queried.

I've tried opening the file directly with code such as

_searchDatabase = await LazyDatabase(() async {
       final dir = await getApplicationDocumentsDirectory();
       final path = p.join(dir.path, "stash_documents.moordb");
       final file = File(path);
       return VmDatabase(file);
     });

but I was unable to query, receiving a LateInitializationError instead.

Is your feature request related to a problem? Please describe.
Recognise stash contributors

Describe the solution you'd like
I would like to use the All Contributors

Describe alternatives you've considered
None

Additional context
Recently a big contribution to stash with cbl

Describe the bug

_diskCache = newHiveCache(
      path,
      cacheName: name,
      maxEntries: diskCountLimit,
      expiryPolicy: diskAgeLimit != null ? ModifiedExpiryPolicy(diskAgeLimit) : null,
      evictionPolicy: const LruEvictionPolicy(),
    );

invoke _diskCache.keys then got this error:

HiveError: Wrong checksum in hive file. Box may be corrupted.
#0 StorageBackendVm.initialize (package:hive/src/backend/vm/storage_backend_vm.dart:97)

Version
stash: 2.0.2
stash_hive: 2.0.2

Flutter 2.5.2 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 3595343e20 (11 months ago) • 2021-09-30 12:58:18 -0700
Engine • revision 6ac856380f
Tools • Dart 2.14.3

Hi, I just discovered your package and I'm planning to use it for my project. I also just discovered Pocketbase, and I think I'd also like to use that. I just looked at that package, but I shouldn't have any serious problems using it with Stash as a backend?
https://pub.dev/packages/pocketbase

Description

The stash package has functionality around cache, such as the hitCount: In order to track the number and keep it updated there's a write operation after a cache hit, which may cause a QuotaExceeded error. The same happens for write operations, but that's expected.

Question(s) - Am I missing some configuration or error handling? Is there any way of disabling the functionality I don't use? Any suggestion? :)

Environment

$ flutter --version
Flutter 3.10.0 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 84a1e904f4 (3 weeks ago) • 2023-05-09 07:41:44 -0700
Engine • revision d44b5a94c9
Tools • Dart 3.0.0 • DevTools 2.23.1

dependencies:
  [...]
  stash: ^4.3.2
  [...]
  stash_shared_preferences: ^4.6.2

    return store.cache<Uint8List>(
      name: defaultCacheName,
      maxEntries: maxEntries,
      evictionPolicy: null,
    );

This is a request to add support for null safety to package:stash.
We depend on your awesome package, so would be great to have null safety enabled.

The Dart/Flutter team already encourages publishing the migrated packages: See this blog post.

See the migration guide for details about enabling null safety.

Using generics throw cast exception when getting a value that is not in the cache.

For example

import 'package:stash/stash_api.dart';
import 'package:stash_memory/stash_memory.dart';

class Task {
  final int id;
  final String title;
  final bool completed;

  Task({required this.id, required this.title, this.completed = false});

  @override
  String toString() {
    return 'Task $id, "$title" is ${completed ? "completed" : "not completed"}';
  }
}

void main() async {
  // Creates a store
  final store = newMemoryCacheStore();

  // Creates a cache with a capacity of 10 from the previously created store
  final cache = store.cache<Task>(name: 'cache1', maxEntries: 10, eventListenerMode: EventListenerMode.synchronous)
    ..on<CacheEntryCreatedEvent<Task>>().listen((event) => print('Key "${event.entry.key}" added to the cache'));

  // ***TRY TO GET A VALUE THAT IS NOT IN CACHE***
  final task = await cache.get('task1');
  print("task1 should be null: $task");
  // *******

  // Adds a task with key 'task1' to the cache
  await cache.put('task1', Task(id: 1, title: 'Run cache store example', completed: true));
  // Retrieves the value from the cache
  print(await cache.get('task1'));
}

Running the example throws this exception:

Unhandled exception:
type 'Null' is not a subtype of type 'Task' in type cast
#0      new Future.value (dart:async/future.dart:334:59)
#1      new GenericCache.<anonymous closure>
package:stash/…/cache/generic_cache.dart:105
#2      GenericCache.get.<anonymous closure>.<anonymous closure>
package:stash/…/cache/generic_cache.dart:328
<asynchronous suspension>
#3      main
example\cache\example.dart:25
<asynchronous suspension>

Describe the bug
While debugging some code I came across a situation where the server returned a response with multiple cache-control header values. This is valid according to RFC2616 but it will currently cause stash_dio to throw an exception ('"cache-control" header has more than one value, please use Headers[name]')

To Reproduce
Make a request to an endpoint returning several cache-control header values

Expected behavior
The values are concatenated, interspersed with commas

Version
Flutter 2.2.0 • channel stable • https://github.com/flutter/flutter.git
Framework • revision b22742018b (5 weeks ago) • 2021-05-14 19:12:57 -0700
Engine • revision a9d88a4d18
Tools • Dart 2.13.0

Additional context
Relevant section of RFC2616
Stack overflow question

Describe the bug
After adding stash as a dependency to my pubspec.yaml, my application will no longer build for web

To Reproduce

1. Add stash: ^2.0.0-nullsafety.3 to pubspec.yaml
2. Run flutter run -d chrome
3. See error

../../Library/flutter/.pub-cache/hosted/pub.dartlang.org/stash-2.0.0-nullsafety.3/lib/src/api/expiry/expiry_policy.dart:8:50: Error: The integer literal 8999999999999999991 can't be represented exactly in JavaScript.
Try changing the literal to something that can be represented in Javascript. In Javascript 9000000000000000000 is the nearest value that can be represented exactly.
static const Duration eternal = Duration(days: 8999999999999999991);

Version
Dart SDK version: 2.13.0-116.0.dev (dev) (Sun Mar 7 18:57:20 2021 -0800) on "macos_x64"

Additional context
Flutter 2.1.0-12.2.pre • channel beta • [email protected]:flutter/flutter.git
Framework • revision 5bedb7b1d5 (4 weeks ago) • 2021-03-17 17:06:30 -0700
Engine • revision 711ab3fda0
Tools • Dart 2.13.0 (build 2.13.0-116.0.dev)

Describe the bug
I am setting a cache policy of 1 day, I'd expect the cache.get method would return a value after navigating to a new screen.

To Reproduce
Steps to reproduce the behavior:

  _getAudioFiles() async {
    final path = Directory.systemTemp.path;
    final cache = newHiveCache(path, maxEntries: 100, expiryPolicy: const CreatedExpiryPolicy(Duration(days: 1)));
    final value = await cache.get('audioFiles');
    if (value != null) {
      print("using cache");
      setState(() {
        _audioFiles = value;
      });
    } else {
      print("In else");
      var audioFiles = await MyApi().getAudioFiles();
      await cache.put('audioFiles', audioFiles);
      List<dynamic> value = await cache.get('audioFiles');
     print(value);  // its being printed
      setState(() {
        _audioFiles = value;
      });
    }
  }

value is always return null therefore is always goes to my else statement.

Version
dart --version
Dart SDK version: 2.12.2 (stable) (Wed Mar 17 10:30:20 2021 +0100) on "macos_x64"

Additional context
I am using this with stash_hive, another odd thing is i have to include stash in my pubspec to use CreatedExpiryPolicy.

Describe the bug

I'm trying to use FileStore via newLocalDiskCache, and I periodically get this error when retrieving data from the cache:

Unhandled Exception: RangeError (byteOffset): Invalid value: Valid value range is empty: 0
_ByteDataView.getUint64     (dart:typed_data-patch/typed_data_patch.dart:4762:7)
BytesReader.readUInt64       package:stash/…/codec/bytes_reader.dart:76
DiskStore._readEntry         package:stash_file/…/file/file_store.dart:214
DiskStore._readFileEntry.<anonymous closure> package:stash_file/…/file/file_store.dart:231
_rootRunUnary               (dart:async/zone.dart:1362:47)
<asynchronous suspension>

Here's my definition of the cache:

...
  final _disk = newLocalDiskCache(
          cacheName: 'disk',
          codec: ImageCacheCodec(),
          path: path,
        );
...

I don't believe the ImageCacheCodec has anything to do with the error. The line where the error actually occurs, is where the CacheEntry is being read from bytes, well before the codec has a chance to read anything. But if you need the source anyway feel free to ask; it just reads/writes ZLib-encoded Uint8List bytes.

To Reproduce

1. Create a cache using newLocalDiskCache.
2. Hammer it with reads/writes.
3. Eventually the error above occurs.

Expected behavior

Reads should succeed if the data exists, or fail silently if it doesn't.

Screenshots

No screenshots.

Version

Dart SDK version: 2.13.0-211.14.beta (beta) (Mon May 3 08:08:14 2021 +0200) on "linux_x64"

Additional context

This occurred in a Pixel 4XL emulator running Android 29 with the latest build tools.

Describe the bug
When trying to read from a named cache, the stash_dio_interceptor crashes when using stash_hive.

As an aside, maybe stash_dio could use a default name (or require a name) for the cache rather than generating a new one each time. Presumably one uses persistent storage like Hive to be able to reuse it between runs no? :)

To Reproduce

1. Make a request to any URL that responds with headers that allow caching while using stash_dio with a named stash_hive cache
2. Restart
3. Make a request to the same URL again.

Expected behavior
The cached response should be returned

Actual behavior
There is a crash:

E/flutter (13275): [ERROR:flutter/lib/ui/ui_dart_state.cc(199)] Unhandled Exception: type '_InternalLinkedHashMap<dynamic, dynamic>' is not a subtype of type 'Map<String, dynamic>'
E/flutter (13275): #0      CacheInterceptorBuilder._onRequest (package:stash_dio/src/dio/interceptor_builder.dart:134:67)
E/flutter (13275): <asynchronous suspension>

Version
dio: 4.0.0
stash: 3.0.0
stash_dio: 3.0.0
stash_hive: 3.0.0

Additional context
This seems related to #9 but I didn't hit this before because I wasn't using a named cache and as such, a new one was created on each run.

Describe the bug
When trying to read from the cache, the stash_dio_interceptor crashes when using stash_hive. It does not seem to work at all.

To Reproduce
Make two requests to seemingly any URL while using stash_dio with stash_hive

Expected behavior
An HTTP-response is returned both times.

Actual behavior
The second request crashes with:

E/flutter ( 2495): [ERROR:flutter/lib/ui/ui_dart_state.cc(186)] Unhandled Exception: type '_InternalLinkedHashMap<String, dynamic>' is not a subtype of type 'CacheValue' in type cast
E/flutter ( 2495): #0      CacheInterceptorBuilder._onRequest (package:stash_dio/src/dio/interceptor_builder.dart:133:43)
E/flutter ( 2495): <asynchronous suspension>

Version
dio: 4.0.0
stash: 2.0.2
stash_dio: 2.0.3
stash_hive: 2.0.3