Is it possible to use SQlite database instead of JSON? about image-classifier

[swimauger]

I mean... It is possible, but it's not elegant (going from JavaScript to SQL so rarely is). With the current version (v1.0.8) you can access the knnClassifier dataset. From there you can use Tensorset to make it readable for ImageClassifier

This will give you the dataset data as JSON without having to save it

const knnDataset = JSON.parse(await Tensorset.stringify(myImageClassifier.classifier.getClassifierDataset())); // Though classifier will most likely not be exposed like this in future updates

knnDataset will look something like this:

[
    {
        "label": "Example 1 Label",
        "values": [ 52, 432, 432, 12, 152, ... ],
        "shape": [ 20, 1024 ]
    },
    {
        "label": "Example 2 Label",
        "values": [ 52, 432, 432, 12, 152, ... ],
        "shape": [ 30, 1024 ]
    },
    ...
]

From there you can make your SQL statement to save the data to your database.

When your ready to read it back in put it in the exact same format as the knnDataset variable above and set the classifier like so:

myImageClassifier.classifier.setClassifierDataset(Tensorset.parse(/* The similar variable to knnDataset */));

[swimauger]

Hope this helped, but I do not plan on making any future updates to support this. My best recommendation would be use a NoSQL database like Firebase or Mongo if possible. I'm closing the issue, but feel free to leave a comment here if you have any more questions about this.

[TrashUwU]

Thanks! The only problem with JSON is it can't handle large data and breaks.

[swimauger]

Np, but if you're referring to the speed at which the dataset is being loaded or unloaded, ImageClassifier hasn't yet been updated to use the latest version of Tensorset, which drastically improves the speed.

All this being said if large amounts of data is an issue, I've been thinking of creating a new format for compressed and diversified datasets, so the performance could improve.