belphemur / node-json-db Goto Github PK

Okay, so I'm using a project that included this as a basic json library. What i'm trying to do is create a part of the database with data for a gamemode. The data will include HP, level, exp, weapon and the username. How would I do this? I know it's a nooby question but i'm coming from Python where I used json.dumps and json.loads for this. Thanks if anyone can help me :P

observed sometimes

When handling errors thrown by the lib, it'd be great if we could compare directly against the DataError and DatabaseError 'classes':

let JsonDB = require('node-json-db'),
    DataError = JsonDB.DataError,
    DatabaseError = JsonDB.DatabaseError;

// ...

try {
    let example = db.getData('/path/that/does/not/exist');
} catch (err) {
    if (err instanceof DataError) {
        // ...
    } else if (err instanceof DatabaseError) {
        // ...
    } else {
        // ...
    }
}

This can easily be done by add the Error classes as static members of the exported JsonDB class:

// line 250 of JsonDB.js
module.exports = JsonDB;
module.exports.DataError = DataError;
module.exports.DatabaseError = DatabaseError;

Hello,

I'm curious about the performance.

I think that having a database that writes and reads to plain JSON files is a great thing. I don't want to have to install mongdb/couchdb or whatever, with their proprietary binary formats.
JSON is an open and standardized format. I want my data to be decoupled from a particular database platform.

I had a look at the code, and by the preliminary looks of it, it does not seem that much performance or scalability optimization was done, which is my concern here.

Since I am not sure what is happening in the code, I think I'd best ask you about the topic - whether there is some indexing, partial loading or such optimization happening, or is the whole data just loaded in memory? If so, do you have plans to change that in the future?

Thanks!

Example:

Code

let data = {
    "norm_key": "data",
    "bad/key":"other data"
}

db.push("normal_key", data);
db.push("bad/key", data);

DB

{
    "normal_key": {
        "norm_key": "data",
        "bad/key":"other data"
    },
    "bad": {
        "key": {
            "norm_key": "data",
            "bad/key":"other data"
        }
    }
}

So you can create a key that has a / in it, but you can push data that does. This makes the "other data" unreachable, since by entering the key /bad/key/bad/key you'll search for

{
    "bad": {
        "key": {
            "bad": {
                "key":"other data"
            }
        }
    }
}

That doesn't exist

Hey @Belphemur,

I had trouble querying my json files while working with node-json-db on a project recently. I needed a way to use data from my json db conditionally. I figured out a workaround and finished the project. Now I have a little time to tinker and I was wondering, could I contribute to your repo? I'd like to suggest a 'find' function.
Right now, getData can't be used for conditionals because it returns an error if the requested entry doesn't exist. Instead, a function like 'find' could return null. 'Find' could take an optional argument for a test that would return from the path specified by the first argument. I imagine 'find' could work for objects or arrays without too much trouble. Doing something like this then becomes easy and clean:

if(books.find('/shakespeare')){
    var shakespearean_comedies = books.find('/shakespeare', function(book){
        book.genre === 'Comedy'
    }
}

What do you think?

Hi,
I'm using the version 0.6.5 from npm, and I think that there is an issue with this command:
db.push("/arraytest/myarray[test]", {obj:'test'}, true);

it creates this:

{
    "arraytest": {
        "myarray": []
    }
}

without the index 'test'.

The devDependency @types/node was updated from 11.10.4 to 11.10.5.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

The devDependency @types/node was updated from 10.12.8 to 10.12.9.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Currently, we use node-json-db which called more than once can cause some issues, for this repo this could be problem if we have more than one instance, so if we implement fs.createWriteStram we could avoid such problems.

My Json Array:
{"paths":[{"id":74185, "name": "John"}, {"id":54874, "name": "Clark"}]}

How can i fetch and puch where id is equal to 74185.
I am trying like this:
var data = db.getData("/paths/id/74185");

But not working. Kindly guide me

Hello,

I have rolled my own system similar to this and am having trouble with corruption of the data when the app closes suddenly (or exits)

Just wondering if the Sync calls you make to the file system are sync because that solves that problem, and then by extension, why are they sync calls instead of Async? I would think that might hurt performance or is there a reason?

I know the answer is "Just try it out" but shoehorning your library in now would take a fair while, and it might not solve anything so I figure its worth a shot asking!

Kind regards,

Ryan

The API was broken with last update:
var JsonDB = require('node-json-db');
now
var JsonDB = require('node-json-db').JsonDB;

Readme states that the Config is located at node-json-db/lib/JsonDBConfig
its actually at node-json-db/dist/lib/JsonDBConfig. Love this package btw, Thanks!

Hi,
the .delete() function is not affected by saveOnPush option, is it intentional?
Can you avoid that the user has to .save() every time that a .delete() is called?
Thank you.

Timing issue?
Env:
Node: v4.1.1, Windows 7
node-json-db: 0.4.6

To reproduce: run

var JsonDB = require('node-json-db');
var db = new JsonDB("myDataBase", true, false);
db.push("/test1","super test");

while the file "myDatabase.json" does not exists.
DatabaseError is thrown.

Work-around is to create myDataBase.json with empty json, then it works.

Would this be a cause?
on initial database save, mkdirp is called which is async

I have always used the following code to "delete" a "database", e.g. set it back to {}:

const jsondb = require('node-json-db');
const db = new jsondb('./mydb.json');

db.push("/id", "1");
db.delete("/");

Since 0.7.4, this fails with the following trace:

/home/ckrooss/sample/node_modules/node-json-db/lib/utils.js:19
        var match = arrayIndexRegex.exec(property.trim());
                                                  ^
TypeError: Cannot read property 'trim' of undefined
    at Object.processArray (/home/ckrooss/sample/node_modules/node-json-db/lib/utils.js:19:51)

Which I could easily fix by adding something like if (!property) return null; to processArray.
Would this be an acceptable solution worth a PR or is my use case just plain wrong?
I can always use db.push("/", {});, of course.

db.push("/arraytest/myarray[]/myTest", 'test', true);
What is the point for the last argument true? I can not find it in reademe

I am trying to delete an item from the database by doing the following:

deleteRoleOnDeletion: (guild, role) => {
    subbableRolesDB.delete(`/${role.name}`);
    // subbableRolesDB.save();
    log.info(`Deleted ${role.name} from db, id: ${role.id}`);
}`
`role.name` is the name of the item in the database, and it's actually right, but it doesn't seem to delete it.
No errors nothing.
Anyone have a clue what's going on?

Is there a switch/parameter to save the JSON file in "minified" form?
So, instead of this:

{
   "test":"test",
   "json":{
      "test":[
         "test"
      ],
      "important":5
   },
   "new":"cool"
}

We get this:

{"test":"test","json":{"test":["test"],"important":5},"new":"cool"}

If not, I would like to suggest you to add this feature.

What's wrong ?

I've installed npm as readme but at runtime i got this error.

Thanks !

In the app we are using React, ElectronJS and webpack 2. when we use 'node-json-db' in electronConfig file then in DEV env it is giving "Cannot find module 'node-json-db'" error, But in PROD env it is working fine.

don't know what is wrong in DEV env.

The devDependency @types/node was updated from 10.12.14 to 10.12.15.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

sometimes the database file just becomes empty

As far as I can tell there's no way to access a db through the working directory. So, for example, if you have a nested file you want to access a db (higher in the file tree) from, you're gonna error out:
const db = new jsonDB("../database.json", true, true);
Is there a work around, or do I need to tinker?

Hi,is the db call done sync or async?

Hey there,

I love this and been using it for a while with no problems. One thing I would love to have though is a uniq ID identifying each DB entry. I've been coding this myself but it aint the same ;)

What do you think?

`var JsonDB = require('node-json-db');
var db = new JsonDB("test.json", true, true);

db.push("/data", {a:null}, false);
db.push("/data", {a:"aaaa"}, false);`

then I got the json data
{ "data": { "a": { "0": "a", "1": "a", "2": "a", "3": "a" } } }

In {N}ativescript , I can use node/ npm packages alright,but node json db throws an error "stat.isDirectory is not a function" and in my logcat it shows an error concerning path something /. And heres how I initialize
db=new jsonDb('myDb',true,false);

The devDependency @types/node was updated from 11.10.5 to 11.10.6.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

The devDependency @types/jest was updated from 24.0.9 to 24.0.10.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/jest is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

I am getting this error -

Uncaught Exception:
NestedError: Can't save the database
    at DatabaseError.Error (native)
    at DatabaseError.NestedError (/Users/akka/dev/zulipwork/zulip/dist/Zulip-darwin-x64/Zulip.app/Contents/Resources/app.asar/node_modules/node-json-db/lib/Errors.js:7:25)
    at new DatabaseError (/Users/akka/dev/zulipwork/zulip/dist/Zulip-darwin-x64/Zulip.app/Contents/Resources/app.asar/node_modules/node-json-db/lib/Errors.js:29:33)
    at JsonDB.save (/Users/akka/dev/zulipwork/zulip/dist/Zulip-darwin-x64/Zulip.app/Contents/Resources/app.asar/node_modules/node-json-db/JsonDB.js:243:25)
    at new JsonDB (/Users/akka/dev/zulipwork/zulip/dist/Zulip-darwin-x64/Zulip.app/Contents/Resources/app.asar/node_modules/node-json-db/JsonDB.js:33:18)
    at Object.<anonymous> (/Users/akka/dev/zulipwork/zulip/dist/Zulip-darwin-x64/Zulip.app/Contents/Resources/app.asar/app/main/main.js:16:12)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:458:32)
    at tryModuleLoad (module.js:417:12)

I am using it here. It works perfectly when app is not packed.

What if I want to push to an Array? Hypothetical code below!

db.push("/arraytest/myarray[0]", "0000", true);
db.push("/arraytest/myarray[]", "0001", true);

It will be much easier to simply push to an array than keep track of the array index for every array push!!

I believe this feature would be very easy to implement with minimal code change! Change the regex to accept empty index and simply push to that array when index is empty.

Can you please implement this syntax and feature!

When I run my code then it just crashes. I have it in a try loop which is why I don't understand why it
crashes.
Code:

var bugIDcount = 1;
var continueWorking = true;
while (continueWorking) {
  console.log("In the loop again?");
  try {
    var data = bugs.getData(`/${bugIDcount}`);
  } catch (error) {
    console.error(error);
    continueWorking = false;
  }
  bugIDcount++;
}

Console Log:

  stack: 'NestedError: Can\'t find dataPath: /2. Stopped at 2
      at DataError.NestedError (/Users/Ben/git/crashlanding-discord/node_modules/node-json-db/lib/Errors.js:7:25)
          at new DataError (/Users/Ben/git/crashlanding-discord/node_modules/node-json-db/lib/Errors.js:37:33)
              at findData (/Users/Ben/git/crashlanding-discord/node_modules/node-json-db/JsonDB.js:100:27)
                  at recursiveProcessDataPath (/Users/Ben/git/crashlanding-discord/node_modules/node-json-db/JsonDB.js:127:17)
                      at JsonDB._getData (/Users/Ben/git/crashlanding-discord/node_modules/node-json-db/JsonDB.js:140:16)
                          at JsonDB.getData (/Users/Ben/git/crashlanding-discord/node_modules/node-json-db/JsonDB.js:70:21)
                              at Object.<anonymous> (/Users/Ben/git/crashlanding-discord/bot.js:27:21)
                                  at Module._compile (module.js:660:30)\n    at Object.Module._extensions..js (module.js:671:10)
                                      at Module.load (module.js:573:32)',
  message: 'Can\'t find dataPath: /2. Stopped at 2',
  inner: undefined,
  id: 5 }

Another issue is:

db.push('/comments/S244/165115/vote[0][]', data) => Only numerical values are accepted for array index

It doesn't matter whenever I put true or false as the third argument

When importing node-json-db into my TypeScript project, TypeScript complains that it cannot find types for node-json-db. The issue appears to be that the types file JsonDB.d.ts is not co-located with JsonDB.js, so the TypeScript compiler does not know where to look for it.

The simplest fix is to add a "types" field to package.json that gives the location of the file:

{
  "name": "node-json-db",
  "version": "0.9.1",
  "description": "Database using JSON file as storage for Node.JS",
  "main": "dist/JsonDB.js",
  "types": "dist/types/JsonDB.d.ts",    <<< types field tells the compiler where to look for types
...
}

Hi, thanks for such a great library :)

I have tried
bdd.push('/comments/244[]', data, true)

But it create that entry in db : "244[]":{"nom":"anon","content":"blah"}}...

I have tried with a non-numeric index and it works just fine...
bdd.push('/comments/aa[]', data, true) => OK

Sorry, havent made a pull request cause I don't understand exactly how you process the []

Thanks ;)

To be able to use it in browser easily and rely on the storage mechanism there.

• Create a storage "interface"
• Implement driver for hard storage (current one)
• Implement storage for LocalStorage

Hey there 👋 , I'm using this wonderful module for music system and I realized when I delete an object which contains an array called queue and then I will log the data from the queue ('Check if it's not there and catch errors) I saw that it logs the array perfectly and it does not inside the JSON any more.

Doing this:
jdb.push('/path[]', {foo: bar}, true/false)

results in this:

{
    "path[]": {
      "foo":"bar"
   }
}

instead of what it used to do which was:

{
   "path": [
      { "foo":"bar" }
   ]
}

In a way to resolve correctly #75.

Let the user choose what separator to use for the keys.

The devDependency @types/node was updated from 11.11.1 to 11.11.2.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

@types/node is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

I am, using Angular 6 and I am running into the following error on build:

ERROR in my_dir/node_modules/mkdirp/index.js
Module not found: Error: Can't resolve 'fs' in 'my_dir/node_modules/mkdirp'
ERROR in my_dir/node_modules/node-json-db/dist/JsonDB.js
Module not found: Error: Can't resolve 'fs' in 'my_dir/node_modules/node-json-db/dist'

The second library that you are importing is fs.

import * as FS from 'fs'

This library is now not really a library, I guess. It's got an empty page on NPM that says it used to be a codebase, but isn't anymore. Have you experienced this yet, and is there a workaround/fix for this issue? I would love to use this JSON DB lib, because it looks like exactly what I would like, but this is a roadblock for me to use this.

db.delete(`./tasks[${taskDBIndex}]`);

If i try to delete a position, the position is not deleted, it just set it as null.
so i have an array of nulls.

    "tasks": [
        null,
        null,
        null,
    ]

how do i merge json data to already existing .json file?

hello, is there an integration with NestJS for this tool?

I have an array which i would like to delete the first index. How do i get to deleting the data?

It seems like db.delete does not accept [0]

I am entering final phases of implementing an Auction system using superb node-json-db as the primary database keeping track of guest bidding on items during auction. The Auction repo will be made public in about month from now once stable for forking.

The Problem:
There are two collections that require atomic update; one collection tracks bid requests, the other collection is the items being bid on. (ie: bid info stored on 'bids' collection is followed immediately by an update of the current price in the 'items' collection.) During tests under load I have discovered inconsistencies between the 'bids' collection and 'items' collection - which is not that surprising given the node event loop. (I could get into some 'Yak shaving' and play around with the ticks - but, as a last resort - I gotta get this project done).

The Question:
Has anyone already implemented atomic update in a fork of node-json-db? (awesome!)
If not :( I will implement - was thinking to use rwLock. But any suggestions or get arounds would be greatly appreciated.

Of course, if I do end up doing it and there is an interested in having transaction protection in node-json-db, I'll submit a pull request when done .

Thanks
@PotOfCoffee2Go

We've been using node-json-db in one of our TypeScript projects for a while, so I was excited to see that this project itself got updated to TypeScript! But I initially had trouble upgrading to 0.9. I eventually figured out how to make it work, but thought maybe others could benefit from having some TypeScript-specific instructions in the README.

The two bits of info that I'd suggest making sure are present in the README are:

1. To use node-json-db, in your own TypeScript project, you MUST make sure that the you have the esModuleInterop compiler option set to true in your project's tsconfig.json. Because of the way the .js and .d.ts files in node-json-db are currently generated, node-json-db cannot be used in a project without esModuleInterop=true.

2. Then, to use JsonDB from a TypeScript file, you import like this:

import JsonDB from "node-json-db";

Just that easy. And then the rest of the instructions should work great!

If you'd like me to make a pull request for the README changes, I'm happy to. Just let me know.

Thanks!

README.md, issues in
first example
1)
db.push("/test2/my/test/",5);
generates an intermittent bug, it might save the value 5 as an unnammed anonymous entry, resulting in:

"test2": {
        "my": {
            "test": {
                "": 5
            }
        }

OR it might crash, with this error:

this.data[this.parent] = toSet;
                                   ^
TypeError: Cannot assign to read only property '' of 5
    at DBParentData.setData

One possible fix is to remove the trailing slash

db.push("/test2/my/test",5);

[Which may or may not be what you intend to give as an example ?]
Results in

   "test2": {
        "my": {
            "test": 5
        }

Then c.a. line 24 should read:

At this point, the results are:
{
    "test1": "super test",
    "test2": {
        "my": {
            "test": 5
        }
    },
    "test3": {
        "test": "test",
        "json": {
            "testy": [
                "testy"
            ],
            "important": 5
        },
        "new": "cool"
    }
}

I would suggest testing live code and excerpting the results directly into the docs.