nozbe / watermelondb

🍉 Reactive & asynchronous database for powerful React and React Native apps ⚡️

A reactive database framework

Build powerful React and React Native apps that scale from hundreds to tens of thousands of records and remain fast ⚡️

	WatermelonDB
⚡️	Launch your app instantly no matter how much data you have
📈	Highly scalable from hundreds to tens of thousands of records
😎	Lazy loaded. Only load data when you need it
🔄	Offline-first. Sync with your own backend
📱	Multiplatform. iOS, Android, Windows, web, and Node.js
⚛️	Optimized for React. Easily plug data into components
🧰	Framework-agnostic. Use JS API to plug into other UI frameworks
⏱	Fast. And getting faster with every release!
✅	Proven. Powers Nozbe Teams since 2017 (and many others)
✨	Reactive. (Optional) RxJS API
🔗	Relational. Built on rock-solid SQLite foundation
⚠️	Static typing with Flow or TypeScript

Why Watermelon?

WatermelonDB is a new way of dealing with user data in React Native and React web apps.

It's optimized for building complex applications in React Native, and the number one goal is real-world performance. In simple words, your app must launch fast.

For simple apps, using Redux or MobX with a persistence adapter is the easiest way to go. But when you start scaling to thousands or tens of thousands of database records, your app will now be slow to launch (especially on slower Android devices). Loading a full database into JavaScript is expensive!

Watermelon fixes it by being lazy. Nothing is loaded until it's requested. And since all querying is performed directly on the rock-solid SQLite database on a separate native thread, most queries resolve in an instant.

But unlike using SQLite directly, Watermelon is fully observable. So whenever you change a record, all UI that depends on it will automatically re-render. For example, completing a task in a to-do app will re-render the task component, the list (to reorder), and all relevant task counters. Learn more.

📺 Next-generation React databases (a talk about WatermelonDB)	✨ Check out web Demo

Usage

Quick (over-simplified) example: an app with posts and comments.

First, you define Models:

class Post extends Model {
  @field('name') name
  @field('body') body
  @children('comments') comments
}

class Comment extends Model {
  @field('body') body
  @field('author') author
}

Then, you connect components to the data:

const Comment = ({ comment }) => (
  <View style={styles.commentBox}>
    <Text>{comment.body} — by {comment.author}</Text>
  </View>
)

// This is how you make your app reactive! ✨
const enhance = withObservables(['comment'], ({ comment }) => ({
  comment,
}))
const EnhancedComment = enhance(Comment)

And now you can render the whole Post:

const Post = ({ post, comments }) => (
  <View>
    <Text>{post.name}</Text>
    <Text>Comments:</Text>
    {comments.map(comment =>
      <EnhancedComment key={comment.id} comment={comment} />
    )}
  </View>
)

const enhance = withObservables(['post'], ({ post }) => ({
  post,
  comments: post.comments
}))

The result is fully reactive! Whenever a post or comment is added, changed, or removed, the right components will automatically re-render on screen. Doesn't matter if a change occurred in a totally different part of the app, it all just works out of the box!

➡️ Learn more: see full documentation

Who uses WatermelonDB

Does your company or app use 🍉? Open a pull request and add your logo/icon with link here!

Contributing

WatermelonDB is an open-source project and it needs your help to thrive!

If there's a missing feature, a bug, or other improvement you'd like, we encourage you to contribute! Feel free to open an issue to get some guidance and see Contributing guide for details about project setup, testing, etc.

If you're just getting started, see good first issues that are easy to contribute to. If you make a non-trivial contribution, email me, and I'll send you a nice 🍉 sticker!

If you make or are considering making an app using WatermelonDB, please let us know!

Author and license

WatermelonDB was created by @Nozbe.

WatermelonDB's main author and maintainer is Radek Pietruszewski (website ⋅ twitter ⋅ engineering posters)

See all contributors.

WatermelonDB is available under the MIT license. See the LICENSE file for more info.

watermelondb's Issues

Is watermelon safe to store sensitive data?

I just curious about it.

[Discussion] Absolute vs relative paths

Currently, most imports in 🍉 are absolute (i.e. import from 'Query'). This is mostly an artifact of the fact that it was extracted from a codebase of an app. But it's also very convenient when working on Watermelon.

But it adds A LOT of complexity, because every tool that has to understand the codebase has to be (often separately) configured:

• Babel (for building) - #22
• Jest
• Eslint (via eslint-babel)
• Flow
• Metro (for integration tests)
• the app and Jest in an app importing Watermelon (we use Babel to rewrite for dist)
• Flow in the app importing Watermelon (also using Babel but have to write a separate copy with Flow annotations — which has its own problems… - see #21 )

I see 3 options:

1. We keep the current setup (This is a big PITA)
2. We give up and just change the absolute paths to relative paths.
3. We meet halfway and change all paths to something like ~/blah or src/blah. I think it's not much better than 1. We still have to configure the same number of tools, it's just that there's one item in the config.

Personally I think it's easiest to go with (2). It's a little less convenient when refactoring, but I think overall a smaller maintenance burden. But I don't have experience with shipping NPM packages, and I don't know what other larger JS libraries do.

Native support for sorting

Hey,

it would be great to have native support for sorting according to a column (i.e. the timestamp) in order to avoid doing that in the JS thread.

Took a brief attempt to roll that on our end here - but I'm not sure about the API.

Using `globalObject: 'this'` in ejected CRA webpack.config.dev breaks the webpack-dev-server

Description of Issue

I am trying to set up a WatermelonDB project for web, by reading the documentation here. I set up a create-react-app app and ejected (by running the eject script) to allow me to edit the webpack.config.dev file so that I can add the necessary lines of code. However, the build breaks when I add globalObject: 'this' to the output object.

Error message in console

> $ npm start                                                                                                                                                                                        

> [email protected] start /Users/rorysmith/WebstormProjects/watermelon-test
> node scripts/start.js

Failed to compile.

Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
 - configuration.output has an unknown property 'globalObject'. These properties are valid:
   object { auxiliaryComment?, chunkFilename?, crossOriginLoading?, chunkLoadTimeout?, devtoolFallbackModuleFilenameTemplate?, devtoolLineToLine?, devtoolModuleFilenameTemplate?, filename?, hashDigest?, hashDigestLength?, hashFunction?, hashSalt?, hotUpdateChunkFilename?, hotUpdateFunction?, hotUpdateMainFilename?, jsonpFunction?, library?, libraryTarget?, libraryExport?, path?, pathinfo?, publicPath?, sourceMapFilename?, sourcePrefix?, strictModuleExceptionHandling?, umdNamedDefine? }
   Options affecting the output of the compilation. `output` options tell webpack how to write the compiled files to disk.

Expected Behaviour

webpack-dev-server not to crash.

Current Behaviour

webpack-dev-server crashes.

Possible Solution

Adding globalObject to the API schema. How do I do this? I don't know much about webpack.

Context (Environment)

create-react-app 1.5.2 which uses react 16.5.2, webpack 3.8.1 and webpack-dev-server 2.11.3.

Write up preferred way to reset local database

What would be the recommended way to trigger a complete deletion (and recreation) of the database?

Add automatic test coverage measurements

can Jest output this? do we need a separate tool for it?

would be great if measurements (and diff in test coverage) were output in pull requests. We could hook it up to Danger/Peril maybe.

Records not saved properly

Hello,

Thanks for all the work on the library, it's exactly what we need on react-native, the api & the code is slick :).

I have a very strange issue which is hard to debug, I spent about two days of debugging but I can't find the root cause of it. I used the examples/native as a base start of my project so everything is setup mostly the same way.

When I save a record, it works temporally (the observable query part works well) but it's not saved properly in the database, it just does not save data. So when I reload the app it does not work.

I've tried to debug watermelondb to find the root cause of it but I can't find why, what I know is that for some reason, the _raw object does not get set.

the model:

import { Model, Q } from '@nozbe/watermelondb'
import { field, children, lazy } from '@nozbe/watermelondb/decorators'


export default class Account extends Model {
  static table = 'accounts'

  @field('name')
  name

  @field('initial_balance')
  initialBalance

  @field('computed_balance')
  computedBalance

  @field('currency')
  currency
}

the schema file

import { appSchema, tableSchema } from '@nozbe/watermelondb'

export const mySchema = appSchema({
  version: 4,
  tables: [
    tableSchema({
      name: 'accounts',
      columns: [
        {name: 'name', type: 'string'},
        {name: 'initial_balance', type: 'number'},
        {name: 'computed_balance', type: 'number'},
        {name: 'currency', type: 'string'}
      ]
    }),
    tableSchema({
      name: 'transactions',
      columns: [
        {name: 'account_id', type: 'string', isIndexed: true},
        {name: 'amount', type: 'number'},
        {name: 'description', type: 'string'},
        {name: 'payed_at', type: 'number'}
      ]
    }),
    tableSchema({
      name: 'blogs',
      columns: [{ name: 'name', type: 'string' }],
    }),
    tableSchema({
      name: 'posts',
      columns: [
        { name: 'title', type: 'string' },
        { name: 'subtitle', type: 'string' },
        { name: 'body', type: 'string' },
        { name: 'blog_id', type: 'string', isIndexed: true },
      ],
    }),
    tableSchema({
      name: 'comments',
      columns: [
        { name: 'body', type: 'string' },
        { name: 'post_id', type: 'string', isIndexed: true },
        { name: 'is_nasty', type: 'bool' },
      ],
    }),
  ],
})

the boot.js file

import { AppRegistry } from 'react-native'

import { Database } from '@nozbe/watermelondb'
import SQLiteAdapter from '@nozbe/watermelondb/adapters/sqlite'

import { mySchema } from './src/models/schema'
import Blog from './src/models/Blog'
import Post from './src/models/Post'
import Comment from './src/models/Comment'
import Account from './src/models/Account'

import { createNavigation } from './src/components/helpers/Navigation'

const adapter = new SQLiteAdapter({
  dbName: 'testdb',
  schema: mySchema,
})

const database = new Database({
  adapter,
  modelClasses: [Account, Blog, Post, Comment],
})

const Navigation = createNavigation({ database })

AppRegistry.registerComponent('App', () => Navigation)

as you can see, everything is the same as the example project. I could also dump a copy of the whole repo if needed, there's nothing confidential in it, it's just a test.

to create the account I just do a

 await this.props.database.collections.get('accounts').create(account => {
   account.name = this.state.name;
   account.initialBalance = 0.0;
   account.currency = this.state.currency;
})

(the values are obviously normal, I've checked). And I do have this:

10-21 17:15:26.353 18537 22922 I ReactNativeJS: [DB] Executed batch of 1 operations (first: create on accounts) in 11.935499966144562ms

in the logs

it does work initially, you can see the new account in the view but when I reload the app, I just have an empty record from the database. I've also dumped the database and the records are indeed empty.

For the watermelondb version I've tried everything and it's still the same issue.

Does that sound like anything you have heard before?

An in-range update of prettier is breaking the build 🚨

The devDependency prettier was updated from 1.14.2 to 1.14.3.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

prettier is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

• ❌ continuous-integration/travis-ci/push: The Travis CI build failed (Details).
• ❌ Travis CI - Branch: The build failed.

---

Release Notes for 1.14.3

🔗 Changelog

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

Override entity IDs

Hey,

we are currently evaluating this for our app. Since we try to implement our own sync engine on top, I'd be curious if it's possible to specify entity IDs whenever you create one.

Right now, if I try to set an ID in the builder function, I get the following error:

Diagnostic error: Attempt to set new value on a property Model.prototype.id marked as @readonly

Great work by the way, this looks very promising!

Qn: In memory database for react-native

Hi,

I tried to use lokijs to run my test / storybook with sample data. Do you see issues using this adapter on react-native env. I currently get

error: bundling failed: Error: Unable to resolve module `fs` from `/Users/bsr/myApp/node_modules/lokijs/src/lokijs.js`: Module `fs` does not exist in the Haste module map

Write up "Getting Started"

A short tutorial for how to get from zero to a working demo would be nice

Online docs

Currently, the whole documentation is available just on GitHub in Markdown form.

It would be nice to have a website (like React Native, Jest, React Navigation, etc.) with those docs, and API reference automatically build from source.

It would be great if someone wanted to help out with this!

withObservables not working

I have schema, model and instance database.

User Model:

// User Model
// @flow

import { Model } from '@nozbe/watermelondb'
import { field, json, } from '@nozbe/watermelondb/decorators'
import { Tables, Columns } from './Schema.js'

const Column = Columns.users
const sanitizeBody = json => json

export default class User extends Model {
  static table = Tables.users

  @json(Column.body, sanitizeBody)
  body: string

  @field(Column.token)
  token: string

  async addUser (body: string, token: string) {
    const result = await this.collections.get('users').create(user => {
      user.body = body
      user.token = token
    })
    return result
  }

  async destroyPermanently () {
    await this.users.destroyAllPermanently()
  }
}

My Schema:

// Schema
// @flow

import {
  tableName,
  columnName,
  type TableName,
  appSchema,
  tableSchema,
} from '@nozbe/watermelondb'
import type User from './User.js'

export const Tables = {
  users: (tableName('users'): TableName<User>),
}

export const Columns = {
  users: {
    body: columnName('body'),
    token: columnName('token'),
  },
}

export const UserSchema = appSchema({
  version: 1,
  tables: [
    tableSchema({
      name: Tables.users,
      columns: [
        { name: Columns.users.body, type: 'string' },
        { name: Columns.users.token, type: 'string' },
      ],
    }),
    // ...
  ],
})

and called it

// @flow

import { Database } from '@nozbe/watermelondb'
import SQLiteAdapter from '@nozbe/watermelondb/adapters/sqlite'
import User from './User'
import { UserSchema } from './Schema'

const adapter = new SQLiteAdapter({
  dbName: 'Oootopia', // ⬅️ Give your database a name!
  schema: UserSchema,
})

const database = new Database({
  adapter,
  modelClasses: [User],
})

export default database

But when i tried to integrate withObservables i got an error.

My component is like

// class Auth
// ...
const enhance = withObservables(['user'], props => {
  console.log('props', props)
  return {
    user: props.user.body.observe(),
  }
})

const EnhancedAuth = enhance(Auth)

export default EnhancedAuth

it seems user is undefined, but I am not really sure what a missing. I hope there is a solution to this problem.
Edit: it works if i called via promise

await db.collections.get('users').find(id)

thank you so much.

Database provider pattern with context

Hello!

I've just been reviewing the examples and I'm wondering why there isn't a database provider using React context?

Was this an architectural decision?

If not, do you think the library would benefit from a provider? In my head it would just have a similar API to MobX or Redux providers, just to reduce the prop drilling and access to the database deep in the tree if needed:

<DatabaseProvider database={database}>
  <App />
</DatabaseProvider>

The context would be consumed like:

@withDatabase
const Posts = ({posts}) => {
  return posts && posts.map(post => <li>{post.name}</li>)
}
 // or
const Posts = ({posts}) => {
  return posts && posts.map(post => <li>{post.name}</li>)
}

export default withDatabase(Posts);

Perhaps the withObservables could call:

// rest of HoC
<Database.Consumer>
  {({database}) => (
   <Component database={database} {...props} />
  )}
</Database.Consumer>

Not 100% on the API.

Many to many Relationships

Thank you for your awesome work on this library!

My question:
Is there support (planed?) for many-to-many relationships?

Example:
I have a cookbook table. A cookbook belongs to a specific user.
I have a recipe table.
A recipe can belong to many cookbooks (a user can „favorite“ the recipe in his personal cookbook).

This could be solved using a associative/pivot table containing foreign keys to the cookbook table and the recipe table.

Write up Sync details

Database failed to load error

After some time spent to set up this package in react native, got this error:

[DB] Uh-oh. Database failed to load, we're in big trouble, [TypeError: undefined is not an object (evaluating 'Native.setUp')]

I am using expo for development.

Fix Metro settings so that it only looks into src/

Currently, integration tests -- yarn test:ios and yarn test:android fail if there's a dev/ or dist/ folder from the build script, or if node_modules are installed in examples/.

This is annoying and makes it so that we can't add integration tests as part of the publish script.

I tried changing project source to src/, but that would fail for some reason

Could someone take a look at this?

Raw Queries

Hi

Is there anyway we can run custom queries on database?

Is there anyway we can access database directly?

Write up Pro tips

Actions:

• invariants
• kwargs
• fast skip

Rendering:

• Prefetching!!!!
• Sorting

Performance tips:

• JSC + standard react basics
• prefetching
• minimy queries
• share query results (filter in js instead of doing multiple similar queries)
• batching

Web installation: webpack-dev-server crashes on last step from docs (LokiJS adapter)

Description of Issue

I am using these docs to set up a WatermelonDV project. I'm using this boilerplate. Installation is all fine up to the last step where I need to import LokiJSAdapter from '@nozbe/watermelondb/adapters/lokijs'.

Error message in console

> $ yarn start                                                                                                                                                                           [±master ●●]
yarn run v1.5.1
$ webpack-dev-server
(node:78559) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead
ℹ ｢wds｣: Project is running at http://localhost:8080/
ℹ ｢wds｣: webpack output is served from /
ℹ ｢wds｣: Content not from webpack is served from /Users/rorysmith/WebstormProjects/react-webpack-boilerplate/build
ℹ ｢wds｣: 404s will fallback to /index.html
/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/worker-loader/dist/index.js:121
      worker.factory = (0, _workers2.default)(worker.file, compilation.assets[worker.file].source(), options);
                                                                                           ^

TypeError: Cannot read property 'source' of undefined
    at worker.compiler.runAsChild (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/worker-loader/dist/index.js:121:92)
    at compile (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/webpack/lib/Compiler.js:296:11)
    at hooks.afterCompile.callAsync.err (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/webpack/lib/Compiler.js:553:14)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:15:1)
    at AsyncSeriesHook.lazyCompileHook (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/Hook.js:154:20)
    at compilation.seal.err (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/webpack/lib/Compiler.js:550:30)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeAssets.callAsync.err (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/webpack/lib/Compilation.js:1295:35)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.optimizeChunkAssets.callAsync.err (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/webpack/lib/Compilation.js:1286:32)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/Hook.js:154:20)
    at hooks.additionalAssets.callAsync.err (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/webpack/lib/Compilation.js:1281:36)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/tapable/lib/HookCodeFactory.js:32:10), <anonymous>:6:1)
error An unexpected error occurred: "Command failed.
Exit code: 1
Command: sh
Arguments: -c webpack-dev-server
Directory: /Users/rorysmith/WebstormProjects/react-webpack-boilerplate

Expected Behaviour

Presumably, this package should be available as part of @nozbe/watermelondb/adapters

Current Behaviour

webpack-dev-server crashes

Possible Solution

When I yarn add lokijs, the build runs successfully, albeit with the following warning:

    WARNING in ../node_modules/lokijs/src/lokijs.js
    Module not found: Error: Can't resolve 'fs' in '/Users/rorysmith/WebstormProjects/react-webpack-boilerplate/node_modules/lokijs/src'
     @ ../node_modules/lokijs/src/lokijs.js
     @ ../node_modules/@nozbe/watermelondb/esm/adapters/lokijs/worker/executor.js
     @ ../node_modules/@nozbe/watermelondb/esm/adapters/lokijs/worker/lokiWorker.js
     @ ../node_modules/@nozbe/watermelondb/esm/adapters/lokijs/worker/index.worker.js

Context (Environment)

"react": "16.5.2",
"webpack": "4.19.1",
"@nozbe/watermelondb": "^0.6.1",

How can I fix this properly?

No directory @nozbe/watermelondb/adapters

After installing the database, a error showed up saying

Unable to resolve module utils/commonfrom /Users/ .. /node_modules/@nozbe/watermelondb/adapters/sqlite/index.js: Module utils/commondoes not exist in the Haste module map

After inspecting the dependencies it looked like that the folder was located into the watermelondb/src/ folder. After copying the adapter folder to watermelondb/adapters, a new error occured, stating

Error: Command failed: /usr/libexec/PlistBuddy -c Print:CFBundleIdentifier build/Build/Products/Debug-iphonesimulator/test4.app/Info.plist
Print: Entry, ":CFBundleIdentifier", Does Not Exist

at checkExecSyncError (child_process.js:601:13)
at Object.execFileSync (child_process.js:621:13)
at Promise.then (/Users/ ... /node_modules/react-native/local-cli/runIOS/runIOS.js:208:5)
at <anonymous>

This breaks the app so hard that it will not build even after undoing everything related to WatermelonDB, removing the "build"-Folder, and so on.

I am using

"@nozbe/watermelondb": "^0.6.1",
"@babel/plugin-proposal-decorators": "^7.1.0",
"react": "16.5.0",
"react-native": "0.57.0",
Mac OS.

Invalid Greenkeeper configuration file

We have detected a problem with your Greenkeeper config file 🚨

Greenkeeper currently can’t work with your greenkeeper.json config file because it is invalid. We found the following issue:

1. Could not parse greenkeeper.json, it appears to not be a valid JSON file.

Please correct this and commit the fix to your default branch (usually master). Greenkeeper will pick up your changes and try again. If in doubt, please consult the config documentation.

Here’s an example of a valid greenkeeper.json:

{
  groups: {
    frontend: {
      packages: [
        'webapp/package.json',
        'cms/package.json',
        'analytics/package.json'
      ]
    },
    build: {
      packages: [
        'package.json'
      ]
    }
  },
  ignore: [
    'standard',
    'eslint'
  ]
}

This files tells Greenkeeper to handle all dependency updates in two groups. All files in the frontend group will receive updates together, in one issue or PR, and the root-level package.json in the build group will be treated separately. In addition, Greenkeeper will never send updates for the standard and eslint packages.

🤖 🌴

RFC: Creating firestore sync engine for watermelondb

Hey @radex @nozbetravis @Nozbe, first of all thank you and congratulations for the amazing work on watermelondb! I have been playing around with the API and am beyond impressed. Was thinking of creating a sync engine with Google Cloud Firestore as the backend as a weekend project and had some questions:

1. Is the sync engine API stable enough for such an undertaking?
2. Are docs for implementing new sync engines in the works or would you like me to convert my notes into basic first draft for docs?
3. I am planning to use the said engine on react-native possibly using the react-native-firebase project. Anyone know if this would be non trivial to implement with the existing adapters?
4. Would it be better to start a new adapter implementation from scratch or override the LokiJS adapter for example (maybe for supporting offline operations)?
5. (Bonus question) Would you use it if I built it? 🚽

[Discussion] Better dist format

Currently, we build and output Watermelon in two formats: ESM and CJS. If I understand correctly (@mobily knows this), it's because React Native needs CJS, but Webpack can do tree shaking with ESM. In #19, we add another format (same as source but with rewritten imports) to support Flow.

This complicates things because users have to include a Babel plugin to rewrite @nozbe/watermelon/xxx imports to @nozbe/watermelondb/{esm,cjs}/xxx.

1. Do we really need ESM for tree shaking? Any 🍉 user is almost surely going to be using almost all features, so I can't see how tree shaking could help. And all potentially tree-shakable things like adapters already require a separate import (import from '@nozbe/watermelondb/adapters') and are not exported from the main import
2. Can we just use CJS for all platforms and remove the need for a babel plugin?
3. We could still write the source code (if we fix #20) in dist, so that an advanced user could set up Babel, tree shaking, whatever they need in their app (We're probably going to do this at Nozbe, since we don't need e.g. class transpilation for React Native)

Request for TypeScript support

This project looks fantastic and it would be great to get TypeScript support as well. TypeScript has quite a big community but with this being quite a new project there doesn't appear to be a community definition yet. I'd be happy to have a crack at helping with this, but I have not written an index.d.ts definition before. Microsoft have a guide over at https://www.typescriptlang.org/docs/handbook/declaration-files/by-example.html.

about src/ package.json

Why does every file in the src/ directory have package.json? Does this make sense?

An in-range update of webpack is breaking the build 🚨

The devDependency webpack was updated from 4.20.2 to 4.21.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

webpack is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details

• ❌ Travis CI - Branch: The build failed.

---

Release Notes for v4.21.0

Features

• add output.libraryTarget: "amd-require" which generates a AMD require([], ...) wrapper instead of a define([], ...) wrapper
• support arrays of strings passed to output.library, which exposes the library to a subproperty

Bugfixes

• fix cases where __webpack_require__.e is used at runtime but is not defined in the bundle
• fix behavior of externals of global type

Performance

• Some performance improvements to the chunk graph generation

Commits

The new version differs by 37 commits.

• 432d2a3 4.21.0
• 0fb6c60 Merge pull request #7038 from marcusdarmstrong/marcusdarmstrong-external-module-fix
• 15b6f8b make afterEach async
• 7bc5c98 Merge branch 'master' into marcusdarmstrong-external-module-fix
• 2228daf Merge pull request #8230 from webpack/revert-8120-rh-silent-reporter
• fadf875 remove dependency
• 7c0b209 Revert "Re-enable jest-silent-reporter #hacktoberfest"
• a868789 Merge pull request #8143 from MLoughry/miclo/optimize-chunk-graph-generation
• 1d71ede Make changes suggested by @sokra to optimize chunk graph generation
• 4d3fe00 Merge pull request #8134 from fscherwi/update-coveralls
• 86f56bf update coveralls
• 4c461e2 Merge pull request #8120 from rickhanlonii/rh-silent-reporter
• 9fe42e7 Merge pull request #8118 from webpack/bugfix/issue-8110
• 0b6ad2a Don't be clever with the set command because idk windows
• 148016e Rerun yarn

There are 37 commits in total.

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

Column names not escaped properly

When using SQL keywords as column names, the application dies on launch.

Example:

tableSchema({
      name: 'ticket_statuses',
      columns: [
        { name: 'name', type: 'string' },
        { name: 'order', type: 'string' },
      ]
    })

while order is a SQL keyword. Stacktrace shows Syntax error near "order"

Any recommended way of preventing duplicate entries while inserting batch records

At the moment am using this custom functions to prevent duplicates. This works great but not sure if this is the recommended way of doing it. I wish there was a functionality such as firstOrCreate that either modifies properties or create a fresh record.

import { Q } from '@nozbe/watermelondb';

async function getExistingContactsBySids(db, contactIds: array<string>) {
  return await db.collections
    .get('contacts')
    .query(Q.where('sid', Q.oneOf(contactIds)))
    .fetch();
}

function makeContact(db, c, existingContacts) {
  const contactToUpdate = existingContacts.find(exC => exC.sid == c.id);
  if (contactToUpdate) {
    return contactToUpdate.prepareUpdate(contact =>
      transformContactForStore(contact, c)
    );
  }

  return db.collections
    .get('contacts')
    .prepareCreate(contact => transformContactForStore(contact, c));
}

function transformContactForStore(contact, c) {
  contact.sid = c.id;
  contact.username = c.username;
  contact.email = c.email;
  contact.firstName = c.first_name;
  contact.middleName = c.middle_name;
  contact.lastName = c.last_name;
}

export async function createLocalContactsFromAPI(db, contacts) {
  const contactIds = contacts.reduce((pv, cv) => {
    pv.push(cv.id);
    return pv;
  }, []);

  const existingContacts = await getExistingContactsBySids(db, contactIds);

  const finalContacts = contacts.map(c => makeContact(db, c, existingContacts));
  return await db.batch(...finalContacts);
}

An in-range update of react-native-prompt-android is breaking the build 🚨

The dependency react-native-prompt-android was updated from 0.3.3 to 0.3.4.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

react-native-prompt-android is a direct dependency of this project, and it is very likely causing it to break. If other packages depend on yours, this update is probably also breaking those in turn.

Status Details

• ❌ Travis CI - Branch: The build failed.

---

Commits

The new version differs by 3 commits.

• 4c41359 chore: bump version to 0.3.4
• b30c839 Merge pull request #37 from rozPierog/master
• 9354a8e Clean & update build.gradle

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.

---

Your Greenkeeper Bot 🌴

DatabaseBridge.swift - Expression pattern of type 'String' cannot match values of type 'String?'

Build works fine on Android, just on iOS having this issue. Tried inside of Xcode and from command line.

Xcode version: 9.2 (9C40b)

"dependencies": {
"@nozbe/watermelondb": "^0.6.0",
"@nozbe/with-observables": "^1.0.0",
"jsc-android": "224109.x.x",
"react": "16.5.1",
"react-native": "0.57.1",
"react-native-prompt-android": "^0.3.3",
"react-navigation": "^2.12.1"
},

Qn: props for `withObservables`

Hi,
Looking at the example code, I see this comment

   // We have to use a little wrapper because React Navigation doesn't pass simple props (and withObservables needs that)

I know, react-navigation wont pass the props, and normally you achieve this with extending the navigator

Please let me know what all props withObservables needs to be passed through?

thanks.
bsr

test

aaa

Inspecting an item using react-devtools throws an error

Steps to reproduce

1. Install https://www.npmjs.com/package/react-devtools
2. Run react-devtools
3. Toggle inspector from react-native dev options cmd+D on simulator
4. Click on an element that is coming from the DB
5. Observe the attached screenshot error is thrown

Can you surport Electron?

Oh! this is a powerful way to edit database!

Can you surport Electron?

[Migrations] Allow custom manual data migrations

#62 (comment)

This is going to be tricky to do, because:

• we can only (reasonably) work on untyped data
• it messes up the queuing logic. In the custom data migration block you need to be able to call find, query, and batch, but they need to be somehow treated differently than the enqueued find/query/batch that happens after the setup
  • changing new Adapter(xx) to adapter = await Adapter.setUp(xxx) would fix this, but we'd sacrifice launch performance this way, which sucks

Add backward compatible migrations for database schema changes

First, thank you for such promising tool. I'm considering to use WatermelonDB in react-native app. For me that's will be enough possibility to recreate local database from scratch. As I see now the easiest way is to create new db and drop old one. So, as I understand for creating we can just change db name? Is there some possibility to drop old db?

Documentation about Native Access

There is currently no documentation about native ( android, ios ) access to the db. Is it possible with the current api, or is it done manually via sqlite, or is there simply no documentation ?
Thanks

Unhandled Promise Rejection if column doesn't exist

Hi,

I am getting an unhandled exception error.

It is because I renamed the field, and may solve by recreating or changing the version. But like to know where I can handle similar error in the future. Can you please give an idea which API call is resulting this error.

Possible Unhandled Promise Rejection (id: 0):

Exception has occurred: Error
Error: table positions has no column named lat (code 1 SQLITE_ERROR): , while compiling: insert into positions ("id", "_status", "_changed", "last_modified", "lat", "lng") values (?, ?, ?, ?, ?, ?)
    at createErrorFromErrorData (/Users/bsr/myapp/node_modules/react-native/Libraries/BatchedBridge/NativeModules.js:146:32)
    at /Users/bsr/myapp/node_modules/react-native/Libraries/BatchedBridge/NativeModules.js:95:31
    at MessageQueue.__invokeCallback (/Users/bsr/myapp/node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:397:13)
    at /Users/bsr/myapp/node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:127:12
    at MessageQueue.__guard (/Users/bsr/myapp/node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:297:9)
    at MessageQueue.invokeCallbackAndReturnFlushedQueue (/Users/bsr/myapp/node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:126:10)
    at /Users/bsr/myapp/.vscode/.react/debuggerWorker.js:131:58
    at process.<anonymous> (/Users/bsr/myapp/.vscode/.react/debuggerWorker.js:40:9)
    at process.emit (events.js:182:13)
    at emit (internal/child_process.js:812:12)

Action required: Greenkeeper could not be activated 🚨

🚨 You need to enable Continuous Integration on all branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please delete the greenkeeper/initial branch in this repository, and then remove and re-add this repository to the Greenkeeper App’s white list on Github. You'll find this list on your repo or organization’s settings page, under Installed GitHub Apps.

Wrong path to MainApplication.java in docs

quickly jotting this down:

in https://github.com/Nozbe/WatermelonDB/blob/master/docs/Installation.md there's

And finally, in android/src/main/java/{YOUR_APP_PACKAGE}/MainApplication.java, add:

The path here is wrong, it's supposed to be android/app/src/main/java/...

Err on attempt to get list of child objects

The query gets the parent object from db by id. -> MyObjectInstance

MyObjectInstance = await database.collections.get('parentTableName').query(Q.where('id', id)).fetch();

When I try to to get MyObjectInstance.collectionOfAssociated, I get:

Exception: TypeError: Cannot read property 'query' of undefined at MyObjectInstance.get
at MyObjectInstance.remoteFunction (:2:14)]

But it works when I do the query on child Model:

await database.collections.get('childTableName').query(Q.where('parent_id', MyObjectInstance.id)).fetch();

Can you please help me to understand why this is happening? When I use your samples that do calls from UI, everything works as expected, it seems to return list of child objects.

Thank you!

Improve web demo for mobile

• remove 300ms delay on iOS
• add the CSS prop iOS needs to get smooth scrolling views
• responsive layout?

Contributions welcome!

[Discussion] Better Flow setup

In #19 I fixed support for using Flow in apps importing Watermelon.

But there's a couple of problems:

1. We can't just export the source code, because we have to rewrite imports first (see #20), which complicates the build process
2. Babel screws up the $FlowFixMe comments in the output files, so we have to re-check Flow in an app importing Watermelon
3. We have to build a separate package for Flow. Use https://babeljs.io/docs/en/babel-plugin-transform-flow-comments maybe to use the same exported code as apps use?
4. Flow in an app will flow-check the entire Watermelon, which takes more time and is likely to break if there's a mismatch of Flow version used in Watermelon and the app. We could manually write flow-typed, but that's extra maintenance burden.

Can we generate flow-typed automatically from source?.

Or, alternatively: Can apps configure Flow to only import types, but don't actually output errors for Flow issues inside Watermelon?

Require cycles

Hi just noticed this warning on chrome console while debugging.

require.js:109 Require cycle: node_modules/@nozbe/watermelondb/Collection/index.js -> node_modules/@nozbe/watermelondb/Query/index.js -> node_modules/@nozbe/watermelondb/observation/observeQuery.js -> node_modules/@nozbe/watermelondb/observation/simpleObserver/index.js -> node_modules/@nozbe/watermelondb/Collection/index.js

Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle.

Is it possible to avoid this?

thanks.

Request for improved debug logs

This is really a great project and am planning to use it in most of my upcoming projects.

While integrating it to an existing project I have run into this issue and I have no way of knowing what am doing wrong.

Checking the swift line on the screenshot and without prior swift knowledge, I just can't figure out how to proceed.

This is my code:-

export default class Contact extends Model {
  static table = 'contacts';

  @field ('sid')
  id;
  @field ('username')
  username;
  @field ('email')
  email;
  @field ('first_name')
  firstName;
  @field ('middle_name')
  middleName;

  static makeContact = (db, c) =>
    db.collections.get ('contacts').prepareCreate (contact => {
      const n = transformRemoteContact (c);
      contact.sid = n.sid;
      contact.username = n.username;
      contact.email = n.email;
      contact.first_name = n.first_name;
    });

  // Model actions goes here.
  static async createContactsFromAPI (db, contacts) {
    const cont = transformRemoteContact (contacts[0]);
    const finalContacts = contacts.map (c => Contact.makeContact (db, c));
    console.warn (finalContacts[0]);
    return await db.batch (...finalContacts);
  }
}

Please help. Thanks.

[Migrations] Warn about incompatible schemas

We could make schemas and migrations safer by warning in development mode about incompatible schemas.

Example:

• I run the app with schema version 5, and tables A, and B
• I change the schema to have tables A, B, C, but don't change the schema version

Currently: nothing will happen. If developer resets database, table C will work, and you'd be able to ship the change to the schema, which could break badly for users, because there's no schema version change (and no migration)

It should be pretty easy to calculate a hash of the schema structure (in development only), and save it to the database, alongside version number. If app is run with same schema version but different schema hash, a warning/error would be shown.

PRs welcome!

Improve error messages (for non-Flow users)

There is a lot of places in Watermelon where it's easy to make a mistake and make the app break with a strange error message — except that Flow can catch those errors before we even run the code.

Still, some of those errors may be hard to understand to someone not familiar with Watermelon internals and especially if you don't use Flow.

A good example is Schema:

WatermelonDB/src/Schema/index.js

Line 54 in e69aa85

isDevelopment && invariant(name, `Missing table name in schema`)

— if you mis-format the schema, Watermelon will throw errors at you.

Some examples of Watermelon not doing a great job:

• forgetting static tables or static associations
• making a Q.on query without the proper association set
• making a @children field without an association
• wrong column name in @field and other decorators
• Database initializer - if you forget to pass some fields, it will blow up somewhere else
• … probably many more — add in comments stuff you came across

We won't be working on this, since we rely on Flow, but it should be easy for anyone to implement and contribute back to Watermelon :)

Is it okay to use WatermelonDB with MobX?

I'm using MobX in my project.
After reading the docs, I think I might have to choose between MobX + some other DB(eg. pouchDB) or WatermelonDB, since both MobX and WatermelonDB will automatically update components with 'observing' feature.
Is this right? or is it okay to use MobX and WatermelonDB together?

Also, I use MobX's 'computed' feature a lot in my project.
For example, there's @observable myColor, and whenever user changes myColor,
@computed myColorMatrixValue have to be calculated automatically. (I want to persist myColorMatrixValue, also.)

Is there a way to do this in WatermelonDB?