Coder Social home page Coder Social logo

seneca-postgres-store's Introduction

Seneca

A Seneca.js data storage plugin

seneca-postgres-store

npm version Build Status Dependency Status Coveralls Gitter

Voxgig This open source module is sponsored and supported by Voxgig.

Description

seneca-postgres-store is a PostgreSQL database plugin for the Seneca MVP toolkit. The plugin is using the node-postgres driver. For query generation it uses internally the seneca-standard-query plugin and the standard functionality can be extended by using the seneca-store-query plugin.

Usage:

var Seneca = require('seneca');
var store = require('seneca-postgres-store');

var DBConfig = {
  name: 'senecatest',
  host: 'localhost',
  username: 'senecatest',
  password: 'senecatest',
  port: 5432
}
...

var si = Seneca(DBConfig)
si.use(require('seneca-postgres-store'), DBConfig)
si.ready(function() {
  var product = si.make('product')
  ...
})
...

Seneca compatibility

Supports Seneca versions 1.x - 3.x

Supported functionality

All Seneca data store supported functionality is implemented in seneca-store-test as a test suite. The tests represent the store functionality specifications.

Usage

You don't use this module directly. It provides an underlying data storage engine for the Seneca entity API:

var entity = seneca.make$('typename')
entity.someproperty = "something"
entity.anotherproperty = 100

entity.save$(function (err, entity) { ... })
entity.load$({id: ...}, function (err, entity) { ... })
entity.list$({property: ...}, function (err, entity) { ... })
entity.remove$({id: ...}, function (err, entity) { ... })

Query Support

The standard Seneca query format is supported. See the seneca-standard-query plugin for more details.

Extended Query Support

By using the seneca-store-query plugin its query capabilities can be extended. See the plugin page for more details.

Column name transformation, backward compatibility

In seneca-postgres-store 2.0 the internal CamelCase to snake_case column names conversion was removed.

To update from seneca-postgres-store 1.x to 2.x on systems built with seneca-postgres-store 1.x you must provide to the plugin through its options the functions that do the CamelCase to snake_case conversion and back. Any other name transformations to and from database column name can be also made with these. Example:

var DefaultConfig = {
...
  fromColumnName: function (attr) {
    // apply some conversion on column names
    return attr.toUpperCase()
  },
  toColumnName: function (attr) {
    // convert back column names
    return attr.toLowerCase()
  }
}
seneca.use(require('seneca-postgres-store'), DefaultConfig)

For a fully functional CamelCase to snake_case implementation sample please look in the postgres.test.js at the 'Column Names conversions' test code.

Limits

By default queries are limited to 20 values. This can be bypassed by passing the nolimit option, which if set to true will not limit any queries.

Fields

To filter the fields returned from the list operation, pass a fields$ array of column names to return. If no fields$ are passed, all fields are returned (i.e. select * is used). e.g.

query.fields$ = ['id', 'name']

Note: The implicit id that is generated on save$ has uuid value. To override this you must provide entity.id$ with a desired value.

Custom ID generator

To generate custom IDs it is exposed a seneca action pattern hook that can be overwritten:

seneca.add({role: 'sql', hook: 'generate_id', target: <store name>}, function (args, done) {
  return done(null, {id: idPrefix + Uuid()})
})

Native Driver

As with all seneca stores, you can access the native driver, in this case, the pg connection object using entity.native$(function (err, connectionPool, release) {...}). Please make sure that you release the connection after using it.

entity.native$( function (err, client, releaseConnection){
  // ... you can use client
  // ... then release connection
  releaseConnection()
} )

Contributing

The Senecajs org encourages open participation. If you feel you can help in any way, be it with documentation, examples, extra testing, or new features please get in touch.

To run tests with Docker

Build the PostgreSQL Docker image:

npm run build

Start the PostgreSQL container:

npm run start

Stop the PostgreSQL container:

npm run stop

While the container is running you can run the tests into another terminal:

npm run test

Testing for Mac users

Before the tests can be run you must run docker-machine env default and copy the docker host address (example: '192.168.99.100'). This address must be inserted into the test/default_config.json file as the value for the host variable. The tests can now be run.

License

Copyright (c) 2012 - 2016, Marian Radulescu and other contributors. Licensed under MIT.

seneca-postgres-store's People

Contributors

adrieankhisbe avatar banzaiman avatar ckiss avatar cristianianto avatar david-cahill avatar dberesford avatar greenkeeperio-bot avatar iantocristian avatar lilsweetcaligula avatar marianr avatar maxnachlinger avatar mcollina avatar mihaidma avatar mirceaalexandru avatar nherment avatar pelger avatar piccoloaiutante avatar rionastokes avatar rjrodger avatar savardc avatar shanel262 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

seneca-postgres-store's Issues

Query error when using reserved words as column names

I am trying to sort by "when" column - that is created by seneca-user.

The resulting query is something like this:

# SELECT * FROM sys_login ORDER BY when DESC;
ERROR:  syntax error at or near "when"
LINE 1: SELECT * FROM sys_login ORDER BY when DESC;

the following query is running correctly:

# SELECT * FROM sys_login ORDER BY "when" DESC;

Some guide on using an entity with an auto generated 'id' field in the database

A default 'id' field of type UUID seems to be automatically generated even if I do not specify one. There seems to be some sorcery involved using the id$ field that is not clearly documented - what is needed while doing a save$, is for the id field not to be even part of the save process, but that the callback have the (saved) id set on the entity. Is it possible to do this, and how ?

Update seneca-store-test to 1.1.0

Update the seneca-store-test dependency to 1.1.0. Over 50 tests added to the new store test version.
29 out of 95 tests are failing, these must be investigated and fixed after update

Audit packages

Audit packages and eliminate vulnerabilities, if any.

JSON data type

Does this adapter support the new Postgre JSON data type and searching with it? It allows Postgre to be used effectively as a document store.

duplicated id column for fields$

When using fields$ if the id is specified explicitly then the id is added twice in the generated query.
foo.list$({ids: ['foo1', 'foo2'], fields$: ['id']} generates SELECT id, id FROM foo WHERE

This appears on both cases: when id or array of ids is used.

UUID awareness and overriding the entity.id$

Please add to the readme that by default the entity.id field is set to UUID()

However, if your PostgreSQL instance isn't setup you will not be able to insert new records.
CREATE EXTENSION 'uuid-ossp'; needs to be enabled on the database.

Also to override the entity.id field please note that you have to call entity.id$

2.0.0 Release notes

Summary

Removed the internal conversion that the plugin was making from CamelCase column names to Snake case. Backward compatibility is ensured by allowing the user to pass into options two functions named toColumnName() and fromColumnName() that make this conversion. These should implement the CamelCase to Snake case conversion. More details are provided in the seneca-standard-query Column name transformation, backward compatibility section.

All query generation code related to basic seneca functionality was moved to seneca-standard-query and the extended query functionality moved to seneca-store-query. This doesn't change functionality but enables functionality reuse into other stores.

Change Log

Please view the change log for a list of changes through the years.

Please visit http://senecajs.org/ for more information on support and how to contribute

Support for schema

If connection object contains schema then this should be used when building queries.

Support for pool

node-pg supports pooling whereas we create a new connection every request in the current scenario.
Was there any reason to not use them in the first place?

undefined 'fromColumnName'

On gitter one user reported an undefined fromColumnName issue. This is the text:

when running unit test with seneca-postresql-store, I got these error

1) user profile test happy:
     Uncaught TypeError: Cannot read property 'fromColumnName' of undefined
      at Object.internals.transformDBRowToJSObject (node_modules/seneca-postgresql-store/lib/postgresql-store.js:299:26)
      at node_modules/seneca-postgresql-store/lib/postgresql-store.js:184:35
      at .<anonymous> (node_modules/seneca-postgresql-store/lib/postgresql-store.js:123:18)

I check source and need to work around to fix this issue

internals.transformDBRowToJSObject = function (row) {
    var obj = {}
   // here where I work around 
    if (StandardQuery == null) {
      StandardQuery = seneca.export('standard-query/utils')
    }
    for (var attr in row) {
      if (row.hasOwnProperty(attr)) {
        obj[StandardQuery.fromColumnName(attr)] = row[attr]
      }
    }
    return obj
  }

Fix reconnect

Because the error function has the call

seneca.fail({code: 'entity/error', store: name}, cb)

it will never retry connection as the following code is never executed.

repo access

Hi @marianr,

can you please add me to the repo contributors so that I can land the branches ?

Thanks a lot !

  • Nicolas

Safe mode : ensure param is not undefined

On safe mode, an object passed with a key and an undefined value is considered as an incomplete query.
This way, when a query is done with an undefined value it will not return the first value found based upon others criterias, but an error or empty object ?
This is to ensure security so that the created query object contains all the required params and doesn't run with incomplete criterias.

What happened to the default limit?

What this intentional?

This is what it currently does:
https://github.com/senecajs/seneca-postgres-store/blob/master/lib/query-builder.js#L396-L398

This is a pretty big problem for people running this code across a microservice fleet right now especially if they are unaware of this. I believe the intention is to keep the 20 item limit since that is mentioned in comments.

The code used to read (we have it on an old private repo):

-    if (q.limit$) {
 -      mq.params.push('LIMIT ' + q.limit$)
 -    }
 -    else {
 -      if (nolimit === false) {
 -        mq.params.push('LIMIT 20')
 -      }
 -    }

native$ no longer passes `release` function

Using native$ is broken under [email protected]. release is no longer being passed to the callback.

Before, handle_result would be called apply with arguments: https://github.com/senecajs/seneca/blob/v3.3.0/seneca.js#L1029

By nature, the release function would come through.

Now, arguments is no longer passed and so this parameter gets lost:
https://github.com/senecajs/seneca/blob/v3.4.2/seneca.js#L904

native is pretty much broken without the ability to close the connection.

I'm not sure if there's a way within seneca to pass additional arguments to callback functions anymore. To mitigate this, it might be good to manage a pool internally within the store and emit a new client when native$ is invoked. Returning the client will allow end-users to call end on it.

This is, AFAIK, available on later versions of pg. This module, as it stands will need to make a few changes to work with pg@7 but I think it's probably a good call to perform the upgrade.

We've already forked this module internally to support schemas.... I'm going to do some work to upgrade to pg@latest and fix up this native issue. I'll see if I can't put together a PR for this repo.

Add compatibility for seneca 2.0

Some tests fail with seneca 2.0. Seem to be related to the ready() functionality in seneca 2.0.
Investigate it and fill a bug report on seneca.

migration capabilities

Hi,

are there any migration capabilities for seneca-postgres? I have worked with sequelize before and they offer good tools for working with postgres.

change updatestm function

var fields = ent.fields$()
var entp = relationalstore.makeentp(ent)

fields should be created after makeentp is called. It should also be created from entp fields because makeentp can add 'seneca' field and this should be considered in the update stm

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.