Incompatibility with pg-promise event receive (with camel casing) about massive-js

[dmfay]

Your guess that the camel casing is interfering with introspection seems to be on the money :) The receive option can be set after initialization, so you'll want to start looking in the reload function in lib/database.js to ensure that the driver (this.pgp) doesn't have it set while the reload is actively going on.

[johnbiundo]

@dmfay: Thanks for the pointer. I've tried to figure out how to set an option after initialization, but can't seem to suss it out. Would you have a further pointer for that?

(It seems I need to access the $config.options member of the underlying db connection, but that is not the same thing as db.pgp. Constructs like db.pgp.$config don't resolve. I'm probably off the track on how I'm looking for this, so would welcome any help).

My thought was to see if I could get this to work by setting up receive post initialization and ignore the reload function until after I prove that the startup condition works.

[dmfay]

I had the name wrong -- try this.instance (the actual instantiation of pg-promise) not this.pgp.

[johnbiundo]

Awesome, that did it, thanks so much.

@dmfay: For my use case, this is probably all I need, as my DB is static and doesn't need to be reloaded during the lifetime of the node process. However, as I offered, I'll be willing to take a look at doing something that might be useful in the more general case if you think it would be worthwhile.

In case anyone else needs this, here's the change from the original problem example above. It's just a matter of setting the receive option after connecting (to avoid fouling up the introspection process), in the promise resolution, instead of in the driver settings. The magic bit is using db.instance.$config.options to access the driver settings.

Like so:

return massive(massiveConnSettings, {}, driverSettings).then(db => {
  console.log('connected');
  db.instance.$config.options.receive = (data, result, e) => {
    camelizeColumns(data);
  };

  db.test.insert({ column_one: 1 });
  db.test.find({}).then(resp => {
    console.log('test rows: ', resp);
  });

[dmfay]

I think it'd be useful! What you'd need to do is cache instance.$config.options.receive in a variable at the beginning of the reload function, delete it, and put it back at the end. For testing, look in test/connect.js; you can copy and paste the "returns a database connection" testcase and add the driver config with receive to validate the new behavior. It should be fairly straightforward as these things go, but let me know if it gets confusing.

[johnbiundo]

OK, I'll take a look as soon as I can, and won't hesitate to ask for help :)

[johnbiundo]

@dmfay Feel free to say no if my experience level appears inadequate for this task. I'm pretty willing to roll up my sleeves, and go backfill areas I need more background on to proceed, but I probably need a little help getting started, and don't want to be a burden.

For what it's worth, I'd probably rate myself a good and diligent, but not greatly experienced JS developer. I do have a ton of RDBMS experience, including primarily PostgreSQL the last couple of years (but going back, believe it or not, to the early Ingres days, when I worked there as a support person), but mostly as a user, not a developer. And FWIW, I'm greatly interested in becoming a useful contributor to an OS project like this one.

Anyway, I've cloned the repo and wanted to start by running tests, but I'm struggling a little to get them to run properly. A first attempt yielded many failures :-( Is there any testing documentation? I presume I need to set up a DB with permissions and stuff, and could probably reverse engineer that by looking at a bunch of tests, but maybe there's a faster way?

Thanks in advance for any help, and willing to participate, or not, at whatever level makes sense.

[johnbiundo]

Oops. Just found https://github.com/dmfay/massive-js/blob/master/CONTRIBUTING.md

I'll start there :-)

[johnbiundo]

@dmfay OK, progress update.

I've added a few lines to the reload function in database.js:

at the top...

Database.prototype.reload = function() {
  this.clean();

  // cache pg-promise receive option config
  const receiveOptionCached = this.instance.$config.options.receive;
  delete this.instance.$config.options.receive;

and at the end...

        // restore pg-promise receive option config
        this.instance.$config.options.receive = receiveOptionCached
          ? receiveOptionCached
          : null;

        return this;
      });
  });
};

One issue and one question:

1. Issue: the transactions > withTransaction > reloads and applies DDL: test fails with:

  1) transactions
       withTransaction
         reloads and applies DDL:
     TypeError: Cannot read property 'options' of undefined
      at Database.reload (lib/database.js:66:72)
      at db.withTransaction.co.wrap.mode.db.pgp.txMode.TransactionMode.tiLevel (test/database/with.js:164:23)
      at db.withTransaction.co.wrap.mode.db.pgp.txMode.TransactionMode.tiLevel.next (<anonymous>)
      at onFulfilled (node_modules/co/index.js:65:19)
      at process.internalTickCallback (internal/process/next_tick.js:77:7)

All other tests pass, including a manual test passing in a receive: camelCase function (exactly like my original code at the top of this report - yay! :) ). I couldn't really figure out why this one failed.

2. Should I save and restore an empty options.receive as null? undefined?

Once I get your feedback, I can build a test and send a PR.

Thanks for your assistance!

[dmfay]

1. It failed because this.instance.$config was undefined when you tried to pull options.receive off. I think that's a peculiarity of reloading within a transaction instead of the global context; just make sure $config exists before you try to do anything with it.
2. I'd check whether receiveOptionsCached is truthy (if (!!receiveOptionsCached) ...) before restoring; that way, if there wasn't anything there you don't need to worry about what to put back.

[dmfay]

Everything looks good, thank you! I've released v5.7.5.

If there's other stuff you're interested in tackling, feel free to have at it and I'll do my best to field questions :)

[johnbiundo]

Awesome. OK, this was my first OSS contribution. I'll keep an eye out for more.... I may get addicted :)

Thanks for supporting the effort.

[dmfay]

congrats! And beware, it's definitely habit-forming....