Testing database functions locally about firebase-functions

[laurenzlong]

Hey malikasinger1, you can check out our documentation for how to write unit tests: https://firebase.google.com/docs/functions/unit-testing

Google Cloud Functions also open-sourced a local emulator, and we are working to build a tighter integration with Cloud Functions for Firebase. In the meanwhile, you can check it at here: https://github.com/GoogleCloudPlatform/cloud-functions-emulator/

[mismith]

Any update on the local Database events @laurenzlong?
Loving functions overall, but developing is currently a nightmare..

[laurenzlong]

Hey Inzamam, thanks for the clarification. I accidentally copied and pasted the unit testing link twice. I've revised my response above. The emulator does allow you to run functions locally. Here's the documentation that explains how to use it: https://cloud.google.com/functions/docs/emulator

[laurenzlong]

@stephenhandley Just released! https://github.com/firebase/firebase-tools/releases/tag/v3.8.0
Only HTTPS events for now, but other ones are coming!

[laurenzlong]

Thanks for following up @msjaber , we've done the first step, which is to enable the Google Cloud Functions emulator to support all event types: googlearchive/cloud-functions-emulator#121. You can use it now, and instructions are in the PR.

I'm now working on integrating this into the Firebase CLI, with a repl-like interface to invoke these emulated functions. I've been running several usability tests to make sure we get the experience right before releasing.

[jaufgang]

I think the locally deployed database function should definitely listen on a real database rather than being invoked through some terminal command. Whether it listens on a production or test DB would be up to the developer, they should be able to choose which database by calling firebase use before deploying locally.

I like @ahaverty's suggestion that when you run a database function locally, it could send a message to firebase to temporarily disable the cloud listener for the same function, and when the local function listener disconnects the cloud function automatically would take over again. That may not be necessary in all cases though, since it may be innocuous to have multiple listeners on the same node depending what the function does. Perhaps a command line switch could specify whether to temporarily disable the cloud function, and by default it would not. Something like:

firebase serve -disable-cloud --only functions:function1

This would disable the cloud listener of function1, while function1 was being served locally. If there were a function2 in the cloud, the listener for function2 would not be affected.

If you omitted the -disable-cloud switch, the cloud and local listeners for function1 would both be triggered.

If -disable-cloud is invoked by two different developers on the same function at the same time, the second one should just see an error message that this action is currently disallowed because another -disable-cloud is already being used on that function. Dev teams could avoid this scenario by using different test databases for different developers if they choose.

[laurenzlong]

Thanks for the feedback and lively discussion everyone!

I see people are divided on the issue of whether local functions should be triggered with the real database tied to the project. What if this was an opt-in behavior? It would not happen by default, but would only happen with an additional flag or something, so you are explicitly telling the CLI "This is not my production project, it is ok to trigger local functions with real database writes". And doing this would stop deployed functions from firing while the local functions are still running.

@fej-snikduj We'd be talking about months due to the complexity involved. I think we will likely do both (triggering local functions with mock events and with real database events). They serve slightly different use cases - mock events are great for testing without side effects, and real database is useful for end-to-end integration testing.

@atomicleopard thanks for the ambitious ideas! No ideas are too wild ;) In terms of breakpoints, we are actually working on supporting breakpoints in locally deployed functions so that you can debug them from an IDE.

@ahaverty Supporting json files is a great idea. If we were to do it, I'm thinking it would make the most sense to be consistent with the usage of firebase:database: commands. So it would look like:
firebase functions:invoke:makeUpperCase /input/path makeUpperCaseTest_valid.json

@mismith What do you think about invoking the function with sample foobar data if there was no data passed in via the flag? Trying to invoke it with the existing data would require the CLI to actually read the database to see what data is there -which would be slow. As well, the same data would not actually trigger the function, since only update/create/delete events will trigger a function.

[laurenzlong]

Hey @atomicleopard this is something that we are still working on. Stay tuned!

[laurenzlong]

@mismith
For #1, it depends on how you declared the function, if you had:

var functions = require('firebase-functions')
var createFunction  = functions.database.ref('/path').onCreate(...)
var deleteFunction = functions.database.ref('/path').onDelete(...)

Then in the shell:

createFunction('new_data') // would mock a create event
deleteFunction('old_data') // would mock a delete event

As for auth, it is {admin: true} or {admin: false} or {variable: {uid: 'abcd' }},

[laurenzlong]

Thanks for the thorough feedback everyone! To answer your questions, a terminal command would probably look something like:

firebase serve
firebase functions:invoke:makeUpperCase /input/path -d '{"word":"foo"}'

Thanks for sharing the perspective that "intercepting" live traffic with a test function could be disastrous for teams. For testing against a real database, I imagine there being a test project that is the default project for "firebase serve". So both hosting and functions would use the test project. This way, you can test interactions like a) user fills out a form and that data gets stored to the database b) a function is invoked and transforms that input c) function writes back to the database d) the new database value affects the web page UI somehow.

The test project can be set using "firebase use --add", or maybe it can be a part of the "firebase init" flow. If a test project is not set, then "firebase serve" would use the production project but give a warning?? I'm not sure about what's best in that case.

Supporting the terminal command is achievable in the short term, but supporting real database triggers would take more time.

[atomicleopard]

Seems a bit sketchy to disable a managed environments distributed event triggers because a script is running somewhere on another machine. If this went wrong it would be very hard to isolate, and hard to coordinate within a team. One of the challenges of firebase functions is that if you miss a write event, your chance to process is missed entirely without being recorded, and this would exacerbate this problem. I've also found that in practice debugging functions as it stands isn't the same as normal operations, because you're generally triggering data writes yourself in the firebase database viewer. This means any operations using the admin sdk inherit different permissions than real usage. Being able to push data in locally without impacting a deployed environment is a great first step, and would facilitate basic testing, which would be a huge win.

…

On Sun, 11 Jun 2017 at 4:28 am, Alan Haverty ***@***.***> wrote: I like the sound of that @jaufgang <https://github.com/jaufgang> , perhaps cloud disabling true by default though as keeping cloud functions running wouldn't be a use case for us right now and may cause confusion with novice users in future (but we could live with either! 👌) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQdDnc33h7oZBBTOJJYdKK8gHcx0CQbks5sCuBfgaJpZM4MaKU0> .

[ahaverty]

We're using separate projects for production/sales/Dev/Dev2 .
Dev2 if more than one of us are working on functions/rules simultaneously.
Local deployed (debuggable?) functions via Firebase tools cli (or IDE/IntelliJ support?) would be great, mainly to avoid the delays with deploying.
I can't imagine how this would work alongside remotely deployed functions, maybe you could give us an idea of how you're thinking of architecting it?
Could a local deployed function overrule a remote one? And automatically switch back on once it's killed locally? How would that work then with multiple local deployments? 🤔
Can't wait👌

[puf]

doing this would stop deployed functions from firing while the local functions are still running.

I love this idea. Especially if I could similarly pause deployed functions from running through an API or the Firebase console. While my returning of promises has improved, I still have the occasional deployed function go wild and it would be great to have a panic button. :-)

[jspri]

I've developed firebase-functions-local to help with my own testing while the firebase team finishes off their own version.

It's handy because you can test without uploading the functions which makes the dev cycle much quicker. You can also attach a debugger to help you get a better idea of what is going on.

All triggers run off live data in the realtime database. If you have a smaller app you could even use this as the back-end for pesky functions that don't get touched often when you want to still have a lower response latency than the cloud function warmup time. Once they start getting hit often it's straightforward to migrate them to the cloud.

[laurenzlong]

Just made a PR to firebase-tools to the emulation of all even types, and a shell to invoke them: firebase/firebase-tools#428.

@Crazometer this is really neat! Thanks for sharing. My PR uses fake data, whereas yours uses real DB events, so they complement each other quite well and are useful for different scenarios.

[mismith]

@laurenzlong I'm curious, so I looked over your PR—and maybe I'm missing something or confused—but why/how do the following two lines perform distinct actions?

# invoke onCreate function
myFunction('new_data')

# invoke onDelete function
myFunction('old_data')

i.e., why does the first invoke a create while the second one a delete? Is there a null param missing maybe?

---

Also, is the following a typo or, if not, what does the auth object need to contain?

# mock auth, for example: to mock unauthenticated user
myFunction('data', {auth, {admin: false}}

[laurenzlong]

If it works on firebase serve, it should work on the functions shell as well since the underlying emulation mechanism are the same. We haven't done a full survey of all IDEs to know that this would work properly on all of them, so there may be kinks. (But it is on our roadmap to address this and make sure breakpoints always work)

[laurenzlong]

Not yet. But coming soon.

[laurenzlong]

Thanks for the follow up! We're prototyping it right now. Would love everyone's feedback on how you would prefer to invoke the emulated database function. Would you rather it be listening to your production database? A test project's datebase? Invokable through a terminal command?

[ryanm2000]

+1 for through a terminal command

Thanks for the great work @laurenzlong

[mismith]

That terminal command looks slick; I for one would definitely love to use it! Especially if the -d flag is optional, and defaults to what is currently in the database at that location (i.e. it self-'refreshes' with existing data).

[mismith]

@ahaverty Re: manual/invoked refresh: one of my functions sends an email whenever a key is updated (e.g. forms/<id>/submitAt), so calling firebase functions:invoke:sendEmail /forms/<id>/submitAt with no -d flag would simply 're-submit' the form with the existing submitAt datestring, thereby triggering my email and allowing me to debug that logic.

[fej-snikduj]

@laurenzlong Thanks for all the updates. I completely agree with @jaufgang that a locally deployed database function should listen to a real database. I know that'll take a bit more time than the terminal commands, but I'm wondering if it's a priority and being worked on? If it's longer term, how long do you estimate until a solution is ready? Are we talking weeks or months?

[msjaber]

Thanks, @laurenzlong. Any updates for supporting database events?

[laurenzlong]

@mismith Oh yes, that is indeed a typo, sorry! It is a nested object: {auth: {admin: false}}

[ahaverty]

@laurenzlong Is it possible to locally debug/breakpoint functions started by experimental:functions:shell yet? (vscode)

It looks like it's possible for https functions using serve, but I haven't found anything about breakpointing realtime database functions, other than a few comments on this issue 🙌.
~ I found this on google cloud functions debugging too. 🤔

[laurenzlong]

Thanks for clarifying, we don't yet support local testing of HTTP callable functions, but that's coming soon.

[derrickrc]

@laurenzlong I realize this issue is years old, but I'm coming from GCF (and Functions Frameworks for local development) and now am writing my first Firebase Function as I want to listen for background events from my RTDB. In 2021 is there no way to listen for remote / production database events when using the emulator? Thank you.

[mInzamamMalik]

Dear @laurenzlong I was not talking about unit testing
I want to run firebase functions locally, just like we can run aws lambda function locally,
either it is possible right now?
if not, are you guise planning to give this facility in near future?

Thank you,
Inzamam Malik.

[atomicleopard]

How this emulator support the firebase specific parts of GCF (i.e. ref().onWrite() ? I haven't found any specific docs for this? Dev cycle while deploying up to the cloud is sort of excruciating, it would be great to be able to do this locally.

[stephenhandley]

hi @laurenzlong any updates on firebase integration in the emulator?

[stephenhandley]

@laurenzlong awesome thanks!

[mismith]

@laurenzlong I'm using a separate 'root' within the same project (e.g. /production/data/1 vs. /development/data/1) simply because this was the easiest to get up and running. It's definitely a tough problem to solve, and as you can already see, tons of different perspectives and alternatives to support. Maybe a terminal utility would make sense, something that kind of 'links' the path(s) being watched to the functions being called? e.g. maybe the dev can map which project/path/etc goes to which functions via some config file (with sensible/configless defaults, of course)? Even hardcoded JSON/dummy-data would potentially be better than nothing, at this point.

Btw, thanks for being so responsive and inclusive on this! 🤘 firebasers 🤘

EDIT: is this along the lines of what you mean by the terminal command?: firebase serve --only functions; firebase trigger-function "foo"? or maybe something more generic with a custom path like firebase simulate-write --path="/foo/path/to/bar/with/function/watcher" --data-file="baz.json"?

[jaufgang]

Also, @laurenzlong, since this issue is actively being discussed and worked on, perhaps it should be re-set to open.... you closed it on March 13.

[ahaverty]

I like the sound of that @jaufgang , perhaps cloud disabling true by default though as keeping cloud functions running wouldn't be a use case for us right now and may cause confusion with novice users in future (but we could live with either! 👌)

[ahaverty]

@atomicleopard Firebase uses pub/sub for events, I've had functions fail before and still handle 100% of events, after coming back online after 1 hour. Although, I do agree with you, it could very easily get messy if one of the team left their local cloud-disabling test on and went out for an hour lunch break!
Pushing local data would be a great start, and would help testing the basics/syntax errors (currently 90% of our errors [we still haven't converted to typescript 😢]). I imagine end to end testing would be more suitable for an online deploy anyway! Interested to hear what @laurenzlong thinks is even possible here

[ahaverty]

@laurenzlong sounds good 👏
My vote is for simulating local events with -d, but default working with the 'production' database.

We're working on entirely separate firebase projects for multiple different projects (dev + our main production), so the test target wouldn't be as beneficial, but would also be no harm to have the option either.

..Unless I'm misunderstanding what 'test project' means? Does that mean another firebase project to target, or is it a local version of the Firebase database? If it's just a target option, wouldn't it make more sense for people to set up their target configurations properly and avoid the chance of writing to their production db's?
If a local Firebase database is possible, then yes! We could use that for a lot of different things (CI, simpler automated testing, reduce the number of dev projects)

[jaufgang]

Would it be possible to add the ability to allow/disallow local testing against a real database using a new rule added to the rules.json? Either that or some other console setting? Production databases could be configured to disallow it, while dev/test databases could allow it.

Also, WRT @atomicleopard's concern about isolating problems if/when they occur, perhaps that could be mitigated by adding messages to the functions log whenever a local function attatches/detaches a listener.

[ahaverty]

@mismith Interested to hear how/why the self-refresh would be beneficial for you? (assuming you mean take all the data at the defined ref and simulate pub/subs)
For us, it'd mean thousands of event writes would go off (Not currently removing our event writes as remove events currently retrigger the function and cost us another invocation. It might be useful once firebase implements specific event type functions, but even then you'd need to handle preventing triggers with the 'online' function, right?

Have you considered piping a cat command into -d ? @laurenzlong Could piping in file contents into -d be considered as a use case for simpler testing/more complicated event writes?
Is it as simple as something like:
cat makeUpperCaseTest_valid.json > firebase functions:invoke:makeUpperCase /input/path -d
or could an explicit flag for file input be simpler?
firebase functions:invoke:makeUpperCase /input/path -f makeUpperCaseTest_valid.json

..Most of my testing involves creating it once via one of our client apps, exporting to json, and reimporting via the console. Not too bad right now, but this would be nice 👌

..Actually. @laurenzlong the ability to create a native firebase .push() on the path would be amazing too!

[atomicleopard]

A few wild ideas: Local testing: in addition to the proposed, it would be great to invoke the function while watching a file so I can edit it and see it trigger like it might from the real DB. Live system: i'd like to see something like the ability to breakpoint the remote code, possibly even edit it while deployed (so a mini IDE basically). That way, my full dev cycle could be: * Write and debug locally using simple data or data I export from firebase db * Unit test (assert output/changes) * Integration test (validate remote systems trigger correctly, such as sending an email etc - I'm sure more advanced tooling is required here but its a great aspiration) * Deploy to test environment, ensure correct functioning * If not, breakpoint and debug, make 'live' changes to validate fixes * Apply locally and redeploy/repeat * Deploy to production * 🙌 + 🍻 all around

…

On 13 June 2017 at 04:06, Lauren Long ***@***.***> wrote: Thanks for the thorough feedback everyone! To answer your questions, a terminal command would probably look something like: firebase serve firebase functions:invoke:makeUpperCase /input/path -d '{"word":"foo"}' Thanks for sharing the perspective that "intercepting" live traffic with a test function could be disastrous for teams. For testing against a real database, I imagine there being a test project that is the default project for "firebase serve". So both hosting and functions would use the test project. This way, you can test interactions like a) user fills out a form and that data gets stored to the database b) a function is invoked and transforms that input c) function writes back to the database d) the new database value affects the web page UI somehow. The test project can be set using "firebase use --add", or maybe it can be a part of the "firebase init" flow. If a test project is not set, then "firebase serve" would use the production project but give a warning?? I'm not sure about what's best in that case. Supporting the terminal command is achievable in the short term, but supporting real database triggers would take more time. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQdDnam6xCQKEsYPY3NY-7f1LpmHAJMks5sDX43gaJpZM4MaKU0> .

[mismith]

@laurenzlong I doubt the dummy data would be that useful (for me) since its form wouldn't be predictable, so I'd probably just end up using/generating my own and passing it via the JSON file or as an inline object.

The second half of your last paragraph is kind of worrisome though: if 'writing' the same exact data wouldn't trigger an onWrite event, then wouldn't that mean that passing the same JSON file wouldn't either? Or event the same -d data? In other words, I would expect that it should (in all these cases), so why not do the same with 'default path' data—albeit with the (presumably minor) performance hit of a database read beforehand?

All this said, I'd still be super happy just to see any of this being made usable soon; I'm being picky when I should really be grateful!

[mismith]

@laurenzlong Re #1, of course! My oversight.

For #2, if it should indeed be (the ES2015 object literal property value shorthand syntax-using) {auth, {admin: false}} (as opposed to {auth: {admin: false}}), how should I instantiate or mock the auth variable?

[medisoft]

@laurenzlong I would love to allow by CLI parameter to choose production/testing database.

[alexbjorlig]

Hi @laurenzlong - I have looked at the links and stackoverflow and can't get firebase functions debugging in VS code to work. Do you have any new links to get more information/help? It feels like a huge step backwards to debug with console.log 😅

[laurenzlong]

@dauledk We currently don't officially support IDE debugging. This is on our radar, but we haven't had a chance to make it work.

[EECOLOR]

@laurenzlong Would there be a way to hook up the emulator for functions to firebase-server?

[KendoJaaa]

hi @laurenzlong I have a noob question. I wrote a firebase function which are triggered by my application directly not by http request. Can we test it by running function locally? i managed to run function locally but I couldn't find a way to trigger the function. Can you guide me a bit? thx in advance.

[laurenzlong]

@KendoJaaa Please see https://firebase.google.com/docs/functions/local-emulator

[KendoJaaa]

@laurenzlong thx for your response. actually I've read that page, but couldn't make it.

I want elaborate more so you understand me better
this is what I did in my app. (calling function directly from the app) see the link below
https://firebase.google.com/docs/functions/callable

and I run the function locally using Cloud Functions shell according to this link
https://firebase.google.com/docs/functions/local-emulator
I assume that calling function directly from the app is actually HTTP request behind the scene.
The point is I don't know how the shell command looks like in order to trigger the function. I couldn't find the example in the page

[KendoJaaa]

Thx ;)

[Yuripetusko]

Hi can I hijack this thread to ask if it is possible to pass authentication to https callabale when testing from firebase shell?

if I do myCallableFunction.post().json({"data": {"test": "test"}}) My context.auth is empty, so I tried myCallableFunction.post('', {"auth": {"uid": "abc"}}).json({"data": {"test": "test"}}) but got Error: no auth mechanism defined. Is there a way to pass auth when testing callables locally?