etimo / photo-garden Goto Github PK

• Create users
• Connect users to providers
• Get user from provider id

• Integrate frontend with photos-api.
• Consume queue and store in db for photos-api.

Should be able to filter by provider in the ui

Should read from Elastic Search

I and @teozkr discussed this a bit today, would be nice to bring this up for a larger discussion on friday.

Should we continue developing micro/nanoservices or should we aim for "domain"-grouped larger services? What is the difficulty/benefit of the different strategies?

What will need scaling? E.g. downloading of images vs normalizing data structures?

How to scale? Or simply, do we think about scaling atm?
Do we aim to scale services by number of workers or by hardware? Perhaps different for different services? How about lambda or similar instead?

How can we manage a fast and clean development environment without current problems (compilation time, dependencies, overhead in docker images)?

How small should services be?
At the current level the number of services will explode in the future. This gives a large overhead in docker images, deployment handling, etc etc

Just writing down some of the thoughts we discussed. Feel free to add more to this issue before friday

That we can use for internal storage, transformations etc. I.e. info from e.g. Google Drive should be transformed to this generic format.

Check out launch darkly etc. but prefer open source. Prefer not to build custom solution...

User has a personal account

We are going to need a queue (for example to process metadata from providers, analyze images and so on, general persistent async operations).

Rabbitmq or some other? I know some of us have worked with rabbitmq. This could also be abstracted if we decide later on to change it.

Current problem: When using pub-sub in rabbitmq, the messages are only delivered if the subscriber is currently connected.

What we need: When using pub-sub and the subscriber is not connect (perhaps it has crashed) then it should receive the messages when it goes online. The subscriber needs to get all messages since it started subscribing (not since beginning of time).

Some images and thumbnails (e.g. from google drive) are short-lived. We need to download them and store them. This will also apply to a lot of other providers (e.g. dropbox that doesn't even provide thumbnails)

Some thoughts from the subway: Should we have one unified API Gateway for the project, and as such letting this API Gayeway be the only API exposed to the internet?

I'm sure you've worked with API Gateways before, and understand their purpose. This will also be relevant in how we set up things like DNS later, so we should determine whether we want one early.

If we decide to create one, we need to:

• Set up a milestone for the API Gateway
• Set up issue for various subtasks related to the API Gateway

Jwt is self-contained and thus we can make requests stateless.

First time a user logs in we start downloading all their data. Since we issue an offline token we are able to listen for future changes in Google drive for the user so we can process new files without the user having to login again. This means that the next time they arrive at Photo garden all their stuff will already be present.

https://developers.google.com/drive/v3/reference/changes/watch

Problems:

• Different package.json with overlapping dependencies
• Tooling that is required by more than one service in dev (e.g. nodemon, mocha etc)
• Dependencies between packages and services (e.g. common code for queue handling and feature flags both require certain dependencies in package.json)

Proposed solution:

• yarn workspaces (https://yarnpkg.com/en/docs/workspaces)

Possible cons:

• Only allows npm and node projects in workspace. Perhaps other projects can be added but they will not be handled by yarn. Will this be an issue or will we continue developing in node/js?

SSL certificate

Arbitrary place definition; e.g. city, geo rect, country. Whatever fits and is easy to implement