photoprism / photoprism Goto Github PK

For people with already organized (and massive) photo collections, it would be really nice to be able to point to our existing photo organization and not have to have photoprism copy/duplicate all of my data.

Not sure if thumbnails should be pre-rendered or rendered on demand. Depends on the final performance. Need to see how the user experience is.

Maybe using https://seregpie.github.io/VueWordCloud/

Do we want this? What is a good technical solution?

Currently photoprism attempts to create the directory, this fails if the folder is a symlink and thus refuses to start.

Instead of providing a huge search form with different fields, PhotoPrism should perform search based on a string that can also easily be bookmarked/remembered.

Example: https://github.com/search/advanced

As a User, I want to have the option to use Google Maps instead of OpenStreetMap, so I have the best experience.

Right now, only OpenStreetMap is supported / used (just the reverse lookup API, not the UI).

Acceptance Criteria:

• Add helpful information to our Developer Guide, Geocoding is just a stub right now
• Change DB structure: We use the Place IDs of OpenStreetMap as location ID right now
• Caching should also be improved to reduce load on API and improve indexing performance
• Implement config option / parameters to configure OpenStreetMap and Google Maps (API key?)
• Check license issues, maybe we can get a special license from Google as we are Open Source?

As a user, I want to create albums, so that I can put related photos together.

Albums should also have a sort order & sort by that can be configured (usually date taken). Photos can be added using the menu in photo search.

Acceptance Criteria:

• User MUST be able to view his albums
• User MUST be able to create albums
• User MUST be able to add photos to an album via the "photos" page (One photo can be part of multiple albums)
• User MUST not be able to add the same photo to an album twice
• User MUST be able to remove photos from an album via the "photos" page and from within the album
• User MUST be able to define the order of photos (either by setting default filters like date or maybe later via drag and drop)

I noticed the Dockerfile depends on apt to work. As someone who uses a distro that does not us apt, it is taking me quite a while to get a working version of photoprism to build. It might be good if multiple dockerfiles are created to support multiple package managers. I'd be willing to help with an Arch Linux version, I just need to know how the Dockerfiles should be organized within the project directory.

Potential fields that can be used:

ImageID, ImageNumber, RawDataUniqueID, OriginalRawFileName

See http://www.exiv2.org/tags.html

Our Wiki should contain the following information:

• Bookmarks: Related projects/products, developer tutorials, design resources, etc
• Developer Guide
• Everything about machine learning / TensorFlow
• Glossary
• Infrastructure
• Use Cases

I don't know the scope of the goals for photoprism, but a instead of using MySQL (or #47 PostgreSQL), why not use boltdb, goleveldb, or one of the many other embeddable stores for Go?

The models look relatively simple making this ideal.

https://github.com/blevesearch/bleve (based on boltdb) would be much easier/faster to integrate than #6 Solr for quick searches.

For Android and iOS. Interface should be similar to the Web UI (Material Design).

Main reason why this is important: Automatic upload / sync of photos stored on the devices, especially if PhotoPrism is hosted at home (devices need to be in the same Wifi network).

Maybe use NativeScript? https://www.nativescript.org/

As a user, I want complete Exif support so that I can search and view all metadata.

Please compare the Exif tags we already support - see exif.go and related test - with the list of standard Exif tags and what others index / display, see screenshots on the Exif page in our Wiki.

For example, flash is not indexed right now. ISO and color space as well. For date and time, it would be awesome to know the time zone to display it properly. And then of course test it with real files, do some code-clean up and add documentation to our Wiki, e.g. list tags we support or add info why we can't support it.

Acceptance Criteria:

• Exif tags we already support MUST be compared with standard tags / other apps
• Support for additional tags SHOULD be added to exif.go
• A test for each supported tag MUST be added (find additional example images containing those tags, if needed)
• Documentation MUST be added to our Wiki (list of tags: supported? yes/no)
• You MAY add fields to the file or photo model for storing additional metadata (indexing is not part of this ticket, but you are invited to improve it along the way)

As a user, I want to share links to albums so that friends can view my photos without giving away my admin password.

In a later release, we might offer hosting on our servers for selected (shared) photos / albums with auto expiration, so that friends don't need direct access to a user's PhotoPrism server instance at home.

Shall we implement photoprism init to create a conf file and maybe some directories? Would it be useful? Most installations are probably using Docker and don't need it.

Just saw this in the docs of another Go project:

Inside an empty directory run:
$ bebop init
This will generate an initial configuration file "bebop.conf" inside the current dir. Edit the configuration file to set the server listen address, the base url, the database and file storage parameters, OAuth credentials, etc.

See https://github.com/disintegration/bebop

Need to check if licensing is a problem

Hi

I was wondering why downloading photoprism container took so much time...

May I suggest a multistage build ? Here's the doc https://docs.docker.com/develop/develop-images/multistage-build/

Basically, if you can make a standalone photoprism binary (aka no need for local third party)

FROM golang:1.7.3
# everything for build, until standalone photoprism bin is in say /bin/photoprism

FROM alpine:latest  
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /bin/photoprism .
ENTRYPOINT ["./photoprism"]
CMD ["start"]  

It seems PNG pictures are not supported yet?

I tried to import PNG pictures (didn't work), JPG/JPEG OK

If PNG are not supported yet, it could be worth adding it to the setup documentation ?

https://docs.photoprism.org/en/latest/setup/

Great project btw :)

Will become relevant as soon as proper logging is implemented.

For people with already organized (and massive) photo collections, it would be really nice to be able to point to our existing photo organization and not have to have photoprism copy/duplicate all of my data.

I initially opened #40 with my work account, so I closed it and reopened it with my personal. Sorry for the churn.

1. For the v-snackbar element no timeout is set, this causes the element to use the default timeout from vuetify instead of the ones set in the addMessage() function.
2. The alert.css is not in use

Acceptance Criteria:

• Set the timeout attribute of the v-snackbar element to 0
• Remove the alert.css

It would be nice to give out little perks for contributing to this project like Amazon vouchers. Or at least virtual badges based on points collected. Seems like an interesting experiment.

Companies such as https://www.cewe.de/ provide an API to directly order prints and photo books.

Hi!

Great project! The Go part needs a bit of polishing I begun doing.

Here are some notes that I started hammering on:

• Remove the convoluted way configuration is created and replace it with proper yaml marshalling (#50)
• Add documentation to exported values (#50)
• Restrict exported fields if they aren't really used, there is no need to export them ( there are lot of exported fields which don't really need to be exported. ). (#50)
• See if it's really necessary to share all the structures with a pointer making them escape to heap. If they aren't modified or required to be modified it's enough to pass them by value and making them stay on stack.
• Make the database injectable ( this is quite likely the biggest of changes and probably needs to be tracked in a separate ticket. ).
• Extract services and put them into their own package so other packages can access them if they are needed.
• Write some more tests
• Reduce complexity by extracting small behaviour and eventually refactor them to use a different approach. Once smaller components are extracted it will be much easier to see what can be thrown out or made simpler.
• Remove . imports
• Indexing can take a lot of time for many pictures and Tensorflow adds to that. This should be extracted into a semaphore controlled parallel process. Since they aren't connected it should be possible to make it parallel rather than concurrent

That's all for now as a quick glance. :) I'll extract stuff in here into different tickets to track them via different PRs if you don't mind of course.

Also, I started into the first three points together since they would cause a lot of code conflicts if done separately.

A fresh docker install (commit 6844c5f) crashed
when importing about 3000 photos.

Moving main jpg file "Camera/IMG_20170425_062158425.jpg" to "/Photos/Originals/2017/04/20170425_062158_2343698269B6.jpg"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x52fbd9]

goroutine 1 [running]:
github.com/RobCherry/vibrant.NewPaletteBuilder(0x0, 0x0, 0x0)
        /go/pkg/mod/github.com/!rob!cherry/[email protected]/palette.go:202 +0x29
github.com/photoprism/photoprism/internal/photoprism.(*MediaFile).GetColors(0xc0003d7b80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/colors.go:33 +0xcf
github.com/photoprism/photoprism/internal/photoprism.(*Indexer).IndexMediaFile(0xc000410da0, 0xc0003d7b80, 0x1, 0x1)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/indexer.go:92 +0x1e81
github.com/photoprism/photoprism/internal/photoprism.(*Indexer).IndexRelated(0xc000410da0, 0xc0003d7ae0, 0xc0003d7ae0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/indexer.go:231 +0x9b
github.com/photoprism/photoprism/internal/photoprism.(*Importer).ImportPhotosFromDirectory.func1(0xc000027920, 0x30, 0xecac40, 0xc00041f450, 0x0, 0x0, 0x1a, 0xc0007a76c0)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/importer.go:104 +0x869
path/filepath.walk(0xc000027920, 0x30, 0xecac40, 0xc00041f450, 0xc00070ca20, 0x0, 0x0)
        /usr/local/go/src/path/filepath/path.go:358 +0x41c
path/filepath.walk(0xc0004203e0, 0x15, 0xecac40, 0xc0003cf930, 0xc00070ca20, 0x0, 0x0)
        /usr/local/go/src/path/filepath/path.go:382 +0x2fe
path/filepath.walk(0xc000026047, 0xe, 0xecac40, 0xc0003cf380, 0xc00070ca20, 0x0, 0x30)
        /usr/local/go/src/path/filepath/path.go:382 +0x2fe
path/filepath.Walk(0xc000026047, 0xe, 0xc00070ca20, 0x30, 0x28)
        /usr/local/go/src/path/filepath/path.go:404 +0x105
github.com/photoprism/photoprism/internal/photoprism.(*Importer).ImportPhotosFromDirectory(0xc00070c9f0, 0xc000026047, 0xe)
        /go/src/github.com/photoprism/photoprism/internal/photoprism/importer.go:38 +0xb5
github.com/photoprism/photoprism/internal/commands.importAction(0xc00046e160, 0xeb9000, 0x33)
        /go/src/github.com/photoprism/photoprism/internal/commands/import.go:35 +0x307
github.com/urfave/cli.HandleAction(0xc87100, 0xdfb058, 0xc00046e160, 0xc0003fc100, 0x0)
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:490 +0xc8
github.com/urfave/cli.Command.Run(0xda5f8c, 0x6, 0x0, 0x0, 0x0, 0x0, 0x0, 0xdaca99, 0xe, 0x0, ...)
        /go/pkg/mod/github.com/urfave/[email protected]/command.go:210 +0x990
github.com/urfave/cli.(*App).Run(0xc0004544e0, 0xc00000c080, 0x2, 0x2, 0x0, 0x0)
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:255 +0x687
main.main()
        /go/src/github.com/photoprism/photoprism/cmd/photoprism/photoprism.go:29 +0x2d7

It should not be required to use the command-line interface to start importing. Users should also be able to upload photos instead of putting them in the import directory directly.

This is an epic, APIs see https://github.com/photoprism/photoprism/wiki/External-APIs - I'll keep this open as a place for discussion.

What APIs should be supported first?

Might be convenient for distributing the final app as a single file (not for development).

See https://docs.docker.com/compose/bundles/

Nice idea lads, I totally support it. Were youever wondering to switch to postgres ? For the deployment size I'm predicting this going to have, mysql might be a bit suboptimal choice :)

Details on possible implementation strategies can be found in this comment:

• #47 (comment)

Using Solr for search would be faster and more powerful... TBD

Is there an option to connect it with my nextcloud and the photos where are stored there?
That would be awesome.

Nextcloud has an gallery plugin but i think this is ways better.

This looks really promising for personal photos. There is also a general purpose personal storage system out there called Perkeep which I think could be a great storage backend for photoprism. Perkeep could be the "database" while photoprism would be the application server and interface. Just wondering if something like this would be possible or even desired for photoprism.

Shall we support files other than JPEG and RAW? Maybe move all videos to a separate video directory?

We focus on photos because when you remember something, your brain also uses images, not videos with sound, or not?

PhotoPrism is for personal use but some form of auth would be great to protect it from guests using the same network etc.

It should be possible to download multiple photos / albums as ZIP file (stored in the export directory).

This is the first ticket tracking the first three points of the Great Refactor epic #49.

Wiki URL: https://github.com/photoprism/photoprism/wiki/FAQ

Some users might want the software to automatically create albums e.g. like the iPhone does. Not sure if this is what we want. I would like to have more control over what the software does.

There are lot of functions who just print errors and never return them. That is not really nice. :P

If friends are sharing their feed with you, so that you don't have to visit social networks just to see their photos (also they are usually getting deleted after a while or when the user deletes the account).

As a user, I want to search for color names so that I can find matching photos.

The color indexing implemented right now is just a proof-of-concept. It finds too many different colors (with complicated names). We need to reimplement indexing using another algorithm like k-means and use a reduced list of colors: red, pink, purple, indigo, blue, cyan, teal, green, lime, yellow, amber, orange, brown, grey, white and black.

Acceptance Criteria:

• You SHOULD use the k-means clustering algorithm (unless you find a better method)
• You MAY use any open-source Go libraries as long as the license is compatible with GPL
• The following dominant colors MUST be detected: red, pink, purple, indigo, blue, cyan, teal, green, lime, yellow, amber, orange, brown, grey, white and black - color codes see Material Design palette
• Only colors.go and related tests MUST be changed (you don't need to touch the indexer etc)
• You MAY add "light" or "dark" as color name prefix, to indicate if it's light or dark

Add a doc.go file to describe the individual packages in internal/. Make sure go docs are properly rendered / displayed. Add a doc target to Makefile.

When docs are working, add badge to README.md.

See:

• https://godoc.org/golang.org/x/tools/cmd/godoc
• https://blog.golang.org/godoc-documenting-go-code
• https://godoc.org/

As a user, I want to be sure another import is not already running when I start it.

When an import is running it should not be possible to start a second import. A lock file would be a solution but it's probably better to use the database as we want to log the import results there anyways.

Might be interesting to try Prometheus for displaying stats from the logs.

As a user, I want to tag familiar human faces so that I can find all photos of these persons.

PhotoPrism must be able to detect and recognize faces. There are existing libraries for this, so we should try those first. See Face Recognition in our Wiki.

Here is a complete howto: https://hackernoon.com/face-recognition-with-go-676a555b8a7e

Acceptance Criteria:

• You SHOULD use go-face (unless you find a better library)
• A unit test demonstrating face recognition with real images and faces MUST be delivered
• A user interface MAY be developed and gives extra karma
• You MUST update our Docker images in /docker as needed (go-face requires dlib as external dependency); let us know when we shall push new images to Docker Hub
• Documentation MUST be updated so that other developers know everything to continue working on this task

Perceptual hashing is already implemented, all we need is a user interface and maybe an easy way to merge multiple files into one photo.

Please add support for Video to be stored / played back.

In order to be able to copy the archive or restore database contents after a problem, it would be very convenient to store a copy of the metadata in the filesystem right next to the photos they belong to.

Contents need to be time stamped and fingerprinted in order to avoid sync/merge issues.

YAML, JSON or XML? Are there existing standard formats?

This way we could also store the original filename during import (simply write it to this file).

zfhj