Support Photos In Place on Hard Drive about photoprism

[lastzero]

@streamnsight Our personal use case is that our photos are distributed across several computers and smartphones. There is no single source of truth. Some of the files might have useful names, others not. In other words: chaos and LOTS of duplicates. Basically we want to throw everything in and let the software do the magic.

We see that this conflicts with other users who might have a single source of truth and want to keep it. The only way I see right now is to add a flag for this use case, so that PhotoPrism knows files are managed manually or by some other tool. Upload/delete would be disabled in that case.

PS: The indexer will find new files and also detect deleted ones. Already implemented. You still need to set a filename / directory for uploads etc. Disabling uploads is the only viable solution, if PhotoPrism is not allowed to choose a file/directory name. Unless you want to add a dialog and ask the user every time.

[lastzero]

That seems easy, but how can you be sure there are no duplicates in your existing collection? You could for example simply move them to Originals and start indexing, but as soon as there are duplicates, PhotoPrism (as it is now) would start wondering which file is the original, if it is found more than once. Also files that are not supported, like PNGs or PDFs, would become "hidden" as PhotoPrism doesn't display them.

You could also move them to Import (or point Import to your existing collection) and import them without needing extra space (files are moved to Originals, not copied), if space is the only problem.

There might be valuable information in existing file names, those will be stored as original_filename in the database in the future (not implemented yet).

I also had the idea of adding a "duplicates" table so that duplicates in Originals can be detected.

[jamisonhyatt]

So files on the PHOTOPRISM_IMPORT_PATH are moved to PHOTOPRISM_ORIGINALS_PATH, and PHOTOPRISM_EXPORT_PATH will not be used unless the "export" is explicitly requested in the UI?

Wouldn't setting a flag which uses a hard link instead of an explicit move be a decent compromise here?

[jmoyerman]

If the files are moved, then I misunderstood the import process slightly, however I have a large amount of valuable information in how my files are currently stored. I use:
YYYY/YYYYMM_Job/DATE_Job_#.nef.

Moving the files would also break any compatibility with other software used, and while it would be great to have one platform that does it all, it will be a while before this app has the power of something like Dark table or Lightroom for raw processing.

I'm not sure how you are defining duplicate.. is it merely two files with the name photo1.jpg? If so those are potentially unique files. If you mean two files of exact content with different names, then I don't know.

I don't see much problem in PDF files etc not being served by photoprism, as theyd also remain in place

[lastzero]

Duplicates: Files with the same sha1 hash.

Note that this isn't an image editing software. You can and should continue using Lightroom, Darktable and/or Photoshop.

Point is: If PhotoPrism is not responsible for file naming, you either need a different software or do it manually. Doing it manually is not a great option, at least if you have more than 10 photos. If you're using a different software for photo management, what do you need this software for? Trying to understand the use case... I see the value of preserving/indexing information that is/was in file and directory names, so that it doesn't get lost and you can search for it. But file naming should not be completely random/manual? Let me know if I'm wrong.

[lastzero]

So let's say you decide to start using this app while keeping your existing file names and directories.

Now you want to add / upload new files. What should happen?

1. Do you want to manually specify a file and/or directory name for new photos?
2. Do you prefer to add / upload files using a different software and basically use this app in a read-only mode?
3. Do you want the app to "automatically" detect your current naming strategy and name new files "somehow" similar?
4. Do you want to configure a custom naming strategy that matches your current one as good as possible? What if some of your existing files don't comply with it? Shall they be renamed or do you want a warning or just ignore it?

[jmoyerman]

I guess mostly a hybrid of 1/2. My current workflow is below:

1. Take Pictures
2. Import with Rapid Photo Downloader to ~/Pictures/ (This automates the name/numbering)
3. Tag/Process RAW with Darktable
4. Export and Share via social media or otherwise
   <- I see photoprism fitting in here
5. Open my laptop and browse through file viewer or darktable when showing friends/family pictures.

Having something that could pick up the gallery at step 4 would be ideal, because then I can still leverage my tags within Darktable, and existing import structure. If I were to add new photos, I could trigger photoprism to rescan. I am potentially mis-understanding the featureset of photoprism, and that it is intended to replace steps 2 and 4 above.

[lastzero]

Yes, actually you can replace steps 2, 3, 4 and 5 (if there are no XMP files, it will use default settings for processing RAWs). Since it's a server-based application, it should not depend on local file names and specific desktop applications to do it's work. It's still possible to share the server file system via Docker or NFS and edit/delete/add photos manually if you stick to the naming scheme. So it's a mix, you still have freedom. More freedom than if you upload your photos to some remote cloud service.

If you want to keep on using your local names and desktop applications for all of the steps, you are probably looking for a "simple" photo gallery - maybe a little more powerful and with nicer UI than existing ones. As soon as you start using PhotoPrism for processing RAWs, uploading and deleting, you can not depend on any name/numbering your other tools previously did for you unless you put a lot of effort in configuration and testing.

Does that make sense to you?

I can imagine to implement some kind of "read-only" mode for the use with existing desktop photo management apps that do all of the file management for you. You would not be able to delete files, upload new files or download photos from social media. Basically the functionality would be reduced to a web gallery that also supports sharing and exporting. Is this what you are looking for?

[lastzero]

To avoid confusion: PhotoPrism uses Darktable in the background to create JPEGs from RAWs, otherwise you can not properly display thumbnails (or at least you would have to create them on the fly which is very slow). If a RAW file comes with a processed JPEG version, that file will be used for thumbnails/browsing and the RAW will not be processed again using Darktable. PhotoPrism is not a photo editing/processing application, but it knows how to use Darktable for you.

[lastzero]

@jamisonhyatt Not a good idea to set a hard (or any) link from import to originals. Otherwise you can use every original file name only once. How do different cameras know they need to use unique file names that don't conflict? Or humans that copy random photos to import? A link would overwrite existing files in the originals directory if they previously had the same name like IMG_1234.JPG.

[trebonius0]

My two cents on that (I'm the creator of photato, another picture gallery)

In my particular case, I like having my pictures on the disk the way I want.
If I want to add files to the gallery, I add them to my pictures folder and I want this addition to be automatically detected. Same for deletions.

I do prefer it this way than having to import them in the UI. Also it allows me to have more complex scenarios like having a syncthing : whenever I take a picture, synced to the pictures folder and consequently into the gallery

Also, I really don't care about the duplicates handling. I do prefer to keep my folders as is

[lastzero]

@trebonius0 I guess that's the difference between a gallery and photo management. A directory is not an album or a gallery for us.

As mentioned above, you can also copy photos directly to Originals, but then you should use the software in a read-only mode. Degraded to a gallery. Also you don't need to upload, you can simply copy files to either Import or Originals. This software uses the file system for storage. Nothing keeps you from syncing.

[lastzero]

Linked this issue in our FAQs: https://github.com/photoprism/photoprism/wiki/FAQ

Nobody should be forced to copy/move anything. We love freedom. But it has consequences for
the workflow and what PhotoPrism can do. Let's see if we find a solution that works for everyone.

[streamnsight]

@lastzero
Michael, Emmanuel here.

I think you are way too focused on file names and duplicates.
The only thing you really need in Photoprism, is the thumbnails, if it's to manage your pictures, you're not likely to often open up the full size image, and if you do, fetch it from original local/network location.

So, you scan the files and generate thumbnails. I suggested using MD5 or SHA1 for the thumbnail filename. So they will be unique, and if there are duplicate identical images in various folders, they will have a unique thumbnail, unique hash, but different path.
How do you manage that? Simple: index each image (including duplicates) with the file location, and the hash value. Then when you sort/search/display images you query by unique thumbnail and you can eventually aggregate by filepath to have the location of the multiple files attached to the thumbnail.
Then every further processing you do on an image would be based on the hash, so you don't duplicate computations.

As far as adding / deleting pictures:
Most media stations simply have a daemon scanner that scans for change in the folder structure. There are efficient ways to detect changes, and force a rescan of the folder that changed.

[lastzero]

See also:

• https://github.com/photoprism/photoprism/wiki/Concerns
• https://github.com/photoprism/photoprism/wiki/Importing