rfgamaral / docker-gphotos-uploader Goto Github PK
View Code? Open in Web Editor NEW🐳 Mass upload media folders to your Google Photos account with this Docker image.
License: MIT License
🐳 Mass upload media folders to your Google Photos account with this Docker image.
License: MIT License
I am running this on raspberry pi, In the step to run oauth.sh start, i got the following error:
docker exec -it gphotos-uploader oauth.sh start
Unable to read configuration file (~/.config/gphotos-uploader-cli/config.hjson).
Please review your configuration or execute 'gphotos-uploader-cli init' to create a new one.
then i changed volume '/config' to '/root/.config/gphotos-uploader-cli/' in container to put in my config file, i still get the same error..
when i use 'docker exec -it gphotos-uploader /bin/sh' to get in, i can 'cat ~/.config/gphotos-uploader-cli/config.hjson' and get correct content, but if i run 'gphotos-uploader-cli' i got this error.
gphotos-uploader-cli seems got stuck finding the absolute path of '~/.config/gphotos-uploader-cli/config.hjson'?
'gphotos-uploader-cli init' couldn't create default config file either
gphotos-uploader-cli init
Failed to create the init config file: failed to get absolute path: ~/.config/gphotos-uploader-cli/config.hjson
I don't know if this issue is from Google Photos or from the uploader.
But I have tons os photos here, all in each folder.
Configured gphotos to create an album with the folder name.
I am getting someduplicated albuns, with the same name, and with the photos from my folder splited in each album. The simtom is that it tries to search for the album and does not find, so it created another.
currently running: rfgamaral/gphotos-uploader:amd64-latest
I follow the steps outlined for CLI authentication
Log information
2019/07/04 20:22:14 You will now be taken to your browser for authentication or open the url below in a browser.
2019/07/04 20:22:14 https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=<snipped>
2019/07/04 20:22:14 If you are opening the url manually on a different machine you will need to curl the result url on this machine manually.
2019/07/04 20:22:15 Failed to open browser, you MUST do the manual process.
2019/07/04 20:22:16 Authentication will be cancelled in 120 seconds
2019/07/04 20:22:56 Shutting down server...
Server gracefully stopped
2019/07/04 20:22:56 stored token for user: [email protected]
2019/07/04 20:22:56 Entering Directory: /photos
2019/07/04 20:22:58 using album: photos / Id: <snipped>
2019/07/04 20:22:58 Uploading den_20190703-071154.jpg
2019/07/04 20:22:58 Uploading den_20190703-071156.jpg
2019/07/04 20:22:58 Uploading den_20190703-071504.jpg
<Snipped upload activity>
[prompt]# docker exec -it gphotos-uploader oauth.sh start
2019/07/04 21:22:15 No Dbus support
2019/07/04 21:22:15 Saving credentials in Leveldb
2019/07/04 21:22:15 Entering Directory: /photos
2019/07/04 21:22:17 using album: photos / Id: <snipped>
2019/07/04 21:22:17 all uploads done
2019/07/04 21:22:17 all deletions done
[prompt]# docker exec -it gphotos-uploader oauth.sh start
2019/07/04 21:23:24 No Dbus support
2019/07/04 21:23:24 Saving credentials in Leveldb
2019/07/04 21:23:24 Need to log login into account [email protected]
2019/07/04 21:23:25 You will now be taken to your browser for authentication or open the url below in a browser.
2019/07/04 21:23:25 https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=<snipped>
2019/07/04 21:23:25 If you are opening the url manually on a different machine you will need to curl the result url on this machine manually.
2019/07/04 21:23:26 Failed to open browser, you MUST do the manual process.
2019/07/04 21:23:26 Authentication will be cancelled in 120 seconds```
While this uploads successfully, it does not put the images in folders.
I've verified that the OAUTH scope includes both "See, upload, and organize items in your Google Photos library ", "Add to your Google Photos library ", and "Manage and add to shared albums on your behalf" (among others, but those seem like the most important ones to this issue)
There do not appear to be any errors/warnings in the logs. Output of a run looks clean, but doesn't appear to mention anything about the album. It just looks like:
[info] Found item to upload: /photos/images.large/2022-08-16_18-24-57 picname.png
... (repeat previous lots more times, then repeat for other project)
[info] 492 files pending to be uploaded in folder '/photos/images.large'.
[info] 893 processed files: 893 successfully, 0 with errors
My configuration is not very complicated.
{
SecretsBackendType: auto,
APIAppCredentials: {
ClientID: "<deleted>",
ClientSecret: "<deleted>",
}
jobs: [
{
account: [email protected]
sourceFolder: /photos/images.medium
makeAlbums: {
enabled: true
use: folderName
}
deleteAfterUpload: false
uploadVideos: true
includePatterns: []
excludePatterns: []
},
{
account: [email protected]
sourceFolder: /photos/images.large
makeAlbums: {
enabled: true
use: folderName
}
deleteAfterUpload: false
uploadVideos: true
includePatterns: []
excludePatterns: []
}
]
}
I run this in Docker in my Synology. It used to run without stopping, which was nice, as I prefer that it stay running so that I can just dump photos onto my NAS and have them show up on Google Photos. I upgraded a while ago for the wildcard include/exclude and login fixes, but since then I’ve been getting the message:
Docker container gphotos-uploader stopped unexpectedly
And indeed, I noticed when watching the log during uploading is that what seems to be happening after it completes a run is:
Upload completed: file=/photos/xxxxx
File uploaded successfully: file=/photos/xxxxx
2020/02/10 10:48:52 Removing file's upload URL from DB: /photos/xxxxx
2020/02/10 10:48:52 all uploads done
2020/02/10 10:48:52 all deletions done
[cmd] run exited 0
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.
It seems to shut itself down after every successful run.
Can it not do that? Thanks.
so glad you created this docker!
While it runs great with invoking the command docker exec -it gphotos-uploader run from host - I cannot get the container to run the command from cron within the container. It is running gphotos-uploader-cli command from cron in the container - which does not upload photos.
Any clues on how to fix this?
Thanks
Everything has been working fine for a month or two.. suddenly this...
2019/06/28 19:21:32 Uploading IMG_0536.JPG
2019/06/28 19:21:36 failed uploading image
--- at /go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:33 (StartFileUploadWorker.func1) ---
Caused by: failed uploading image
--- at /go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:53 (FileUpload.upload) ---
Caused by: status message should be OK, found: Success
--- at /go/src/github.com/nmrshll/google-photos-api-client-go/lib-gphotos/client.go:147 (Client.UploadFile)
created by
sudo docker create \
--name=gphotos-uploader \
-e GPU_SCHEDULE="0 */8 * * *" \
-v /home/redacted/conf/gphotos:/config \
-v /mnt/redacted/iCloud_Sync/:/photos:ro \
--restart unless-stopped \
rfgamaral/gphotos-uploader
with config.hjson
{
APIAppCredentials: {
ClientID: "redacted",
ClientSecret: "redacted",
}
jobs: [
{
account: redacted
sourceFolder: /photos
makeAlbums: {
enabled: true
use: folderNames
}
deleteAfterUpload: false
uploadVideos: true
}
]
}
Attempting to run on Raspi3, everything works until I run the following for first startup of the container:
docker run -d \
--name=gphotos-uploader \
-v /home/pi/GPhotos/config:/config \
-v /home/pi/GPhotos/media:/photos \
--restart unless-stopped \
rfgamaral/gphotos-uploader
I can see the /config
and /photos
directories appear, but no /config/config.hsjon
being created. If I run docker logs -f gphotos-uploader
I get:
$ docker logs -f gphotos-uploader
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
standard_init_linux.go:207: exec user process caused "exec format error"
Which is probably due to the GOARCH=amd64
in the Dockerfile being incompatible with my architecture.
Rather than having the URL redirect to a failed localhost page and having to paste in the entire broken URL to /oauth.sh
, you should use access_type=offline
to get a fancy result page that gives the token to copy/paste into the offline app.
Describe the bug
I've a problem with authenticating in docker. I've the message in my browser ( "You are authenticated, you can now return to the program. This will auto-close" ). But in the shell, I have the message : "
root@nas:~#` docker exec -it gphoto run
[info] Opening browser to complete authorization.
[warn] Browser was not detected. Complete the authorization browsing to: http://localhost:29070
[error] error while authorization: error while exchanging authorization code and token: Post
https://oauth2.googleapis.com/token: x509: certificate signed by unknown authority
[fatal] could not get a token: error while exchanging authorization code and token: Post https://oauth2.googleapis.com/token: x509: certificate signed by unknown authority
Desktop & Client versions (please complete the following information):
Precision : I'm running the container on a headless server. So I've got to authenticate through a browser on another machine. In the browse I get the success page but if I log the container I've got what I mentioned above.
As per gphotosuploader/gphotos-uploader-cli#59, nmrshll/gphotos-uploader-cli upgraded some core libraries. Can you update the docker package?
Hi
Are you still maintaining this container? gphotos-uploader-cli has reached v1.2.0 while this container is stuck to 1.0.6.
Thanks!
Hello, first time trying this out and I've noticed my photos are getting uploaded correctly, but a new album with the photos does not get created despite having makeAlbums
enabled.
I'm running on Raspbian Buster and I've tried both the :latest
and :preview
tags of docker-gphotos-uploader. Let me know if this is something I should take up with the actual gphotos-uploader-cli, but I wanted to post here first in case it's an issue with the docker image.
[
{
"Id": "sha256:9ab43754c45118d7ac996aa0c9fd44c46e5e99b6e224641d926dd6240ed258ee",
"RepoTags": [
"rfgamaral/gphotos-uploader:preview"
],
"RepoDigests": [
"rfgamaral/gphotos-uploader@sha256:9d5edf4352c396ae95738b6dbc51f0d9757a427f1a5c7e34d5f55e139b56aa5e"
],
"Parent": "",
"Comment": "",
"Created": "2020-04-12T16:25:10.130368282Z",
"Container": "04765c1840022356218e33679b31b2d3b20d59ece20edb8229e1cd6eed9ffb45",
"ContainerConfig": {
"Hostname": "04765c184002",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"GPU_SCHEDULE=0 */8 * * *"
],
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ",
"ENTRYPOINT [\"/init\"]"
],
"Image": "sha256:5f214418202c7d1e249197e54a956b3ead08d27d9902ffd2e1216247f9cbe167",
"Volumes": {
"/config": {},
"/photos": {}
},
"WorkingDir": "",
"Entrypoint": [
"/init"
],
"OnBuild": null,
"Labels": {
"maintainer": "[email protected]",
"org.label-schema.build-date": "2020-04-12T16:23:28.978Z",
"org.label-schema.description": "Mass upload media folders to your Google Photos account with this Docker image.",
"org.label-schema.name": "rfgamaral/gphotos-uploader",
"org.label-schema.schema-version": "1.0",
"org.label-schema.vcs-ref": "0fa16ed",
"org.label-schema.vcs-url": "https://github.com/rfgamaral/docker-gphotos-uploader.git"
}
},
"DockerVersion": "3.0.11+azure",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"GPU_SCHEDULE=0 */8 * * *"
],
"Cmd": null,
"Image": "sha256:5f214418202c7d1e249197e54a956b3ead08d27d9902ffd2e1216247f9cbe167",
"Volumes": {
"/config": {},
"/photos": {}
},
"WorkingDir": "",
"Entrypoint": [
"/init"
],
"OnBuild": null,
"Labels": {
"maintainer": "[email protected]",
"org.label-schema.build-date": "2020-04-12T16:23:28.978Z",
"org.label-schema.description": "Mass upload media folders to your Google Photos account with this Docker image.",
"org.label-schema.name": "rfgamaral/gphotos-uploader",
"org.label-schema.schema-version": "1.0",
"org.label-schema.vcs-ref": "0fa16ed",
"org.label-schema.vcs-url": "https://github.com/rfgamaral/docker-gphotos-uploader.git"
}
},
"Architecture": "arm",
"Os": "linux",
"Size": 35469705,
"VirtualSize": 35469705,
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/8f48e5fc17949d55932615496ee9d937bf201b9d13a6a3f8d8c3e26f99b1633b/diff:/var/lib/docker/overlay2/c36bcee1e1089c835b692c6ea2528dc309adc84ac761ac559218496c03f88185/diff:/var/lib/docker/overlay2/962be7eabef1bf6aa27aecffed74df21ddf34d38be6dc13af05ad33dc499b107/diff:/var/lib/docker/overlay2/e81974fe90cee1b40e7a2c432a888d9ab3ca660b8361f154d822aef907519e16/diff:/var/lib/docker/overlay2/da52ab70781f2bff9217e237f3b6b2032a16733606a5969b2048ef6199ba4c07/diff",
"MergedDir": "/var/lib/docker/overlay2/ea65f110c35f4eaa3730de9c91ac6a380267b4f518f48729504c559ef99b6014/merged",
"UpperDir": "/var/lib/docker/overlay2/ea65f110c35f4eaa3730de9c91ac6a380267b4f518f48729504c559ef99b6014/diff",
"WorkDir": "/var/lib/docker/overlay2/ea65f110c35f4eaa3730de9c91ac6a380267b4f518f48729504c559ef99b6014/work"
},
"Name": "overlay2"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:a6a4e2cc29605ebe999068e75d49b94ef2ad4b9564426fe4b316259dd31fcd76",
"sha256:614eb43de3206ac7cb889d1b94c1fe7035708949b9b5ffb0ac4a2181aff4a873",
"sha256:75a5ec1b243f8082e77e7c220cc79720a389d4d1fcae72b0bbe258a220707d81",
"sha256:d521c6590e5199cd231ac1345bb448ce300ee7fabfd248c8fb0ef98d91c887da",
"sha256:78cc35ee1a25c26f31ec4918a525b920693fd356fe48bcb08442f3d2063b8fed",
"sha256:5d8a511c98f348b4ebd8cde0ea9c68be989d33ae67ac248c8e913ae21d36ebb8"
]
},
"Metadata": {
"LastTagTime": "0001-01-01T00:00:00Z"
}
}
]
{
SecretsBackendType: auto,
APIAppCredentials: {
ClientID: "<my_client_id>",
ClientSecret: "<my_client_secret>",
}
jobs: [
{
account: [email protected]
sourceFolder: /test_photos
makeAlbums: {
enabled: true
use: folderName
}
deleteAfterUpload: false
uploadVideos: true
includePatterns: []
excludePatterns: [ "*__id" ]
}
]
}
docker run -d --name=gphotos-uploader -p 29070:29070 -v /home/pi/photo-sync/config:/config -v /home/pi/photo-sync/test_photos/:/test_photos --restart unless-stopped rfgamaral/gphotos-uploader:preview
$ docker exec -it gphotos-uploader run
[info] Opening browser to complete authorization.
[warn] Browser was not detected. Complete the authorization browsing to: http://localhost:29070
[info] Found item to upload: /test_photos/2020-04-11_21-23-11_UTC.jpg
[info] 1 files pending to be uploaded in folder '/test_photos'.
[info] 1 processed files: 1 successfully, 0 with errors
After this is done, there is no new album in my Google Photos library. I would expect there to be one titled "test_photos".
It seems that any changes I make to the config.hsjon file get overwritten everytime the docker vm restarts. I have the folder mapped to the local folder. Ideas?
Will this only upload new photos from the source location? In other words, if I have a photos folder and keep updating it with new photos, this will only upload the photos that hadn't been uploaded yet? Or will it attempt to upload every photo in the source folder, even if it had already been uploaded in a past job?
Hello, I have this docker installed on a synology nas and appear to be having an issue with the files uploading. Any thoughts are much appreciated!
My docker settings are:
{
"cap_add" : null,
"cap_drop" : null,
"cmd" : "",
"cpu_priority" : 50,
"devices" : null,
"enable_publish_all_ports" : false,
"enable_restart_policy" : false,
"enabled" : true,
"entrypoint_default" : "/init",
"env_variables" : [
{
"key" : "PATH",
"value" : "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
},
{
"key" : "GPU_SCHEDULE",
"value" : "0 */8 * * *"
}
],
"exporting" : false,
"id" : "55d442804c1b51190ec3376045d27e65aff4acfd739a056e58563c4d4b129cee",
"image" : "rfgamaral/gphotos-uploader:latest",
"is_ddsm" : false,
"is_package" : false,
"links" : [],
"memory_limit" : 1073741824,
"memory_limit_slider" : 1024,
"name" : "gphotos-uploader",
"network" : [
{
"driver" : "bridge",
"name" : "bridge"
}
],
"network_mode" : "bridge",
"port_bindings" : [
{
"container_port" : 29070,
"host_port" : 29070,
"type" : "tcp"
}
],
"privileged" : false,
"shortcut" : {
"enable_shortcut" : false
},
"use_host_network" : false,
"volume_bindings" : [
{
"host_volume_file" : "/Photos/Albums/test",
"mount_point" : "/albums",
"type" : "rw"
},
{
"host_volume_file" : "/docker/gphotos_uploader",
"mount_point" : "/config",
"type" : "rw"
}
]
}
And my config file is:
{
SecretsBackendType: auto,
APIAppCredentials: {
ClientID: "xxx.apps.googleusercontent.com",
ClientSecret: "xxxx",
}
jobs: [
{
account: [email protected]
sourceFolder: /albums
makeAlbums: {
enabled: true
use: folderNames
}
deleteAfterUpload: false
uploadVideos: true
includePatterns: [ "*.jpg", "*.png" ]
excludePatterns: [ ".thumbs.db/*" ]
}
]
}
when i run docker exec -it gphotos-uploader run
i get:
2019/11/08 16:13:23 config doesn't allow to upload this item - skipping: file=/albums
2019/11/08 16:13:23 all uploads done
2019/11/08 16:13:23 all deletions done
It doesn't appear that it is finding any of the files within the mapped folder of /albums
which is:
/albums
/albums/New Orleans
If I run docker exec -it gphotos-uploader /bin/sh
and then:
/ # ls -l /albums
total 0
d--------- 1 1026 users 788 Sep 14 2017 New Orleans
/ # ls -l "/albums/New Orleans"
total 1412
drwxrwxrwx 1 root root 44 Nov 8 16:08 @eaDir
---------- 1 1026 users 112201 Feb 17 2013 AnneRicesHome.jpg
---------- 1 1026 users 57596 Feb 17 2013 BourbonStreet.jpg
---------- 1 1026 users 65527 Feb 17 2013 QHBedroom.jpg
---------- 1 1026 users 0 Feb 17 2013 QHCourtyardLarry.jpg
---------- 1 1026 users 96986 Feb 17 2013 QHCourtyardLaura.jpg
---------- 1 1026 users 70862 Feb 17 2013 QHKitchen.jpg
---------- 1 1026 users 49267 Feb 17 2013 QHLivingRoom.jpg
---------- 1 1026 users 104812 Feb 17 2013 QuarterHouseLobby.jpg
---------- 1 1026 users 766084 Feb 17 2013 Run.jpg
---------- 1 1026 users 71365 Feb 17 2013 TWWroteHere.jpg
---------- 1 1026 users 27136 Mar 28 2010 Thumbs.db
It does seems odd to me that the permissions are all dashes.
I have 3g2 files in my picture collection. What happens is:
2019/05/21 13:47:06 Uploading SSPX0014.3g2.skip
2019/05/21 13:47:07 failed uploading image
--- at /go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:33 (StartFileUploadWorker.func1) ---
Caused by: failed uploading image
--- at /go/src/github.com/nmrshll/gphotos-uploader-cli/upload/fileUpload.go:53 (FileUpload.upload) ---
Caused by: status message should be OK, found: NOT_IMAGE: There was an error while trying to create this media item.
--- at /go/src/github.com/nmrshll/google-photos-api-client-go/lib-gphotos/client.go:147 (Client.UploadFile) ---
Is there a way to tell the uploader to keep going?
I am running gphotos sync, configured to create an album for each folder.
The result is that I noticed that I have a lot of albuns (almost all of then) that should have more photos. The photos are being uploaded, but some of them are not being added to the album. If I go to Gphotos and click on add to the album, phey are there, uploaded.
The tool uploads the videos as uncompressed files, which reduces the available storage on gdrive.
Can this be changed with a parameter?
Hi, is this project still alive ?
This is a life saver or me. I want to upload ALL my library to Google Photos, and I'm stuck on the following problem:
The first time it runs, after analysing the library and select the files, it gives the following error:
.
.
.
[info] Found item to upload: /photos/Montagens e Fotos Gerais/Colagem gerações.jpg,
[info] Found item to upload: /photos/Montagens e Fotos Gerais/Colagem gerações.jpg.xmp,
[info] Found item to upload: /photos/Montagens e Fotos Gerais/IMG_0916.jpg,
[info] 315748 files pending to be uploaded in folder '/photos'.,
[error] error while trying to create this media item, err=Failed: There was an error while trying to create this media item.,
[error] error while trying to create this media item, err=Failed: There was an error while trying to create this media item.
of course I ommited all other file listing....
May this error be caused by the .xmp files ? Or is this a bigger problem...
And whats is the syntax for it to ignore ?
Thanks.
I've done everything as your guide describe but after launching and configuring the API, I cannot run the container to proceed to the authentification, I get the following error :
/ # run
[fatal] open completed uploads tracker failed: path=/config/uploads.db, err=resource temporarily unavailable
I checked in the config files and the uploads.db is present with it's file, would you happen to know what would be the issue ?
Thank you
Is it possible to specify multiple folders for upload?
e.g.,
sourceFolder1: /photos1
sourceFolder2: /photos2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.