Comments (27)
steilerDev
commented on June 23, 2024
1
There are a couple of weird things here:
- Each run seems to download an expected amount of files
- No errors in the logs when downloading and writing files to disk - so whatever we are getting should be written to disk
- However, on the next sync run, no additional assets are loaded (it's stuck at
Loaded local state: 86107 assets & 0 albums). It's not even that the assets that were previously added are invalid (in this scenario the number of 'to be deleted' assets would be higher - having a couple of files being invalid is expected)
The only thing I could think of is that we are downloading files that already exist. This should not happen because diffing should notice that the local and remote file are the same. The worst case scenario would be, that we are having file naming conflicts (the UUID I'm using for naming files is not actually a UUID). Because if we are at the point of writing files to disk I'm simply overwriting them.
Let me add some additional logging (once I fixed a current bug) and then we can validate if this is the case.
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
1
That's good and bad news!
This means our assumption is correct, however I think this will not be addressed fully until #354 - however it is an important consideration!
You should see more progress now because already existing files will be skipped - before the change it would re-download the file.
Now the important question would be: Are those duplicates or distinct files that have the same ID. Let me clean up the warnings and create a separate folder, where those conflicted files are written to - you can then check if those are duplicates (and it would be safe to ignore) or actually different files that we need to keep.
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
The sync failing every hour is expected - Apple's URL tokens expire after that and currently the easy solution is to fail and retry automatically.
The weird thing though is that in each execution run it is downloading a couple of thousand assets, but the number of local assets does not increase when they are loaded in the next run (and only a small numbers of files fails validation and get deleted). How many files do you see in your _All Photos folder / does the disk usage increase?
And from which run are those logs?
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
How many files do you see in your _All Photos folder / does the disk usage increase?
| Command | 23:30 | 7:30 | diff |
|---|---|---|---|
du -k ./_All-Photos |
581683716 | 581714664 | +30.948 |
du -k ./_Shared-Photos |
320221000 | 319965012 | -255.988 |
find ./_All-Photos -type f | wc -l |
57772 | 57772 | 0 |
find ./_Shared-Photos -type f | wc -l |
28538 | 28538 | 0 |
So it seems like there is absolutely no progress being made. 😢
And from which run are those logs?
Those logs were from yesterday 2023-09-28 10:35 - 15:30
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
1.3.0-beta.1 should log a warning, in case my assumption is correct. Could you try and run it and report back @halbtuerke
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
There are ~5700 entries of the Skipping download of... message in the logs.
- ~3200 in the
All-Photosfolder - ~2500 in the
Shared-Photosfolder
Here are some examples:
[2023-10-02T07:57:02.537Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AfOyvgkFncw-xHiRSYu7n29m3sqnAf-yidbOU19sKfsoNbTD9FIeHNwO/${f}?o=ApvIB_fNfcgWYjScTntc1sfsc2pxFfP_D6i2wBAu7U0o&v=1&x=3&a=CAogxl53NxRm_H_cVBh6-uEz-xjGKSBR35_dkx2vdp7H-KYSaxC7gYr7rjEYu97l_K4xIgEAUgRm3sqnWgQeHNwOaiX806682rfBo4apszTjNcOS9BPGOvz1fV4wjj0Y5mwuo-YsuuJqciUz59usNOWiKxQXidWo9O0vwoNNPE_hGITLf37cgqmh7E6S53Q9&e=1696236924&fl=&r=ee1fb29d-fbb9-451b-8e9d-32ace7cd8650-1&k=UlvQlG5UPJobZd9K4cEVPQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=rRdq9SAFtvrI90vjknj_i5wyTv0 to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AfOyvgkFncw-xHiRSYu7n29m3sqn.jpeg as file already exists
[2023-10-02T07:57:02.537Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/ATikmtq01iWPgrlBu7F7cZoGF8puAXnR5pMDm4qpFgmK9l4a-KTG5I1F/${f}?o=Attq_hQjW7VbmQoUZJWT_uv0hNTjXofzcEnzyURVBJgb&v=1&x=3&a=CAogJiKdhmSw6vT5_hGi53LRYiS0e-ihbd2EWdIFt1GEX3QSaxCPjYr7rjEYj-rl_K4xIgEAUgQGF8puWgTG5I1FaiUwyucBLpKv9cWYqOcK43O9DwPvLq5U-oIbBPpSnpGnPZStYtkVciWRjedkurPGdZxI9p8Hj2wY10Wn0rAHFrPA1dQ_1jY1YZgtyvy4&e=1696236926&fl=&r=fcc97b55-40bf-417d-8c29-cddd3c12719b-1&k=eYSamczhdroOMzPRK5559Q&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=y1Psi2x-COswIdxTRaAx1GRnwLQ to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/ATikmtq01iWPgrlBu7F7cZoGF8pu.heic as file already exists
[2023-10-02T07:57:02.538Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/ATc43vPaLgGwVIQzj3y5qrv5dOs3AQZHCga0rMmVHwgVFJgtGf6U5XWM/${f}?o=AgllOklBGLP2mjIUKArMnR_0oCgNu9O52Y0t2olXhqJB&v=1&x=3&a=CAogLuahuEe79xUydVKDG0LU4Han6ypGKZFzBsCdBHfVUa0SaxCujIr7rjEYrunl_K4xIgEAUgT5dOs3WgSU5XWMaiUNsqtBveMFKGS-XKFg8y3PwyLA2aE7kFiBEIGZSlod2lQ5cLZociUYX9OSkAkw9Cc_rYAXhspaeSmzVGc0Ad8WnSy2zEkCQD8aF59H&e=1696236926&fl=&r=0cc2ba43-2842-490f-b9f0-f79b9bc3cc31-1&k=wfGpmLpqmv0h5kM2F7ciig&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=DvEPMc6iCpOr519ENpDVBwuiSn8 to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/ATc43vPaLgGwVIQzj3y5qrv5dOs3.jpeg as file already exists
[2023-10-02T07:57:02.538Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AeOi5QkJ-TwQ71lep6v0JuyaYHGyAR5n7eJS4zw4tNp1nfki8MX8AuWW/${f}?o=AnQbFf4e_7z6t4jXOGGvl31via885YWQnDDgir9L0nlG&v=1&x=3&a=CAogP6nBxipY34m_dMGFCEc9NWLWduToGxdfMoshMqN5kMoSaxCkkYr7rjEYpO7l_K4xIgEAUgSaYHGyWgT8AuWWaiWCYg-8QCkFClQDR36-Nwd3WYeVL3mmnNaBTskZ6BhhAEZ00CqDciUWu952ICeBQlHWyX8KqyiWv9BKSLLrPP1c5hZ3aHSvjubT4SAm&e=1696236926&fl=&r=b96de211-f7bf-4ec6-9155-d6ed78ec6e95-1&k=GMbNOAJ95h2E515oSWT5eg&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=u9eQPOvBxi-nNKTNZh6TU0DhPN4 to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AeOi5QkJ-TwQ71lep6v0JuyaYHGy.heic as file already exists
[2023-10-02T07:57:02.538Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AVU3zstvhywey0aYlhDLSfZ0PbcDAchQsVizMI4_NcRjSYNSexPGKbUt/${f}?o=AlcuePXHnL1wGUYSAQWaKTJKskF0IjQ3jjskABiBvm9U&v=1&x=3&a=CAogat3EDNpFic2ja-X7zewaDzC-pcTZwznLcm6NEqAu2KoSaxCxkor7rjEYse_l_K4xIgEAUgR0PbcDWgTGKbUtaiXNHR5zwfvb5WP9J44n3SUVFMSgzlqZ9ZfI7eqSCaTGjQqCV5vkciXYslTpDvJMVb37HG3e6WoxEkpc---uxXaHNEW5lNTiqPbE47_z&e=1696236926&fl=&r=9661e640-24e9-489d-9680-2e947274a683-1&k=WyWy2iJlL6_imPTPa6yBkA&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=LB0izroFNDSGfqATy-SniNk7d0g to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AVU3zstvhywey0aYlhDLSfZ0PbcD.heic as file already exists
[2023-10-02T07:57:02.538Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AV7HjHt5i2eAnxTmJE9qdVlxXDn0AVeEEN9FYv-YD1YH-ymx7G8BwMms/${f}?o=Ao4JAbj6T0ABCXJZxyqmoAfD5B2mk6q-HCHDwo-z82e9&v=1&x=3&a=CAog9SH1UwgBi-QqmXNLFCY96X6zDmP2xqExmWz5qDx4gYgSaxD2j4r7rjEY9uzl_K4xIgEAUgRxXDn0WgQBwMmsaiVmOcVACkvnfKPOmKOedgKq8oAnz3_hdeUe0iekFCxcDcLIVrieciXD1XcyMsgiXPRm-j0agnLv2RWtcrbBL7CG8Crm2NLCADe7tbDF&e=1696236926&fl=&r=6a78c4e7-c28b-48d8-b86e-15e23b1971ec-1&k=DrA7y0vcTb4KmNk67Nc8cg&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=6AGMVhvsZuQO0vmp49f0ghwd5FQ to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AV7HjHt5i2eAnxTmJE9qdVlxXDn0.jpeg as file already exists
[2023-10-02T07:57:02.558Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/ATIb4L3pzTAgNA2Wz0BXaIEqTKYlATZHeRTIB8RXyaPeLdWDkmZvM5YM/${f}?o=ArnG-GmGAHK8V0v3cYLnrmBRv_e5pJVenq_uAqp8KRG8&v=1&x=3&a=CAogIqmOEHX48DyQtgQnTa1JNd7HaKp3ZeZ4efXYfr_vbjMSaxDeqIr7rjEY3oXm_K4xIgEAUgQqTKYlWgRvM5YMaiUJlGNm-rJ73d0Yp-jCnDTVjjSq_Tr3Il-QdnLNpal6R3KHNA3pciXIxgKhn3uc19PUV58MravqwDMuzMeebPjC0kEcIcqwbqUewYML&e=1696236929&fl=&r=6171002c-11e2-4b1c-a6e0-d0a26b5218e9-1&k=PIStNQ-0dlFgE8BHqtsVdw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=14TlSBRJ7d2eAAuCI5kqbOVmn-I to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/ATIb4L3pzTAgNA2Wz0BXaIEqTKYl.jpeg as file already exists
[2023-10-02T07:57:02.560Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AaPtUcO-Pr2VeNgDB0whOvQMFbipASrM_vI9hQqUmksK8Xw1NyF8I29U/${f}?o=AsXkB_vcV3srdXM-szlq-GOH3k6MlPTUbocnXtEURyzE&v=1&x=3&a=CAogkA0XPMWzssXEsa_BHGFU7BI03IRSJgDK4XB-2YS_we4SaxDYq4r7rjEY2Ijm_K4xIgEAUgQMFbipWgR8I29UaiXWY8_dHFYLO8bnca_Ze29gmt2rGIGMSPrLgQiM-jkdoncTS88iciX9yW5KB5U97uiQtWZ3op_cPxdgT-Jk0QS4b6Vzzu6K_EkpngfX&e=1696236930&fl=&r=42537e58-8749-4827-8df6-d9313ace6372-1&k=C1CXgq-f4NyJJZbfEc8dJA&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=hewYachcjHB3LhgJ8clBXwjf7CY to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AaPtUcO-Pr2VeNgDB0whOvQMFbip.jpeg as file already exists
[2023-10-02T07:57:02.560Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AdPMjgH05zOBx1M_9FjpVvlwk_yoARlIQTdSByFJkFb-AeGe2-5u52BP/${f}?o=AhFtkxsGCkjPyUG40eo-HZ-0ApvaGlt19a73NLVLwN1O&v=1&x=3&a=CAogx62UWsiWvSoMcyAQo02UoZFvpiVtdzWDQnnYxr5OwoYSaxCRrYr7rjEYkYrm_K4xIgEAUgRwk_yoWgRu52BPaiVHL9c1JpLbpT9gP2a0C9_-3t3t5vochOnwXe0UUj_wf7NlG5d3ciWchKtQUJSm9LE2kirHuygviGNP5tx5z2dIiwoF3_rqMNSttJiq&e=1696236930&fl=&r=f0518c0a-5feb-492d-8c4a-e7cc4ae8fe6d-1&k=is29TzrbF6fm8sKvPwAITQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=Yoyl92a2MirHL-MkXxFshjHNOAo to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AdPMjgH05zOBx1M_9FjpVvlwk_yo.jpeg as file already exists
[2023-10-02T07:57:02.560Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/ARkKPPyn_yQ3QCmRhnDHr-9gn88hAQ19TVsN7pgaJnh0FiR0s-PCtjjw/${f}?o=Alk8KMlvrWO0oAwttUwQg7b0pThZEb_zw9JadiyzTMUz&v=1&x=3&a=CAogbmsr3--pnB03bMMNLQJpkMBIBR2zcOR7OedCcYVcnuUSaxDdr4r7rjEY3Yzm_K4xIgEAUgRgn88hWgTCtjjwaiWvs774yPfmQb5Maga5PnEMG4UcP7p_CfQZWcvnB0H8bA_BgW1ZciVPt6UJihCbXrjqnS0btyAmctcup5179h1RsLkxXndW1lA9vcFO&e=1696236930&fl=&r=9eeb359e-a219-4b3f-8266-ef712a4274c9-1&k=-YSA5-aVwrd1BlWtFstGZQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=eOaCMg2RUnCAQmhOpOb1Insgu28 to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/ARkKPPyn_yQ3QCmRhnDHr-9gn88h.jpeg as file already exists
[2023-10-02T07:57:02.561Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/ARQmBHEtYnXU4jwoQrQ20khA-JQbAarefYd2p8jc6u65QpuQsrpxdOiO/${f}?o=ApW3SLpIeIX-JxEXG8Wd6CZztTQ4JZWTk9CPDcp-dg2O&v=1&x=3&a=CAoghk-s4ryxwJE1DrPAeTeOMSqdGSTnuj7I9pA6QCF4Wq4SaxDytIr7rjEY8pHm_K4xIgEAUgRA-JQbWgRxdOiOaiWnylSRSW8kLxQnlKuMtcRbkkRM7r3_O396Fz0Id6o-PKZkkKNDciXUMiHRLCbSVSwDrpY5tsgrgYtrWeUuyR4v1W2pNVdl46g6VFx4&e=1696236931&fl=&r=df060f69-36d0-41fe-87e2-6394426f36d4-1&k=jVoB1K4S0Ruoz0zYRJC_Fg&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=DhCL8XNBi7jZS8nmh1URvUPaFXI to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/ARQmBHEtYnXU4jwoQrQ20khA-JQb.jpeg as file already exists
[2023-10-02T07:57:02.561Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AXxv6TmcecKdj84QhEEmyh1V4E-sAciSZBhUmkmM677ePJPnen-Swjnv/${f}?o=AiAD5yrwbjwfXcSvTOp1rWgnNeE0bmZatuo0kO8wF5iV&v=1&x=3&a=CAog2NQLuEZzcYK4WYcDg8rPUpIecS0dKeqN30k0S3Hl9jMSaxCSvIr7rjEYkpnm_K4xIgEAUgRV4E-sWgSSwjnvaiVR3Ze_cq-G25jr5DwP0KFSohwiQYB6xatHenpiC8iYIBEfYUsrciXuTmvNd8pdrGSm92qv4oJYoUBUj71Q1YpUiU4ShiiZxE9JhLxH&e=1696236932&fl=&r=a79882c3-8e6c-46a5-851d-4bd2743d7e1e-1&k=4TiMl4sHpxwB7qchw-hEvQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=7QUP8kDInfY0r42H6wVOLfeMKN0 to /Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AXxv6TmcecKdj84QhEEmyh1V4E-s.mov as file already exists
[2023-10-02T07:57:53.083Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AW5I0mhjHlvqlWkMwNcHFvfuFVTFAQor9ZUuYL3I4MnYD3HDkaSZSFw6/${f}?o=AoyClGsOPkb7a2I0fKQt4UOAHZZGkRf5kN9CBLUYc3nv&v=1&x=3&a=CAogdb5YBkLEPOyMaDlxlKqsv_4ukr0CcJpaIYpybJ-CfCASaxCOiYv7rjEYjubm_K4xIgEAUgTuFVTFWgSZSFw6aiXEiQpB5OWxxZCAH69EaJNBguraXmCtrT7-YW2KzAB5nuT7vneEciWqubPxHmN1Dl7aJadpeOzOorvLDR_WNcj5nfhMZwgfnANH-mpk&e=1696236942&fl=&r=e4cfef16-1e5f-44ff-a9ba-8e597bb85d65-1&k=SpXXjBiyD8_R2CajzsKo9Q&ckc=com.apple.photos.cloud&ckz=SharedSync-50A3C536-9BE0-457A-8A26-A8FD65DE083A&y=1&p=58&s=jqIJRxr_cgP7X_C4Z0eeWz-0yTM to /Volumes/Extreme SSD/icloud-photos-sync/_Shared-Photos/AW5I0mhjHlvqlWkMwNcHFvfuFVTF.heic as file already exists
[2023-10-02T07:57:53.083Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AcUVSByyL23BpgEUQ_JMn8lXHGs_ATj-u7PL5-WsgcOxF0ad7u_muAU0/${f}?o=ApVav2ebI8dOdk6266fvYEAjUMtmwptgqqQFl4kj-IAF&v=1&x=3&a=CAogOvaooR5f0tfPFVpOMTGPxI24dNxV-PyrSPNzNRPVktkSaxDShov7rjEY0uPm_K4xIgEAUgRXHGs_WgTmuAU0aiWIW-kds3ByT2_GcuGuV3woPgu8bWK9DdqxnEMKQPcRZxaCgSe4ciVJ35gQMpQOOsVvwMiglTt3H7Iy7WjluaSyZirSnW_8ARpbtj4x&e=1696236941&fl=&r=75119965-e1af-4d16-ab4e-17ff4da3803e-1&k=2HV54FkjXzuc8L46Wmn1WQ&ckc=com.apple.photos.cloud&ckz=SharedSync-50A3C536-9BE0-457A-8A26-A8FD65DE083A&y=1&p=58&s=q17ErsjMVSN6wDlHGh52k67RQrY to /Volumes/Extreme SSD/icloud-photos-sync/_Shared-Photos/AcUVSByyL23BpgEUQ_JMn8lXHGs_.heic as file already exists
[2023-10-02T07:57:53.083Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AQ2UBfAA4TfovvOiFMJSHG3POVRuAew7MUBsrJKXuKYMmuBZZQsh45Pm/${f}?o=AkH2KWDeI_409wNA-KnAd3ULPPzgYApchNT91z7D50PV&v=1&x=3&a=CAogRPq1Tnhr9zSQt8TdOPKbaI8txiDl7gK6y4GzWijmxn0SaxDZhov7rjEY2ePm_K4xIgEAUgTPOVRuWgQh45PmaiWszJnHGk5kqEb2LDzOZy9DctMJQUji8TlC50qyML8suVKwMm_qciXsgvKzvMPehhxM6VaI3pSerSrZBUOkPMHsxH-goh5mXvolBCmi&e=1696236941&fl=&r=75119965-e1af-4d16-ab4e-17ff4da3803e-1&k=KeFP81qhY5D8iPHoB24b9Q&ckc=com.apple.photos.cloud&ckz=SharedSync-50A3C536-9BE0-457A-8A26-A8FD65DE083A&y=1&p=58&s=VyQEUceKUvnBBmMj5plVxOLjkDU to /Volumes/Extreme SSD/icloud-photos-sync/_Shared-Photos/AQ2UBfAA4TfovvOiFMJSHG3POVRu.heic as file already exists
[2023-10-02T07:57:53.083Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AULqwlReOizZfb5jAcpYqoZWK7rNAauV4UnoWLVA7V-VxF_016IB-VzC/${f}?o=AvjJSB09cRYoVw32FWt5QMmBaCwMTfExxnKoqWqrRlme&v=1&x=3&a=CAoguRXJiyKKZCK1Ljz9qadIU2sZprKtuvYhiuZulE69icISaxDfhov7rjEY3-Pm_K4xIgEAUgRWK7rNWgQB-VzCaiUeo8SL3U4kRUnjfpRT8eyuHbb9JqPAeG0MP2MW4htI4Hzu6JaIciWpOVEzI4sB3IgFoN8tPY9XBkAln_qP7GVd9zFs_l25nmwFkg2I&e=1696236941&fl=&r=75119965-e1af-4d16-ab4e-17ff4da3803e-1&k=oXddsGknUzqVIqWEEUP1Bw&ckc=com.apple.photos.cloud&ckz=SharedSync-50A3C536-9BE0-457A-8A26-A8FD65DE083A&y=1&p=58&s=tJHBx32QxUn0BCArVFo68BDyShQ to /Volumes/Extreme SSD/icloud-photos-sync/_Shared-Photos/AULqwlReOizZfb5jAcpYqoZWK7rN.heic as file already exists
[2023-10-02T07:57:53.083Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/AWivSppuKBSK3Xs9Qj8dbtNTcg0cAbLCk62VzgaXLyulf5swyeoVFVdv/${f}?o=AosLFDvT3HlHhvGwLavB40AGuF_xZ76YZddbcdN--yAw&v=1&x=3&a=CAogkaIkYlNtNc5cgMXVDfyheFqvb-pe-lvscbG5_Rtt0wUSaxDmhov7rjEY5uPm_K4xIgEAUgRTcg0cWgQVFVdvaiW1HyNrPyLjfQgE6iMhS5ldsuMMvI0-HUXzxX5iqSPQyldtvtsVciWLZfU-8TGNPCpzim8oobh2Zq8luDP4wpiLA1lrhuJ3W3fkX_Da&e=1696236941&fl=&r=75119965-e1af-4d16-ab4e-17ff4da3803e-1&k=ccPtzagNUroskCgXfd_QzA&ckc=com.apple.photos.cloud&ckz=SharedSync-50A3C536-9BE0-457A-8A26-A8FD65DE083A&y=1&p=58&s=Y_gI7EUoO-jt9m-pfiSnPBSQ5yk to /Volumes/Extreme SSD/icloud-photos-sync/_Shared-Photos/AWivSppuKBSK3Xs9Qj8dbtNTcg0c.heic as file already exists
[2023-10-02T07:57:53.084Z] WARN NetworkManager: Skipping download of https://cvws.icloud-content.com/B/ASr6HovI9ZB7cgD7C-fRWgcsBpPtAZuk9Jt6_OQv5oDmv0P8oVgFPpSZ/${f}?o=AtdiRyU39ZFHr6tuO4d1aBAAZaFXbLKx-qr1XRX_SknD&v=1&x=3&a=CAogJ_PnZzkBvGfxKbzU0vBoFlMzH2TPd3TiJqAQxefgvF4SaxDthov7rjEY7ePm_K4xIgEAUgQsBpPtWgQFPpSZaiX8ZPEHlF5lTU_P50YL1JtYLjVzVIsrVchItG4R6adSaCkJNG9_ciV062f8Ha0UxHmoQGau_8VBTWWwy0EdSpwtn-63uONdJEKmrtSO&e=1696236941&fl=&r=75119965-e1af-4d16-ab4e-17ff4da3803e-1&k=eBv6rmUnVqBybJE0_6nW8w&ckc=com.apple.photos.cloud&ckz=SharedSync-50A3C536-9BE0-457A-8A26-A8FD65DE083A&y=1&p=58&s=068gXnkhALSmVYNOQ6PBZ9L03_o to /Volumes/Extreme SSD/icloud-photos-sync/_Shared-Photos/ASr6HovI9ZB7cgD7C-fRWgcsBpPt.heic as file already exists
Interesting enough there is still some progress being made in the last hours, but that could only be a coincidence:
Syncing assets, by keeping 86269 and removing 12 local assets, as well as adding 14751 remote assets...
■■■■■■ 15% | Elapsed: 59m34s | 2281/14751 assets downloaded
Syncing assets, by keeping 87135 and removing 19 local assets, as well as adding 13887 remote assets...
■■■■■■■ 16% | Elapsed: 58m52s | 2335/13887 assets downloaded
Syncing assets, by keeping 88016 and removing 20 local assets, as well as adding 13006 remote assets...
■■■■■■■ 18% | Elapsed: 58m49s | 2399/13006 assets downloaded
Syncing assets, by keeping 88948 and removing 20 local assets, as well as adding 12074 remote assets...
■■■■■■■■■ 22% | Elapsed: 56m11s | 2733/12074 assets downloaded
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
I've just added some rough (and untested) debugging logic to v1.3.0-nightly.4 that should help us understand the issue better:
If a duplicate is detected (file already exists) the tool will create a new folder within the data dir (duplicates) and:
- Create a link to the file that is already there (named
<filename>-orig) - Download the duplicate directly into this folder (named
<filename>-dup-<randomInt>- I've added the random int in case there are multiple duplicates)
If you could then check the duplicate filenames to see if those are actually the same file or different ones (i.e. is it a naming conflict, or a de-duplication problem)
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
Seems like the logic is not quite right. I'm getting the following error message and the sync immediately fails:
Syncing assets, by keeping 92316 and removing 0 local assets, as well as adding 8726 remote assets...
0% | Elapsed: 0s | 0/8726 assets downloaded
Detected error during sync: SYNC_UNKNOWN: Unknown error during sync caused by EEXIST: file already exists, link '/Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AeOi5QkJ-TwQ71lep6v0JuyaYHGy.heic' -> '/Volumes/Extreme SSD/icloud-photos-sync/duplicates/AeOi5QkJ-TwQ71lep6v0JuyaYHGy.heic-orig'
Refreshing iCloud connection & retrying (attempt #5)...
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
ah of course...one sec....and I guess there is already some stuff in the duplciate folder?
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
Yes, there are 12 items in the duplicates folder:
AeOi5QkJ-TwQ71lep6v0JuyaYHGy.heic-dup-68595
AeOi5QkJ-TwQ71lep6v0JuyaYHGy.heic-orig
AfOyvgkFncw-xHiRSYu7n29m3sqn.jpeg-dup-439576
AfOyvgkFncw-xHiRSYu7n29m3sqn.jpeg-orig
ATc43vPaLgGwVIQzj3y5qrv5dOs3.jpeg-dup-319799
ATc43vPaLgGwVIQzj3y5qrv5dOs3.jpeg-orig
ATikmtq01iWPgrlBu7F7cZoGF8pu.heic-dup-172507
ATikmtq01iWPgrlBu7F7cZoGF8pu.heic-orig
AV7HjHt5i2eAnxTmJE9qdVlxXDn0.jpeg-dup-299436
AV7HjHt5i2eAnxTmJE9qdVlxXDn0.jpeg-orig
AVU3zstvhywey0aYlhDLSfZ0PbcD.heic-dup-866613
AVU3zstvhywey0aYlhDLSfZ0PbcD.heic-orig
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
And how is the content of those files looking? Are e.g. AeOi5QkJ-TwQ71lep6v0JuyaYHGy.heic-dup-68595 and AeOi5QkJ-TwQ71lep6v0JuyaYHGy.heic-orig the same file?
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
Also: try 1.3.0-nightly.5 (which is currently getting deployed)
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
The files are looking the same and they also have the same file sizes.
For easier comparison it would be helpful if the files would be called AeOi5QkJ-TwQ71lep6v0JuyaYHGy-dup-68595.heic and AeOi5QkJ-TwQ71lep6v0JuyaYHGy-orig.heic, otherwise I have to rename them manually in order to open them.
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
Try 1.3.0-nightly.6 (once deployed) :)
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
Unfortunately I'm still getting
Detected error during sync: SYNC_UNKNOWN: Unknown error during sync caused by EEXIST: file already exists, mkdir '/Volumes/Extreme SSD/icloud-photos-sync/duplicates'
Refreshing iCloud connection & retrying (attempt #2)...
and
Detected error during sync: SYNC_UNKNOWN: Unknown error during sync caused by EEXIST: file already exists, link '/Volumes/Extreme SSD/icloud-photos-sync/_All-Photos/AfOyvgkFncw-xHiRSYu7n29m3sqn.jpeg' -> '/Volumes/Extreme SSD/icloud-photos-sync/duplicates/AfOyvgkFncw-xHiRSYu7n29m3sqn-orig.jpeg'
Refreshing iCloud connection & retrying (attempt #3)...
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
Have a try with 1.3.0-nightly.7 - this should handle the errors gracefully.
I also added additional logging to the diffing algorithm, to try and understand why those files end up in the queue to download (even though they are already on disk) - I you could search the logs for the duplicated file IDs and share the relevant pieces, this should help.
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
The most recent version is unfortunately hard crashing and therefore it's not producing any logs in which those duplicated IDs are shown:
Syncing assets, by keeping 92314 and removing 0 local assets, as well as adding 8798 remote assets...
0% | Elapsed: 0s | 0/8798 assets downloadednode:internal/process/promises:289
triggerUncaughtException(err, true /* fromPromise */);
^
[Error: EEXIST: file already exists, mkdir '/Volumes/Extreme SSD/icloud-photos-sync/duplicates'] {
errno: -17,
code: 'EEXIST',
syscall: 'mkdir',
path: '/Volumes/Extreme SSD/icloud-photos-sync/duplicates'
}
Node.js v20.8.0
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
The most recent version is unfortunately hard crashing
After some more investigations I figured out that the crash was caused by the newer Node.js version I have installed on my machine. After installing and using the current LTS, the crashes disappeared.
If you could search the logs for the duplicated file IDs and share the relevant pieces, this should help.
Here are the log lines for one of the duplicates (Aa_fDRoooKJJY-0nHI30XCM9rfi). The sync ran for ~9 hours and is now stuck at the last 8722 assets:
200806 [2023-10-06T22:46:21.467Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/AfItei6RTLo19UyN2h091dhYLsPXAckdzfuZ5sxghoCg0gB0FyrpAU65/${f}?o=AmZ-Ys62bZnPrC_v8We2vANZKL4rrhur6MbKAfvQSPDD&v=1&x=3&a=CAogIxdKM7eRaPlbGl3WJ5l8lKjjzzOio7OukGWduTk6GMgSaxDSy-u3sDEY0qjHubAxIgEAUgRYLsPXWgTpAU65aiVN7lR9SKsok3lm_PzVjkQIoOYoRNBAvZ4PcLb2GODPDNNiLLNrciVwhbvE2WZ_XxlSyCxeYoGJFUW_TQAhtot6L83EDF4UOgS68IiT&e=1696632788&fl=&r=31b8f91d-c030-41f7-9814-4dd83c11175f-1&k=2yePyBKJJ8UYKtGXkrp85Q&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=9KanIi9qCI7Y6CdMXPx88j5mRJA
200807 [2023-10-06T22:46:21.467Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/AfItei6RTLo19UyN2h091dhYLsPXAckdzfuZ5sxghoCg0gB0FyrpAU65/${f}?o=AmZ-Ys62bZnPrC_v8We2vANZKL4rrhur6MbKAfvQSPDD&v=1&x=3&a=CAogIxdKM7eRaPlbGl3WJ5l8lKjjzzOio7OukGWduTk6GMgSaxDSy-u3sDEY0qjHubAxIgEAUgRYLsPXWgTpAU65aiVN7lR9SKsok3lm_PzVjkQIoOYoRNBAvZ4PcLb2GODPDNNiLLNrciVwhbvE2WZ_XxlSyCxeYoGJFUW_TQAhtot6L83EDF4UOgS68IiT&e=1696632788&fl=&r=31b8f91d-c030-41f7-9814-4dd83c11175f-1&k=2yePyBKJJ8UYKtGXkrp85Q&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=9KanIi9qCI7Y6CdMXPx88j5mRJA
200808: [2023-10-06T22:46:21.472Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-369127.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-369127.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
200809: [2023-10-06T22:46:21.472Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AnCcyqOmsb2xQTljccFFYi74MqV-1jig5vzO0w6yFd6_&v=1&x=3&a=CAogmi1CkwuVf4jSZ1lBKU1MB2XYAX9ckKQWEG94i4P1nZ0SaxD8y-u3sDEY_KjHubAxIgEAUgQ9rfi-WgS9OF6NaiV-Fc1cUpgsOLD6OtKID5KAUoXlusj6X8Q-PwGAnVuMfpSXdqG6ciWko1zSP5rJaxuKcaoDiIDlF1xrUkp2tdwt76y5gIoIG4FHd6nq&e=1696632788&fl=&r=31b8f91d-c030-41f7-9814-4dd83c11175f-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=TJMXDFosE0s5J9gDOfhX6Xx3PTc
200810: [2023-10-06T22:46:21.979Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AnCcyqOmsb2xQTljccFFYi74MqV-1jig5vzO0w6yFd6_&v=1&x=3&a=CAogmi1CkwuVf4jSZ1lBKU1MB2XYAX9ckKQWEG94i4P1nZ0SaxD8y-u3sDEY_KjHubAxIgEAUgQ9rfi-WgS9OF6NaiV-Fc1cUpgsOLD6OtKID5KAUoXlusj6X8Q-PwGAnVuMfpSXdqG6ciWko1zSP5rJaxuKcaoDiIDlF1xrUkp2tdwt76y5gIoIG4FHd6nq&e=1696632788&fl=&r=31b8f91d-c030-41f7-9814-4dd83c11175f-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=TJMXDFosE0s5J9gDOfhX6Xx3PTc to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-369127.heic
200811: [2023-10-06T22:46:23.787Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AnCcyqOmsb2xQTljccFFYi74MqV-1jig5vzO0w6yFd6_&v=1&x=3&a=CAogmi1CkwuVf4jSZ1lBKU1MB2XYAX9ckKQWEG94i4P1nZ0SaxD8y-u3sDEY_KjHubAxIgEAUgQ9rfi-WgS9OF6NaiV-Fc1cUpgsOLD6OtKID5KAUoXlusj6X8Q-PwGAnVuMfpSXdqG6ciWko1zSP5rJaxuKcaoDiIDlF1xrUkp2tdwt76y5gIoIG4FHd6nq&e=1696632788&fl=&r=31b8f91d-c030-41f7-9814-4dd83c11175f-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=TJMXDFosE0s5J9gDOfhX6Xx3PTc
200811 [2023-10-06T22:46:23.791Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-357440.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-357440.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
200812 [2023-10-06T22:46:23.791Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-357440.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-357440.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
402749 [2023-10-06T23:55:44.108Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AjC3fE4S9V3fRIJiARbrUft_sNqyrHkxSENHF-Qg9Li6&v=1&x=3&a=CAogVXagR3msFCuo87l8d8JDLrsrya7x0sfpsxCWjFwDx34SaxDzyeS5sDEY86bAu7AxIgEAUgTQ6AqTWgTt6wTaaiUp52oJ28Dy0r-FH-Il-NqNJl6SyDexFRilBBibQYE7qL3EKPJGciWkO98MBMx1wQINC6uDNlNSCsfJ1ltDagnuDIXCdSNd3i8IYPEZ&e=1696636867&fl=&r=e36c9381-d6ea-446a-8921-6d06da3b9939-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=86AhpaoTMrlzFh14SA7iqG7bTvY
402750 [2023-10-06T23:55:44.108Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AjC3fE4S9V3fRIJiARbrUft_sNqyrHkxSENHF-Qg9Li6&v=1&x=3&a=CAogVXagR3msFCuo87l8d8JDLrsrya7x0sfpsxCWjFwDx34SaxDzyeS5sDEY86bAu7AxIgEAUgTQ6AqTWgTt6wTaaiUp52oJ28Dy0r-FH-Il-NqNJl6SyDexFRilBBibQYE7qL3EKPJGciWkO98MBMx1wQINC6uDNlNSCsfJ1ltDagnuDIXCdSNd3i8IYPEZ&e=1696636867&fl=&r=e36c9381-d6ea-446a-8921-6d06da3b9939-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=86AhpaoTMrlzFh14SA7iqG7bTvY
402751: [2023-10-06T23:55:44.111Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-901792.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-901792.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
402752: [2023-10-06T23:55:44.111Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=Av8LhBhzr-llIWrMzyfrvVVrxPlHKwJbR6Wy_b5w38Tb&v=1&x=3&a=CAog_yt9Ew7ZXmRS0GyKwwDxgGuaJQrMfAME10Jjug1GSxsSaxChyuS5sDEYoafAu7AxIgEAUgQ9rfi-WgS9OF6NaiW_b4Y-MeDSh2hkShozsPZURjQjQtaJ7M9cDFG84zy-iZJWWedhciURt0lqf0lX8ZTCj2WePZAWKBoG0lKw_lboOoxCd_Q9lWazmS6H&e=1696636867&fl=&r=e36c9381-d6ea-446a-8921-6d06da3b9939-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=NWIE1f4O6f8UZ80XNn4l-DLx1LY
402752 [2023-10-06T23:55:44.480Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=AmkkuEUSRhrK4cauU1akNi5h3jwWsp4tPGO6ANYG-vzD&v=1&x=3&a=CAog64NRKbF1ZDsz1N_djWC21gaa3Qbpm9x_E7Q52pmeSu0SaxCDyuS5sDEYg6fAu7AxIgEAUgSmkmy3WgREmuFuaiWUyCoTon5_mYqFMkujfUZnGC-0E3AuIuWUnb0rGv8J43zdEqMEciXvMLjd0QqoO0GibVWzdkE9o7pR5AHGJCXTFeTIDOZDWO3vvApA&e=1696636867&fl=&r=e36c9381-d6ea-446a-8921-6d06da3b9939-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=F4Uxg1pmSYQkbEhly3O36MorLx0 to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-59159.mov
402753 [2023-10-06T23:55:44.480Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=AmkkuEUSRhrK4cauU1akNi5h3jwWsp4tPGO6ANYG-vzD&v=1&x=3&a=CAog64NRKbF1ZDsz1N_djWC21gaa3Qbpm9x_E7Q52pmeSu0SaxCDyuS5sDEYg6fAu7AxIgEAUgSmkmy3WgREmuFuaiWUyCoTon5_mYqFMkujfUZnGC-0E3AuIuWUnb0rGv8J43zdEqMEciXvMLjd0QqoO0GibVWzdkE9o7pR5AHGJCXTFeTIDOZDWO3vvApA&e=1696636867&fl=&r=e36c9381-d6ea-446a-8921-6d06da3b9939-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=F4Uxg1pmSYQkbEhly3O36MorLx0 to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-59159.mov
402754: [2023-10-06T23:55:44.540Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=Av8LhBhzr-llIWrMzyfrvVVrxPlHKwJbR6Wy_b5w38Tb&v=1&x=3&a=CAog_yt9Ew7ZXmRS0GyKwwDxgGuaJQrMfAME10Jjug1GSxsSaxChyuS5sDEYoafAu7AxIgEAUgQ9rfi-WgS9OF6NaiW_b4Y-MeDSh2hkShozsPZURjQjQtaJ7M9cDFG84zy-iZJWWedhciURt0lqf0lX8ZTCj2WePZAWKBoG0lKw_lboOoxCd_Q9lWazmS6H&e=1696636867&fl=&r=e36c9381-d6ea-446a-8921-6d06da3b9939-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=NWIE1f4O6f8UZ80XNn4l-DLx1LY to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-901792.heic
402755: [2023-10-06T23:55:45.157Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=Av8LhBhzr-llIWrMzyfrvVVrxPlHKwJbR6Wy_b5w38Tb&v=1&x=3&a=CAog_yt9Ew7ZXmRS0GyKwwDxgGuaJQrMfAME10Jjug1GSxsSaxChyuS5sDEYoafAu7AxIgEAUgQ9rfi-WgS9OF6NaiW_b4Y-MeDSh2hkShozsPZURjQjQtaJ7M9cDFG84zy-iZJWWedhciURt0lqf0lX8ZTCj2WePZAWKBoG0lKw_lboOoxCd_Q9lWazmS6H&e=1696636867&fl=&r=e36c9381-d6ea-446a-8921-6d06da3b9939-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=NWIE1f4O6f8UZ80XNn4l-DLx1LY
402755 [2023-10-06T23:55:45.161Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-581831.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-581831.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
402756 [2023-10-06T23:55:45.161Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-581831.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-581831.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
604706 [2023-10-07T01:05:20.376Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AjPiuJHmDp7lzCZgptjx8KTePf7onRUPyyNt4-dJkf02&v=1&x=3&a=CAogmzEvYGD5BBjZkaxYjZ1_Bir6Jo2ZZpr4vtDT2gmg214SaxDA9eC7sDEYwNK8vbAxIgEAUgTQ6AqTWgTt6wTaaiXccDvHRNzmgrRWFijreJnuqcN2jlVV7EZJh2cwE6l4qm15MZOQciVgSXX6PWMtmmT_h7gTmpvxoxy-Ftb-amYqCicJsf254foOVs0b&e=1696641001&fl=&r=c8323a30-f10d-4db5-82e5-f92bb78339e9-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=dHfLDtyrxNNX3RvYrpX-f456B2E
604707 [2023-10-07T01:05:20.376Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AjPiuJHmDp7lzCZgptjx8KTePf7onRUPyyNt4-dJkf02&v=1&x=3&a=CAogmzEvYGD5BBjZkaxYjZ1_Bir6Jo2ZZpr4vtDT2gmg214SaxDA9eC7sDEYwNK8vbAxIgEAUgTQ6AqTWgTt6wTaaiXccDvHRNzmgrRWFijreJnuqcN2jlVV7EZJh2cwE6l4qm15MZOQciVgSXX6PWMtmmT_h7gTmpvxoxy-Ftb-amYqCicJsf254foOVs0b&e=1696641001&fl=&r=c8323a30-f10d-4db5-82e5-f92bb78339e9-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=dHfLDtyrxNNX3RvYrpX-f456B2E
604708: [2023-10-07T01:05:20.379Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-885537.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-885537.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
604709: [2023-10-07T01:05:20.379Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AqPASkZ6B-XeGp7043ynJ0KcwUMuP4-LHUHtBQ5Od2kM&v=1&x=3&a=CAogebcMeoUNry9yjCZ0ltiSnx6VbKDzM2zBFREZRl3LOwASaxDf9eC7sDEY39K8vbAxIgEAUgQ9rfi-WgS9OF6NaiUZ4JQFuN_IrJrm9n6WtpM-hYgZ4cOvrEZNkaIp8MCvNnseZ7MKciWYbAXHnb1crN-JusBrrUzJBQTvfzIds1JLMwd7EO_Xl8fybAC9&e=1696641001&fl=&r=c8323a30-f10d-4db5-82e5-f92bb78339e9-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=SA2pCUHion2TCOY6yQYQLxWD-i0
604710: [2023-10-07T01:05:20.821Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AqPASkZ6B-XeGp7043ynJ0KcwUMuP4-LHUHtBQ5Od2kM&v=1&x=3&a=CAogebcMeoUNry9yjCZ0ltiSnx6VbKDzM2zBFREZRl3LOwASaxDf9eC7sDEY39K8vbAxIgEAUgQ9rfi-WgS9OF6NaiUZ4JQFuN_IrJrm9n6WtpM-hYgZ4cOvrEZNkaIp8MCvNnseZ7MKciWYbAXHnb1crN-JusBrrUzJBQTvfzIds1JLMwd7EO_Xl8fybAC9&e=1696641001&fl=&r=c8323a30-f10d-4db5-82e5-f92bb78339e9-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=SA2pCUHion2TCOY6yQYQLxWD-i0 to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-885537.heic
604711: [2023-10-07T01:05:21.376Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AqPASkZ6B-XeGp7043ynJ0KcwUMuP4-LHUHtBQ5Od2kM&v=1&x=3&a=CAogebcMeoUNry9yjCZ0ltiSnx6VbKDzM2zBFREZRl3LOwASaxDf9eC7sDEY39K8vbAxIgEAUgQ9rfi-WgS9OF6NaiUZ4JQFuN_IrJrm9n6WtpM-hYgZ4cOvrEZNkaIp8MCvNnseZ7MKciWYbAXHnb1crN-JusBrrUzJBQTvfzIds1JLMwd7EO_Xl8fybAC9&e=1696641001&fl=&r=c8323a30-f10d-4db5-82e5-f92bb78339e9-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=SA2pCUHion2TCOY6yQYQLxWD-i0
604711 [2023-10-07T01:05:21.380Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-899640.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-899640.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
604712 [2023-10-07T01:05:21.380Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-899640.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-899640.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
806216 [2023-10-07T02:14:08.917Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AnIXSQV-hEbOQU6u8z8hBdxTHKQkszG1VP_3yC2LWxHo&v=1&x=3&a=CAog17MuVEcoez6a12oDzPDwwdSwsuHmRjZb8dY2nYeZuIgSaxCo8d29sDEYqM65v7AxIgEAUgTQ6AqTWgTt6wTaaiVHYY6JwIFeFJmlMrwnPFyXmtqaB3kcgKtDjN75kLBtvNG-z3B9ciWauxtLlSh1tXc0-ml8VjjMNrypMS8w0yP2K-qDaK8YBbk8iAed&e=1696645146&fl=&r=be1d549f-fa42-4e4a-8b49-5f7b0db5d5f3-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=ze_11mYgI5TYYq8r-blH-B6Ee4Y
806217 [2023-10-07T02:14:08.917Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AnIXSQV-hEbOQU6u8z8hBdxTHKQkszG1VP_3yC2LWxHo&v=1&x=3&a=CAog17MuVEcoez6a12oDzPDwwdSwsuHmRjZb8dY2nYeZuIgSaxCo8d29sDEYqM65v7AxIgEAUgTQ6AqTWgTt6wTaaiVHYY6JwIFeFJmlMrwnPFyXmtqaB3kcgKtDjN75kLBtvNG-z3B9ciWauxtLlSh1tXc0-ml8VjjMNrypMS8w0yP2K-qDaK8YBbk8iAed&e=1696645146&fl=&r=be1d549f-fa42-4e4a-8b49-5f7b0db5d5f3-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=ze_11mYgI5TYYq8r-blH-B6Ee4Y
806218: [2023-10-07T02:14:08.923Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-698092.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-698092.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
806219: [2023-10-07T02:14:08.923Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AhSfIG-khdBMTY5PETD4ZuLxD5BA7JS09E_4zz02uQPl&v=1&x=3&a=CAogiLJ60ysKG964vhVGC1IeR5UmVoNDXppvJq5zfKN74TgSaxDG8d29sDEYxs65v7AxIgEAUgQ9rfi-WgS9OF6NaiX9dgrE5ZzWzJLiyjUpIvgQTLFkgkkZRVfrRnJ71Pqxc9Ge1YR5ciWLHC_DjfJxjmyPrUKCP8G5-WgWplrL5puP7bbPmIW95W_FEkI0&e=1696645146&fl=&r=be1d549f-fa42-4e4a-8b49-5f7b0db5d5f3-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=rj7VSPt9VNX4xcVg1AAeLPAzon4
806220: [2023-10-07T02:14:09.347Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AhSfIG-khdBMTY5PETD4ZuLxD5BA7JS09E_4zz02uQPl&v=1&x=3&a=CAogiLJ60ysKG964vhVGC1IeR5UmVoNDXppvJq5zfKN74TgSaxDG8d29sDEYxs65v7AxIgEAUgQ9rfi-WgS9OF6NaiX9dgrE5ZzWzJLiyjUpIvgQTLFkgkkZRVfrRnJ71Pqxc9Ge1YR5ciWLHC_DjfJxjmyPrUKCP8G5-WgWplrL5puP7bbPmIW95W_FEkI0&e=1696645146&fl=&r=be1d549f-fa42-4e4a-8b49-5f7b0db5d5f3-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=rj7VSPt9VNX4xcVg1AAeLPAzon4 to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-698092.heic
806221: [2023-10-07T02:14:09.909Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AhSfIG-khdBMTY5PETD4ZuLxD5BA7JS09E_4zz02uQPl&v=1&x=3&a=CAogiLJ60ysKG964vhVGC1IeR5UmVoNDXppvJq5zfKN74TgSaxDG8d29sDEYxs65v7AxIgEAUgQ9rfi-WgS9OF6NaiX9dgrE5ZzWzJLiyjUpIvgQTLFkgkkZRVfrRnJ71Pqxc9Ge1YR5ciWLHC_DjfJxjmyPrUKCP8G5-WgWplrL5puP7bbPmIW95W_FEkI0&e=1696645146&fl=&r=be1d549f-fa42-4e4a-8b49-5f7b0db5d5f3-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=rj7VSPt9VNX4xcVg1AAeLPAzon4
806221 [2023-10-07T02:14:09.913Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-35360.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-35360.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
806222 [2023-10-07T02:14:09.913Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-35360.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-35360.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1008144 [2023-10-07T03:22:36.058Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=Am0XJVnFmyOCQVAvmcZMAsQ_GW-6uKZSmZrOdvyw6X7h&v=1&x=3&a=CAogx9TRzeNfaKP7e8-7nORtAZnutUGJWpbyIuTDEwvm5OoSaxDntti_sDEY55O0wbAxIgEAUgTQ6AqTWgTt6wTaaiXedIPjDLR6pEzF38HlZg_eke6F92SYO4b-BcoDfxxYnxByATE_ciWalRwbvxQP-Noj1DjTN6SYrpwh4GZ8u_Se5wNqVt_DI9si9PIX&e=1696649251&fl=&r=8e11f3e2-9796-411b-8773-3eac2ad47a03-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=Ua5iMJZNR35ML2zI4RZmzOJ0J8s
1008145 [2023-10-07T03:22:36.058Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=Am0XJVnFmyOCQVAvmcZMAsQ_GW-6uKZSmZrOdvyw6X7h&v=1&x=3&a=CAogx9TRzeNfaKP7e8-7nORtAZnutUGJWpbyIuTDEwvm5OoSaxDntti_sDEY55O0wbAxIgEAUgTQ6AqTWgTt6wTaaiXedIPjDLR6pEzF38HlZg_eke6F92SYO4b-BcoDfxxYnxByATE_ciWalRwbvxQP-Noj1DjTN6SYrpwh4GZ8u_Se5wNqVt_DI9si9PIX&e=1696649251&fl=&r=8e11f3e2-9796-411b-8773-3eac2ad47a03-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=Ua5iMJZNR35ML2zI4RZmzOJ0J8s
1008146: [2023-10-07T03:22:36.062Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-665359.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-665359.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1008147: [2023-10-07T03:22:36.062Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AnjBPW1JcT0KkA0HtXmHq9pFeoyq-HICLm0-kFy5wOFs&v=1&x=3&a=CAogN5wtCOlSefdsCADhlnnAkMUqkX9PoOVTl4tE3Ll9UakSaxCTt9i_sDEYk5S0wbAxIgEAUgQ9rfi-WgS9OF6NaiVJnqd7wTfb4nfRY1okvzC1xHoDW69T2Sz6pruekDgYjYPYjBlGciXj7k5bq6baQkRt3i9tKRSQD1nVyibPLvirC_9PVYIvRoIiGgdF&e=1696649251&fl=&r=8e11f3e2-9796-411b-8773-3eac2ad47a03-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=kFzURHMGW5SA8C5Ox4vYxzgZ2ps
1008148: [2023-10-07T03:22:36.484Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AnjBPW1JcT0KkA0HtXmHq9pFeoyq-HICLm0-kFy5wOFs&v=1&x=3&a=CAogN5wtCOlSefdsCADhlnnAkMUqkX9PoOVTl4tE3Ll9UakSaxCTt9i_sDEYk5S0wbAxIgEAUgQ9rfi-WgS9OF6NaiVJnqd7wTfb4nfRY1okvzC1xHoDW69T2Sz6pruekDgYjYPYjBlGciXj7k5bq6baQkRt3i9tKRSQD1nVyibPLvirC_9PVYIvRoIiGgdF&e=1696649251&fl=&r=8e11f3e2-9796-411b-8773-3eac2ad47a03-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=kFzURHMGW5SA8C5Ox4vYxzgZ2ps to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-665359.heic
1008148 [2023-10-07T03:22:36.517Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=AjLa8FX28hO0PzHncIi4XVI7gNyp3D3PsBCAh1ksj5Te&v=1&x=3&a=CAog3AsqS-koQ9T5GA93KtNnFmzWcIf5vCHW8EL73viJ9wESaxD2tti_sDEY9pO0wbAxIgEAUgSmkmy3WgREmuFuaiVw6H9TztO1cgVdJ9T6OUeRViNEevwagWkFHetyM9R1ZgSKiuYJciXW5dZzhJnK4_2Ij9Az-1MsygKKdfQM7yLG8rOm0DF2y-SfIyjd&e=1696649251&fl=&r=8e11f3e2-9796-411b-8773-3eac2ad47a03-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=2KoAbpZKsb5AQfHv_OZ3bJi7Acg to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-141970.mov
1008149 [2023-10-07T03:22:36.517Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=AjLa8FX28hO0PzHncIi4XVI7gNyp3D3PsBCAh1ksj5Te&v=1&x=3&a=CAog3AsqS-koQ9T5GA93KtNnFmzWcIf5vCHW8EL73viJ9wESaxD2tti_sDEY9pO0wbAxIgEAUgSmkmy3WgREmuFuaiVw6H9TztO1cgVdJ9T6OUeRViNEevwagWkFHetyM9R1ZgSKiuYJciXW5dZzhJnK4_2Ij9Az-1MsygKKdfQM7yLG8rOm0DF2y-SfIyjd&e=1696649251&fl=&r=8e11f3e2-9796-411b-8773-3eac2ad47a03-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=2KoAbpZKsb5AQfHv_OZ3bJi7Acg to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-141970.mov
1008150: [2023-10-07T03:22:37.215Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AnjBPW1JcT0KkA0HtXmHq9pFeoyq-HICLm0-kFy5wOFs&v=1&x=3&a=CAogN5wtCOlSefdsCADhlnnAkMUqkX9PoOVTl4tE3Ll9UakSaxCTt9i_sDEYk5S0wbAxIgEAUgQ9rfi-WgS9OF6NaiVJnqd7wTfb4nfRY1okvzC1xHoDW69T2Sz6pruekDgYjYPYjBlGciXj7k5bq6baQkRt3i9tKRSQD1nVyibPLvirC_9PVYIvRoIiGgdF&e=1696649251&fl=&r=8e11f3e2-9796-411b-8773-3eac2ad47a03-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=kFzURHMGW5SA8C5Ox4vYxzgZ2ps
1008150 [2023-10-07T03:22:37.222Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-200160.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-200160.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1008151 [2023-10-07T03:22:37.222Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-200160.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-200160.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1209904 [2023-10-07T04:31:29.796Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=Aj_qorT6ybmsBpEp6f7KpZqBcZcgxQh4H8Fzw3bDA6_J&v=1&x=3&a=CAogig_Xta2ouQ2rPnp7_T8dmG9KNcOyPkFG3YbcbUjHc64SaxDy-9TBsDEY8tiww7AxIgEAUgTQ6AqTWgTt6wTaaiVswoU09SUbh1NBQBAkUsPp4krTXFQvtRTyl-G2YIlcmXht5T1UciXuWTqjsDC0zGF2sM3HPl6fsrwONCb-j_lHVY2SEJ21uN_s5RZX&e=1696653388&fl=&r=c5e78d2e-2b0b-4f81-b248-005895e45672-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=vINAoU-_LyAqzQETSQerlF3qU3c
1209905 [2023-10-07T04:31:29.796Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=Aj_qorT6ybmsBpEp6f7KpZqBcZcgxQh4H8Fzw3bDA6_J&v=1&x=3&a=CAogig_Xta2ouQ2rPnp7_T8dmG9KNcOyPkFG3YbcbUjHc64SaxDy-9TBsDEY8tiww7AxIgEAUgTQ6AqTWgTt6wTaaiVswoU09SUbh1NBQBAkUsPp4krTXFQvtRTyl-G2YIlcmXht5T1UciXuWTqjsDC0zGF2sM3HPl6fsrwONCb-j_lHVY2SEJ21uN_s5RZX&e=1696653388&fl=&r=c5e78d2e-2b0b-4f81-b248-005895e45672-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=vINAoU-_LyAqzQETSQerlF3qU3c
1209906: [2023-10-07T04:31:29.803Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-957426.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-957426.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1209907: [2023-10-07T04:31:29.803Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=Ag6SX7EDZ0cuWOZtKSKCcuw3Nwoc_yZOOPi4_MXp-V3P&v=1&x=3&a=CAogWN9m6tH321CcC5U4fjkpT_NejD3f4lH8XeVpxRX-PAYSaxCR_NTBsDEYkdmww7AxIgEAUgQ9rfi-WgS9OF6NaiXsnrFbSY7fC16aOJUPHHekKbzb5DN9M9q5gXUEUEoLrkEsIqjnciXPuVx8cc1i_ja3LYq6Vh3L4diGLWGbEJNPQdERIX1YbcfdNpOT&e=1696653388&fl=&r=c5e78d2e-2b0b-4f81-b248-005895e45672-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=6CPhG2ULFoM9H4yz8A0jrOJB0rw
1209907 [2023-10-07T04:31:30.147Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=AigkmWxH6mmufZTuqAnvZXgOO6jJwdMia5XcC5clZ67K&v=1&x=3&a=CAogDo9N12PvBAZbX_C_ZT1f3iO0s4ivajz3PxbtRBaTyaASaxD8-9TBsDEY_Niww7AxIgEAUgSmkmy3WgREmuFuaiVsVvqaD-7x-VHOK8akI2GVnDLkK2kih1fuob56Kl3qr9A-t6VhciW9MfZIm54vX_23bhsJUnVpnmrV8UDGfRHuTCU-gPQdGoasztLN&e=1696653388&fl=&r=c5e78d2e-2b0b-4f81-b248-005895e45672-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=DP8yxkj_yTComqYK5FY2OUaKDJE to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-425077.mov
1209908 [2023-10-07T04:31:30.147Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=AigkmWxH6mmufZTuqAnvZXgOO6jJwdMia5XcC5clZ67K&v=1&x=3&a=CAogDo9N12PvBAZbX_C_ZT1f3iO0s4ivajz3PxbtRBaTyaASaxD8-9TBsDEY_Niww7AxIgEAUgSmkmy3WgREmuFuaiVsVvqaD-7x-VHOK8akI2GVnDLkK2kih1fuob56Kl3qr9A-t6VhciW9MfZIm54vX_23bhsJUnVpnmrV8UDGfRHuTCU-gPQdGoasztLN&e=1696653388&fl=&r=c5e78d2e-2b0b-4f81-b248-005895e45672-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=DP8yxkj_yTComqYK5FY2OUaKDJE to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-425077.mov
1209909: [2023-10-07T04:31:30.417Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=Ag6SX7EDZ0cuWOZtKSKCcuw3Nwoc_yZOOPi4_MXp-V3P&v=1&x=3&a=CAogWN9m6tH321CcC5U4fjkpT_NejD3f4lH8XeVpxRX-PAYSaxCR_NTBsDEYkdmww7AxIgEAUgQ9rfi-WgS9OF6NaiXsnrFbSY7fC16aOJUPHHekKbzb5DN9M9q5gXUEUEoLrkEsIqjnciXPuVx8cc1i_ja3LYq6Vh3L4diGLWGbEJNPQdERIX1YbcfdNpOT&e=1696653388&fl=&r=c5e78d2e-2b0b-4f81-b248-005895e45672-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=6CPhG2ULFoM9H4yz8A0jrOJB0rw to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-957426.heic
1209910: [2023-10-07T04:31:32.448Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=Ag6SX7EDZ0cuWOZtKSKCcuw3Nwoc_yZOOPi4_MXp-V3P&v=1&x=3&a=CAogWN9m6tH321CcC5U4fjkpT_NejD3f4lH8XeVpxRX-PAYSaxCR_NTBsDEYkdmww7AxIgEAUgQ9rfi-WgS9OF6NaiXsnrFbSY7fC16aOJUPHHekKbzb5DN9M9q5gXUEUEoLrkEsIqjnciXPuVx8cc1i_ja3LYq6Vh3L4diGLWGbEJNPQdERIX1YbcfdNpOT&e=1696653388&fl=&r=c5e78d2e-2b0b-4f81-b248-005895e45672-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=6CPhG2ULFoM9H4yz8A0jrOJB0rw
1209910 [2023-10-07T04:31:32.456Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-685695.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-685695.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1209911 [2023-10-07T04:31:32.456Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-685695.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-685695.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1411859 [2023-10-07T05:40:53.920Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AjnCSpKSaU4jIm6AIWuQ4nOC4QSMneeHf6h6pgAcQ6r5&v=1&x=3&a=CAogQ9Viqbr_uAzE18nGYfXvovX96V4XnHSv-adRHAkyHDkSaxCx7tDDsDEYscusxbAxIgEAUgTQ6AqTWgTt6wTaaiV15PBMWF_-5SpbI6QrkYM--17AeyxwRx4fAGC3tq8GuE2hZ3vVciXQUJwu23xmHPuPMFVuLErm8K0j_Xf33Kj1XcU_ttNViIigJdci&e=1696657515&fl=&r=c04d359c-e77c-43d7-b9bb-14cdd13c1fb4-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=_mcDd82l7KOr4mGqkLZj4UXpeuM
1411860 [2023-10-07T05:40:53.920Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AjnCSpKSaU4jIm6AIWuQ4nOC4QSMneeHf6h6pgAcQ6r5&v=1&x=3&a=CAogQ9Viqbr_uAzE18nGYfXvovX96V4XnHSv-adRHAkyHDkSaxCx7tDDsDEYscusxbAxIgEAUgTQ6AqTWgTt6wTaaiV15PBMWF_-5SpbI6QrkYM--17AeyxwRx4fAGC3tq8GuE2hZ3vVciXQUJwu23xmHPuPMFVuLErm8K0j_Xf33Kj1XcU_ttNViIigJdci&e=1696657515&fl=&r=c04d359c-e77c-43d7-b9bb-14cdd13c1fb4-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=_mcDd82l7KOr4mGqkLZj4UXpeuM
1411861: [2023-10-07T05:40:53.922Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-719637.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-719637.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1411862: [2023-10-07T05:40:53.922Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AjUBKWS0K8ubRxHQCs7J6k-RleAY4ShCdhmg3DfxxNzh&v=1&x=3&a=CAog9hn3NlCt0Tx6TemAqbyIqLxlPx0mzGkphlbtxH5Mu1wSaxDQ7tDDsDEY0MusxbAxIgEAUgQ9rfi-WgS9OF6NaiXNha-LakU2K_xFa3ca7Bw2ztNAo-kU-KlHvVLgM6_bM0YQLEgNciU4oqHPxSnbeo4iAXJt0da8hqoJa5gchYYYxHjR6HMKH9sh8Ynd&e=1696657515&fl=&r=c04d359c-e77c-43d7-b9bb-14cdd13c1fb4-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=fIm8kCXUUEgun0Xe7u7CRsCJR8s
1411863: [2023-10-07T05:40:54.332Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AjUBKWS0K8ubRxHQCs7J6k-RleAY4ShCdhmg3DfxxNzh&v=1&x=3&a=CAog9hn3NlCt0Tx6TemAqbyIqLxlPx0mzGkphlbtxH5Mu1wSaxDQ7tDDsDEY0MusxbAxIgEAUgQ9rfi-WgS9OF6NaiXNha-LakU2K_xFa3ca7Bw2ztNAo-kU-KlHvVLgM6_bM0YQLEgNciU4oqHPxSnbeo4iAXJt0da8hqoJa5gchYYYxHjR6HMKH9sh8Ynd&e=1696657515&fl=&r=c04d359c-e77c-43d7-b9bb-14cdd13c1fb4-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=fIm8kCXUUEgun0Xe7u7CRsCJR8s to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-719637.heic
1411864: [2023-10-07T05:40:54.931Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AjUBKWS0K8ubRxHQCs7J6k-RleAY4ShCdhmg3DfxxNzh&v=1&x=3&a=CAog9hn3NlCt0Tx6TemAqbyIqLxlPx0mzGkphlbtxH5Mu1wSaxDQ7tDDsDEY0MusxbAxIgEAUgQ9rfi-WgS9OF6NaiXNha-LakU2K_xFa3ca7Bw2ztNAo-kU-KlHvVLgM6_bM0YQLEgNciU4oqHPxSnbeo4iAXJt0da8hqoJa5gchYYYxHjR6HMKH9sh8Ynd&e=1696657515&fl=&r=c04d359c-e77c-43d7-b9bb-14cdd13c1fb4-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=fIm8kCXUUEgun0Xe7u7CRsCJR8s
1411864 [2023-10-07T05:40:54.939Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-350150.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-350150.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1411865 [2023-10-07T05:40:54.939Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-350150.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-350150.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1613474 [2023-10-07T06:49:53.150Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AmG9EiV6UOjHIlOG4KQYfrAVCr8WIiBjSzdVCvHmCkyJ&v=1&x=3&a=CAogioM6WWWP25ddANjrTeKT1UrNRJ57lj92TxNBIOkEaOQSaxCooc_FsDEYqP6qx7AxIgEAUgTQ6AqTWgTt6wTaaiXdDNS49jIY93WIMiX-HaWkqS664EbOYrxeYrBtliRMgfW07zr8ciViEs1JnKdMBYR4oBsLocRvxf1s9D78FCJTpxiFjEud5U0NsSvC&e=1696661684&fl=&r=d54c382a-5bb5-433e-a62e-c0e6778f8df7-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=G8xr2kPFJ4Ij1-fVGlVgPE0cmf8
1613475 [2023-10-07T06:49:53.150Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Ac_ngH2YWYZ_xI_lYwSyGknQ6AqTAch_3To-MHigmis3w5XE-NPt6wTa/${f}?o=AmG9EiV6UOjHIlOG4KQYfrAVCr8WIiBjSzdVCvHmCkyJ&v=1&x=3&a=CAogioM6WWWP25ddANjrTeKT1UrNRJ57lj92TxNBIOkEaOQSaxCooc_FsDEYqP6qx7AxIgEAUgTQ6AqTWgTt6wTaaiXdDNS49jIY93WIMiX-HaWkqS664EbOYrxeYrBtliRMgfW07zr8ciViEs1JnKdMBYR4oBsLocRvxf1s9D78FCJTpxiFjEud5U0NsSvC&e=1696661684&fl=&r=d54c382a-5bb5-433e-a62e-c0e6778f8df7-1&k=BR8ttrz7bHEHuvyO_4jKgQ&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=G8xr2kPFJ4Ij1-fVGlVgPE0cmf8
1613476: [2023-10-07T06:49:53.155Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-80254.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-80254.heic and linking original file Aa_fDRoooKJJY-0nHI30XCM9rfi--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1613477: [2023-10-07T06:49:53.155Z] DEBUG NetworkManager: Starting download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AkZRQdL0g8zBl0Et_8TBYiOlixe7_fbHf67vUnKazhrz&v=1&x=3&a=CAogLOM0-GJEygZWm-mcyLoshfnm6Cjne8xthD4jOtZ2suMSaxDKoc_FsDEYyv6qx7AxIgEAUgQ9rfi-WgS9OF6NaiWyATAJMGzbABkx68_Aj3Ae1mXA-8XmD4ttoBLwFluYBlhZis9ZciXJmX4x6K-NvzxUTZokcAITpwpuhu_KIJgohA5ZcIFl7Ov7-7Uk&e=1696661684&fl=&r=d54c382a-5bb5-433e-a62e-c0e6778f8df7-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=jmbFgI__I-E5_mnYEEuUpnXk53E
1613477 [2023-10-07T06:49:53.545Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=Auk8petY0mO6II1laczBetIXe9weLCdbkUcqXHqBdJ1X&v=1&x=3&a=CAogcB_UIVsqZ0VHmjX6YEuIdL80660oaiUknfsUwH0Mf8MSaxCzoc_FsDEYs_6qx7AxIgEAUgSmkmy3WgREmuFuaiWBr8XL24vve84WYXzsP81EVxjc_c8LGXOk7Ok74f1ezsjh1pagciXSUO-eaLzmUmfd6uZBWZfI-SS9IPaFrEf8tkXuJq7smwQrYF8N&e=1696661684&fl=&r=d54c382a-5bb5-433e-a62e-c0e6778f8df7-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=sR3yR6EOYTSgKr9iAQQXWxTlk9s to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-305011.mov
1613478 [2023-10-07T06:49:53.545Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/AZZWSm6gsVdoSmzLK2OINh6mkmy3AVp_SHXYXgYqepvku1EjoLxEmuFu/${f}?o=Auk8petY0mO6II1laczBetIXe9weLCdbkUcqXHqBdJ1X&v=1&x=3&a=CAogcB_UIVsqZ0VHmjX6YEuIdL80660oaiUknfsUwH0Mf8MSaxCzoc_FsDEYs_6qx7AxIgEAUgSmkmy3WgREmuFuaiWBr8XL24vve84WYXzsP81EVxjc_c8LGXOk7Ok74f1ezsjh1pagciXSUO-eaLzmUmfd6uZBWZfI-SS9IPaFrEf8tkXuJq7smwQrYF8N&e=1696661684&fl=&r=d54c382a-5bb5-433e-a62e-c0e6778f8df7-1&k=EtBJ5EAWb4kWKl-RiKhb3g&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=sR3yR6EOYTSgKr9iAQQXWxTlk9s to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AZZWSm6gsVdoSmzLK2OINh6mkmy3-dup-305011.mov
1613479: [2023-10-07T06:49:53.584Z] DEBUG NetworkManager: Starting to write https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AkZRQdL0g8zBl0Et_8TBYiOlixe7_fbHf67vUnKazhrz&v=1&x=3&a=CAogLOM0-GJEygZWm-mcyLoshfnm6Cjne8xthD4jOtZ2suMSaxDKoc_FsDEYyv6qx7AxIgEAUgQ9rfi-WgS9OF6NaiWyATAJMGzbABkx68_Aj3Ae1mXA-8XmD4ttoBLwFluYBlhZis9ZciXJmX4x6K-NvzxUTZokcAITpwpuhu_KIJgohA5ZcIFl7Ov7-7Uk&e=1696661684&fl=&r=d54c382a-5bb5-433e-a62e-c0e6778f8df7-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=jmbFgI__I-E5_mnYEEuUpnXk53E to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/Aa_fDRoooKJJY-0nHI30XCM9rfi--dup-80254.heic
1613480: [2023-10-07T06:49:54.394Z] DEBUG NetworkManager: Finished download of https://cvws.icloud-content.com/B/Aa_fDRoooKJJY-0nHI30XCM9rfi-AeHznfCvqP0c10kik4I4KI69OF6N/${f}?o=AkZRQdL0g8zBl0Et_8TBYiOlixe7_fbHf67vUnKazhrz&v=1&x=3&a=CAogLOM0-GJEygZWm-mcyLoshfnm6Cjne8xthD4jOtZ2suMSaxDKoc_FsDEYyv6qx7AxIgEAUgQ9rfi-WgS9OF6NaiWyATAJMGzbABkx68_Aj3Ae1mXA-8XmD4ttoBLwFluYBlhZis9ZciXJmX4x6K-NvzxUTZokcAITpwpuhu_KIJgohA5ZcIFl7Ov7-7Uk&e=1696661684&fl=&r=d54c382a-5bb5-433e-a62e-c0e6778f8df7-1&k=ImcQTbItcwDlrjnLY3QjLw&ckc=com.apple.photos.cloud&ckz=PrimarySync&y=1&p=58&s=jmbFgI__I-E5_mnYEEuUpnXk53E
1613480 [2023-10-07T06:49:54.402Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-108783.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-108783.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
1613481 [2023-10-07T06:49:54.402Z] WARN NetworkManager: /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-108783.heic already exists - saving duplicate to /Volumes/Extreme SSD/icloud-photos-sync/duplicates/AUJ0QTGAj6TeCZ-NTHIICD-ne4c--dup-108783.heic and linking original file AUJ0QTGAj6TeCZ-NTHIICD-ne4c--orig.heic to /Volumes/Extreme SSD/icloud-photos-sync/duplicates
Let me know if you need any additional information.
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
I think I also made a wrong assumption how chaining promises work - should be fixed now as well.
Is there any other reference to the file in your logs?
Could you also please share how the 'duplicates' folder looks like (ls -al). Are all duplicate files the exact same? maybe you can use diff to compare them down to the byte level?
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
Is there any other reference to the file in your logs?
No, these are all the occurrences.
Could you also please share how the 'duplicates' folder looks like (ls -al).
Here is the output of ls -la: https://gist.github.com/halbtuerke/ca17c2d2a1915eb951363e8a0712ce70
Are all duplicate files the exact same? maybe you can use diff to compare them down to the byte level?
As requested I have compared the files on a byte level. For that I wrote the following Python script, which determines all the unique IDs and then compares all duplicates of the individual IDs. The result is that almost all of those duplicates are completely the same.
#!/usr/bin/env python3
import os
from collections import defaultdict
import filecmp
def compare_files_by_prefix(directory_path):
# Initialize a dictionary to hold file prefixes and corresponding file names
files_by_prefix = defaultdict(list)
# Iterate through the directory to populate the files_by_prefix dictionary
for file_name in os.listdir(directory_path):
prefix = file_name.split('-orig')[0].split('-dup')[0]
files_by_prefix[prefix].append(os.path.join(directory_path, file_name))
# Initialize a dictionary to hold comparison results
comparison_results = {}
# Compare files by prefix
for prefix, file_list in files_by_prefix.items():
print(prefix)
# Skip if there's only one file with this prefix
if len(file_list) <= 1:
comparison_results[prefix] = "Only one file with this prefix."
continue
# Compare the files on a byte level
are_identical = all(filecmp.cmp(file_list[0], other_file, shallow=False) for other_file in file_list[1:])
comparison_results[prefix] = "Identical" if are_identical else "Different"
return comparison_results
def print_comparison_results(results):
for prefix, status in results.items():
print(f"Files with prefix '{prefix}': {status}")
# For demonstration purposes, let's assume a directory path.
# Note: This is a placeholder, and the actual directory path should be provided.
directory_path = "/Volumes/Extreme SSD/icloud-photos-sync/duplicates"
# Get the comparison results
results = compare_files_by_prefix(directory_path)
# Print the results
print_comparison_results(results)There are only 2 cases where there are actual differences, but only with single duplicates:
Files with prefix 'AY5TDE3mVEXwtU1Oyo2RYM1xV07g': Different
Files with prefix 'AW0tYkIVqOpECWEBGzW4hZELQRGV': Different
Files with prefix 'AcfheUEZsdw7922tT4y2C7y6D4T0': Only one file with this prefix.
Files with prefix 'AUfNRVZbCaJmzXgX_5U4CBX7cQTG': Only one file with this prefix.
In both cases it seems like those files are broken and cannot be opened.
So in general it seems like we are mostly downloading legit duplicates.
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
Okay - so these are legit duplicates - and I have no idea why the diffing algorithm does not catch them.
In 1.3.0-nightly.10 I've reverted the debug logic and it should now skip already existing files. This should unblock your sync executions and I will need to address this properly with the planned improvements of the epic in #354.
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
@halbtuerke did you succeed syncing your assets with the workaround introduced in the latest version?
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
Yes, the initial synchronization was successful. However, I encounter issues when attempting to re-synchronize for updating photos. Below are the error messages displayed:
Sign in successful!
iCloud Photos setup completed, checking indexing status...
iCloud Photos ready!
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Loading local & fetching remote iCloud Library state...
Detected error during sync: SYNC_UNKNOWN: Unknown error during sync caused by ICLOUD_PHOTOS_FETCH_RECORDS: Unable to fetch records (album 'All photos') caused by Request failed with status code 408
Refreshing iCloud connection & retrying (attempt #35)...
I am using version 1.3.0.
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
Is this the 'full' log? Meaning: does it go straight to attempt 35?
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
No, there were 34 previous attempts. I just terminate after 35 attempts. 🥲
from icloud-photos-sync.
steilerDev
commented on June 23, 2024
And what's the timing of those 408's? could you set retryCount to 1, enable debugging logs, re-run and share the logs and output here?
from icloud-photos-sync.
halbtuerke
commented on June 23, 2024
Command used:
FORCE=true LOG_LEVEL=debug icloud-photos-sync \
-u USER \
-p PASSWORD \
-d "/Volumes/Extreme SSD/icloud-photos-sync" \
--enable-crash-reporting \
--metadata-rate '1/20' \
--max-retries 1 \
--export-metrics \
sync;
Output of the run (started at 7:05 PM):
Welcome to icloud-photos-sync, v.1.3.0!
Made with <3 by steilerDev
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Authenticating user...
Device trusted
Sign in successful!
iCloud Photos setup completed, checking indexing status...
iCloud Photos ready!
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Starting sync at 10/30/2023, 7:07:40 PM
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Loading local & fetching remote iCloud Library state...
Detected error during sync: SYNC_UNKNOWN: Unknown error during sync caused by ICLOUD_PHOTOS_FETCH_RECORDS: Unable to fetch records (album 'All photos') caused by Request failed with status code 408
Refreshing iCloud connection & retrying (attempt #2)...
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sign in successful!
iCloud Photos setup completed, checking indexing status...
iCloud Photos ready!
Error: APP_SYNC: Sync failed caused by SYNC_MAX_RETRY: Sync did not complete successfully within expected amount of tries (2) (error code: 3bc11b53-5f84-44e3-81b1-a7ce4178ddfc)
You can find the logs at https://gist.github.com/halbtuerke/406cced473934ba7684c1d7bfe4269e1
from icloud-photos-sync.
Related Issues (20)
- Unexpected error while setting up iCloud Photos caused by VALIDATOR_PHOTOS_SETUP_RESPONSE HOT 4
- TypeError: Converting circular structure to JSON HOT 16
- Provided Grafana Dashboard example does not show any data HOT 12
- Feature Request: Remove password from environment HOT 7
- LIBRARY_ASSET_MODIFICATION_TIME: File's modification time does not match iCloud record HOT 4
- Handle EACCES MFA Server Error HOT 2
- Handle 'incorrect MFA code' scenario HOT 2
- Library Version 2 HOT 2
- Implementing new iCloud authentication process HOT 7
- Use Continuation Marker instead of start rank
- Hard to read logs on Synology HOT 30
- PID locked when attempting sync HOT 10
- [83730a] icloud-photos-sync: iCloudAuthError: Unable to acquire PCS cookies
- [b63792] icloud-photos-sync: SyncError: Sync did not complete successfully within expected amount of tries
- [9b978c] icloud-photos-sync: iCloudPhotosError: Unexpected error while setting up iCloud Photos
- Too many files - how to analyze? HOT 1
- token command not working properly? HOT 5
- exec format error(aarch64) rassberry pi 5 HOT 2
- Reintroduce upstream axios-har-tracker
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from icloud-photos-sync.