Symlinks not preserved about rome

[tmspzz]

Ok, small update:

• I haven't forgot this.
• this is an issue with the zip library
• I'm trying to patch the zip library (you would not believe how messed up zip is)
• if I don't succeed I will just used zip form the command line

[chrispcampbell]

Thanks for the fix, I will try it out today and will let you know!

[chrispcampbell]

@blender I tried v0.14.0.38 and can confirm that everything appears to be working correctly now.
I tried upload, download, and manual unzip extraction and symlinks were created correctly without permission issues. Thank you!

[tmspzz]

Thank you for the patience! I'll make remove the draft tag from the release and update brew too.

[tmspzz]

Sounds like a bug. I'll try to fix it asap.

[tmspzz]

@chrispcampbell Please try pre-release https://github.com/blender/Rome/releases/tag/v0.14.0.36

I haven't tried to build but the contents of the archive seem to preserve symlinks

[tmspzz]

@chrispcampbell any feedback on this?

[chrispcampbell]

@blender I tested with v0.14.0.36. The good news is that it appears to partially preserve symlinks when doing a rome upload to the local Rome cache. Like, after a rome upload I can unzip a zip file (in Finder) in the Rome cache and see that the unzipped contents contain symlinks. So, that's a start.

However, when I clear out my Carthage/Build directory and then do a rome download for that framework, it gives an error. First, this is what it looks like as built by Carthage:

% % ls -lR Carthage/Build/Mac/CocoaLumberjack.framework
total 32
lrwxr-xr-x  1 campbell  staff   32 Jan 16 17:03 CocoaLumberjack -> Versions/Current/CocoaLumberjack
lrwxr-xr-x  1 campbell  staff   24 Jan 16 17:03 Headers -> Versions/Current/Headers
lrwxr-xr-x  1 campbell  staff   24 Jan 16 17:03 Modules -> Versions/Current/Modules
lrwxr-xr-x  1 campbell  staff   26 Jan 16 17:03 Resources -> Versions/Current/Resources
drwxr-xr-x  4 campbell  staff  136 Jan 16 17:03 Versions

Carthage/Build/Mac/CocoaLumberjack.framework/Versions:
total 8
drwxr-xr-x  6 campbell  staff  204 Jan 16 17:03 A
lrwxr-xr-x  1 campbell  staff    1 Jan 16 17:03 Current -> A

Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A:
total 504
-rwxr-xr-x   1 campbell  staff  255500 Jan 16 17:03 CocoaLumberjack
drwxr-xr-x  18 campbell  staff     612 Jan 16 17:03 Headers
drwxr-xr-x   3 campbell  staff     102 Jan 16 17:03 Modules
drwxr-xr-x   3 campbell  staff     102 Jan 16 17:03 Resources

Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Headers:
total 248
<stuff>

Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Modules:
total 8
-rw-r--r--  1 campbell  staff  679 Jan 16 17:03 module.modulemap

Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Resources:
total 8
-rw-r--r--  1 campbell  staff  1249 Jan 16 17:03 Info.plist

Then, testing the upload to an empty Rome cache:

% ls -la ~/Library/Caches/Rome
ls: /Users/campbell/Library/Caches/Rome: No such file or directory

% ~/Downloads/rome --version
0.14.0.36 - Romam uno die non fuisse conditam.

% ~/Downloads/rome upload CocoaLumberjack
Copied CocoaLumberjack.framework to: /Users/campbell/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjack.framework-3.3.0.zip
Copied CocoaLumberjack.dSYM to: /Users/campbell/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjack.framework.dSYM-3.3.0.zip
Copied CocoaLumberjackSwift.framework to: /Users/campbell/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjackSwift.framework-3.3.0.zip
Copied CocoaLumberjackSwift.dSYM to: /Users/campbell/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjackSwift.framework.dSYM-3.3.0.zip
Copied .CocoaLumberjack.version to: /Users/campbell/Library/Caches/Rome/CocoaLumberjack/.CocoaLumberjack.version-3.3.0

Next, I moved the Carthage/Build directory aside and attempted a rome download, which gives the error:

% mv Carthage/Build Carthage/Build-Old

% ~/Downloads/rome download CocoaLumberjack
Found CocoaLumberjack in local cache at: /Users/campbell/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjack.framework-3.3.0.zip
rome: CRC32Mismatch "Carthage/Build/Mac/CocoaLumberjack.framework/CocoaLumberjack"

It's worth noting that I see similar CRC-related messages when I try to unzip the thing from the command line:

% unzip ~/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjack.framework-3.3.0.zip -d /tmp/rometest
Archive:  /Users/campbell/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjack.framework-3.3.0.zip
   creating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/
    linking: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/CocoaLumberjack  -> Versions/Current/CocoaLumberjack  bad CRC 0933b4ab  (should be 29929617)
    linking: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Headers  -> Versions/Current/Headers  bad CRC 8ead94f3  (should be 00000000)
    linking: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Modules  -> Versions/Current/Modules  bad CRC 571998a1  (should be 00000000)
    linking: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Resources  -> Versions/Current/Resources  bad CRC 8702472c  (should be 00000000)
   creating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/
   creating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/
  inflating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/CocoaLumberjack  
   creating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Headers/
  inflating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Headers/CLIColor.h  
  inflating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Headers/CocoaLumberjack.h  
<more headers...>
   creating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Modules/
  inflating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Modules/module.modulemap  
   creating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Resources/
  inflating: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Resources/Info.plist  
    linking: /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/Current  -> A  bad CRC d3d99e8b  (should be 00000000)
finishing deferred symbolic links:
  /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/CocoaLumberjack -> Versions/Current/CocoaLumberjack
  /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Headers -> Versions/Current/Headers
  /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Modules -> Versions/Current/Modules
  /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Resources -> Versions/Current/Resources
  /tmp/rometest/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/Current -> A

[tmspzz]

I think I know why. Thanks for testing. Back on it.

[tmspzz]

@chrispcampbell

please try https://github.com/blender/Rome/releases/tag/v0.14.0.37 I have fixed CRC32 errros and I see no issue with the upload/download flow. Command line unzipping is clean too.

[chrispcampbell]

@blender I just tried v0.14.0.37 and can confirm that the CRC32 errors are fixed. Thanks again for the fixes! Actually I might have been using the wrong binary; still seeing some issues. Will add another comment once I can confirm.

[chrispcampbell]

@blender I just tried again and can confirm that rome upload does seem to preserve the symlinks (unzipping the zip file from the Rome cache shows correct symlinks with no CRC errors), but now the rome download step is back to not creating the symlinks when the thing gets unzipped into Carthage/Build. Any ideas?

[chrispcampbell]

@blender Here's the output of the unzip (for a cached zip file that was created with v0.14.0.37):

% unzip ~/Library/Caches/Rome/CocoaLumberjack/Mac/CocoaLumberjack.framework-3.3.0.zip -d /tmp/rometest4
Archive:  /Users/campbell/Library/Caches/Rome/Pup/CocoaLumberjack/Mac/CocoaLumberjack.framework-3.3.0.zip
   creating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/
    linking: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/CocoaLumberjack  -> Versions/Current/CocoaLumberjack 
    linking: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Headers  -> Versions/Current/Headers 
    linking: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Modules  -> Versions/Current/Modules 
    linking: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Resources  -> Versions/Current/Resources 
   creating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/
   creating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/
  inflating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/CocoaLumberjack  
   creating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Headers/
  inflating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Headers/CLIColor.h  
<more headers...>
   creating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Modules/
  inflating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Modules/module.modulemap  
   creating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Resources/
  inflating: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Resources/Info.plist  
    linking: /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/Current  -> A 
finishing deferred symbolic links:
  /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/CocoaLumberjack -> Versions/Current/CocoaLumberjack
  /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Headers -> Versions/Current/Headers
  /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Modules -> Versions/Current/Modules
  /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Resources -> Versions/Current/Resources
  /tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/Current -> A

And a listing of the unzipped content:

% ls -lR /tmp/rometest4
total 0
drwxr-xr-x  3 campbell  wheel  102 Jan 17 09:49 Carthage

/tmp/rometest4/Carthage:
total 0
drwxr-xr-x  3 campbell  wheel  102 Jan 17 09:49 Build

/tmp/rometest4/Carthage/Build:
total 0
drwxr-xr-x  3 campbell  wheel  102 Jan 17 09:49 Mac

/tmp/rometest4/Carthage/Build/Mac:
total 0
drwxr-xr-x  7 campbell  wheel  238 Jan 17  2018 CocoaLumberjack.framework

/tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework:
total 32

ls: CocoaLumberjack.framework/CocoaLumberjack: Permission denied
l---------  1 campbell  wheel   32 Jan 17 09:49 CocoaLumberjack

ls: CocoaLumberjack.framework/Headers: Permission denied
l---------  1 campbell  wheel   24 Jan 17 09:49 Headers

ls: CocoaLumberjack.framework/Modules: Permission denied
l---------  1 campbell  wheel   24 Jan 17 09:49 Modules

ls: CocoaLumberjack.framework/Resources: Permission denied
l---------  1 campbell  wheel   26 Jan 17 09:49 Resources
drwxr-xr-x  4 campbell  wheel  136 Jan 17  2018 Versions

/tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions:
total 8
drwxr-xr-x  6 campbell  wheel  204 Jan 17  2018 A

ls: Versions/Current: Permission denied
l---------  1 campbell  wheel    1 Jan 17 09:49 Current

/tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A:
total 504
-rwxr-xr-x   1 campbell  wheel  255500 Jan 17  2018 CocoaLumberjack
drwxr-xr-x  18 campbell  wheel     612 Jan 17  2018 Headers
drwxr-xr-x   3 campbell  wheel     102 Jan 17  2018 Modules
drwxr-xr-x   3 campbell  wheel     102 Jan 17  2018 Resources

/tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Headers:
total 248
-rw-r--r--  1 campbell  wheel   1609 Jan 17  2018 CLIColor.h
<more headers...>

/tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Modules:
total 8
-rw-r--r--  1 campbell  wheel  679 Jan 17  2018 module.modulemap

/tmp/rometest4/Carthage/Build/Mac/CocoaLumberjack.framework/Versions/A/Resources:
total 8
-rw-r--r--  1 campbell  wheel  1249 Jan 17  2018 Info.plist

Looks like it could be a permissions issue?

[tmspzz]

Strange. I'll investigate.

[chrispcampbell]

@blender Have you had a chance to look into the permissions issue?

[tmspzz]

Yes, I have addressed it in the zip library jgm/zip-archive#39. Will make a build asap.

[tmspzz]

@chrispcampbell here you go https://github.com/blender/Rome/releases/tag/v0.14.0.38

I have tested:

• upload, extraction via unzip and have seen no issue from CLI
• download, permission and symlinks are preserved

Let me know if it works for you as well and I will make a full release