tmspzz / rome Goto Github PK
View Code? Open in Web Editor NEWCarthage cache for S3, Minio, Ceph, Google Storage, Artifactory and many others
License: MIT License
Carthage cache for S3, Minio, Ceph, Google Storage, Artifactory and many others
License: MIT License
Main use case: pulling in the cache from CI, seeing there isnt a rome cache, and then moving onto running `carthage update` followed by `rome upload`.
**Current and suggested behavior**
Currently it outputs the "could not find .frameworkname.version in local cache at: CACHE_DIR but with a 0 exit code.
**Why would the enhancement be useful to most users**
Automatically cache and upload if the cache isn't present
**Rome version:** 0.11.27
**OS and version:** Sierra
Speed up list
command execution
Steps which explain the enhancement
Probe caches in parallel for the presence of key.
Current and suggested behavior
Currently listing in done sequentially, parallelising execution would improve user experience
Why would the enhancement be useful to most users
Less waiting, more fun :)
Set up CI for Rome
If there are empty lines in Cartfile.resolved
like so
github "Alamofire/Alamofire" "3.5.1"
github "JohnEstropia/GCDKit" "1.2.5"
github "swisspol/GCDWebServer" "3.3.3"
github "typelift/Swiftz" "v0.5.1"
github "JohnEstropia/CoreStore" "1.6.10"
github "typelift/Tyro" "v0.0.8"
CoreStore and Tyro are ignored.
Rome version: 0.7.1.13
OS: macOS 10.11.6 El Capitan
CoreStore and Tyro not are ignored.
Some repositories use Carthage as a simple mechanism to include other git repositories that do not produce frameworks. Even Carthage itself does this, to include xcconfigs.
To accommodate for this existing paradigm, Rome should allow for certain repositories to be omitted entirely from the upload/download/list commands.
One could imagine that this would be an addition to the Romefile
, along the same lines as the existing RepositoryMap
section. Perhaps it could be named something along the lines of Ignored
.
As a hypothetical example, let's imagine the following:
Cartfile:
github "Quick/Nimble"
github "jspahrsummers/xcconfigs"
Romefile:
[Ignored]
xcconfigs
We then perform the following commands:
$ rome upload
Successfully uploaded Nimble to: Nimble/Nimble.framework-5.0.0.zip
Successfully uploaded Nimble to: Nimble/Nimble.framework.dSYM-5.0.0.zip
$ rome list
Nimble 5.0.0 ✔︎
$ rome download
Donwloaded: Nimble.framework-5.0.0.zip
Unzipped: Nimble.framework-5.0.0.zip
Donwloaded: Nimble.framework.dSYM-5.0.0.zip
Unzipped: Nimble.framework.dSYM-5.0.0.zip
Note that xcconfigs
was never included in any of the hypothetical output.
Currently, in scenarios like these, users have to manually filter out repositories that fall into this category, since otherwise they will always show up when invoking rome list --missing
.
While less of a hassle, this also pollutes the output of the download command with errors that will never be resolved:
$ rome download
...
Error downloading xcconfigs.framework-cc451b08e052b6146f5caf66bc1120420c529c7b.zip : The specified key does not exist.
Error downloading xcconfigs.framework.dSYM-cc451b08e052b6146f5caf66bc1120420c529c7b.zip : The specified key does not exist.
This is my app: http://github.com/palleas/memoires
I run the following commands make update
to run carthage
and upload on S3 with rome
. This is the output:
carthage update --no-use-binaries --platform ios
*** Fetching Tentacle
*** Fetching Nimble
*** Fetching keychain-swift
*** Fetching Quick
*** Fetching ReactiveSwift
*** Fetching Argo
*** Fetching Curry
*** Fetching Result
*** Fetching Runes
*** Checking out Curry at "v3.0.0"
*** Checking out keychain-swift at "7.0.1"
*** Checking out Nimble at "v6.1.0"
*** Checking out Quick at "v1.1.0"
*** Checking out Result at "3.2.1"
*** Checking out ReactiveSwift at "1.1.1"
*** Checking out ReactiveCocoa at "b27c62bf8743a6644219a862f2c51c4334f225d6"
*** Checking out Runes at "v4.0.1"
*** Checking out Argo at "v4.1.2"
*** Checking out Tentacle at "2141159b750a0cd170264445da01eb56f1b5a38c"
*** Checking out YamlSwift at "3.3.1"
*** xcodebuild output can be found in /var/folders/96/1g222jyn23v3d1jllp969gvh0000gn/T/carthage-xcodebuild.VOxyAu.log
*** Building scheme "Curry-iOS" in Curry.xcodeproj
*** Building scheme "KeychainSwift" in KeychainSwift.xcodeproj
*** Building scheme "Nimble-iOS" in Nimble.xcodeproj
*** Building scheme "Quick-iOS" in Quick.xcworkspace
*** Building scheme "Result-iOS" in Result.xcodeproj
*** Building scheme "ReactiveSwift-iOS" in ReactiveSwift.xcworkspace
*** Building scheme "ReactiveCocoa-iOS" in ReactiveCocoa.xcworkspace
*** Building scheme "Runes-iOS" in Runes.xcodeproj
*** Building scheme "Argo-iOS" in Argo.xcworkspace
*** Building scheme "Tentacle-iOS" in Tentacle.xcworkspace
*** Building scheme "Yaml iOS" in Yaml.xcodeproj
rome upload --platform ios
Uploaded Nimble to: Nimble/iOS/Nimble.framework-v6.1.0.zip
Uploaded Nimble.dSYM to: Nimble/iOS/Nimble.framework.dSYM-v6.1.0.zip
Uploaded Quick to: Quick/iOS/Quick.framework-v1.1.0.zip
Uploaded Quick.dSYM to: Quick/iOS/Quick.framework.dSYM-v1.1.0.zip
Uploaded ReactiveCocoa to: ReactiveCocoa/iOS/ReactiveCocoa.framework-b27c62bf8743a6644219a862f2c51c4334f225d6.zip
Uploaded ReactiveCocoa.dSYM to: ReactiveCocoa/iOS/ReactiveCocoa.framework.dSYM-b27c62bf8743a6644219a862f2c51c4334f225d6.zip
Uploaded ReactiveSwift to: ReactiveSwift/iOS/ReactiveSwift.framework-1.1.1.zip
Uploaded ReactiveSwift.dSYM to: ReactiveSwift/iOS/ReactiveSwift.framework.dSYM-1.1.1.zip
Uploaded Result to: Result/iOS/Result.framework-3.2.1.zip
Uploaded Result.dSYM to: Result/iOS/Result.framework.dSYM-3.2.1.zip
Uploaded Yaml to: YamlSwift/iOS/Yaml.framework-3.3.1.zip
Uploaded Yaml.dSYM to: YamlSwift/iOS/Yaml.framework.dSYM-3.3.1.zip
Uploaded KeychainSwift to: keychain-swift/iOS/KeychainSwift.framework-7.0.1.zip
Uploaded KeychainSwift.dSYM to: keychain-swift/iOS/KeychainSwift.framework.dSYM-7.0.1.zip
Uploaded Tentacle to: Tentacle/iOS/Tentacle.framework-2141159b750a0cd170264445da01eb56f1b5a38c.zip
Uploaded Tentacle.dSYM to: Tentacle/iOS/Tentacle.framework.dSYM-2141159b750a0cd170264445da01eb56f1b5a38c.zip
Uploaded Argo to: Argo/iOS/Argo.framework-v4.1.2.zip
Uploaded Argo.dSYM to: Argo/iOS/Argo.framework.dSYM-v4.1.2.zip
Uploaded Curry to: Curry/iOS/Curry.framework-v3.0.0.zip
Uploaded Curry.dSYM to: Curry/iOS/Curry.framework.dSYM-v3.0.0.zip
Uploaded Runes to: Runes/iOS/Runes.framework-v4.0.1.zip
Uploaded Runes.dSYM to: Runes/iOS/Runes.framework.dSYM-v4.0.1.zip
Uploaded .Nimble.version to: Nimble/.Nimble.version-v6.1.0
Uploaded .Quick.version to: Quick/.Quick.version-v1.1.0
Uploaded .ReactiveCocoa.version to: ReactiveCocoa/.ReactiveCocoa.version-b27c62bf8743a6644219a862f2c51c4334f225d6
Uploaded .ReactiveSwift.version to: ReactiveSwift/.ReactiveSwift.version-1.1.1
Uploaded .Result.version to: Result/.Result.version-3.2.1
Uploaded .YamlSwift.version to: YamlSwift/.YamlSwift.version-3.3.1
Uploaded .keychain-swift.version to: keychain-swift/.keychain-swift.version-7.0.1
Uploaded .Tentacle.version to: Tentacle/.Tentacle.version-2141159b750a0cd170264445da01eb56f1b5a38c
Uploaded .Argo.version to: Argo/.Argo.version-v4.1.2
Uploaded .Curry.version to: Curry/.Curry.version-v3.0.0
Uploaded .Runes.version to: Runes/.Runes.version-v4.0.1
After that, the result of rome list --platform ios --missing
is empty so everything worked and I can run my project without issue.
Now if I remove my Carthage
folder and run make bootstrap
(which runs carthage update
and rome download
), I get the following (quite similar) output:
*** Fetching Tentacle
*** Fetching Nimble
*** Fetching keychain-swift
*** Fetching Quick
*** Fetching ReactiveSwift
*** Fetching Argo
*** Fetching Curry
*** Fetching Result
*** Fetching Runes
*** Checking out Curry at "v3.0.0"
*** Checking out keychain-swift at "7.0.1"
*** Checking out Nimble at "v6.1.0"
*** Checking out Quick at "v1.1.0"
*** Checking out Result at "3.2.1"
*** Checking out ReactiveSwift at "1.1.1"
*** Checking out ReactiveCocoa at "b27c62bf8743a6644219a862f2c51c4334f225d6"
*** Checking out Runes at "v4.0.1"
*** Checking out Argo at "v4.1.2"
*** Checking out Tentacle at "2141159b750a0cd170264445da01eb56f1b5a38c"
*** Checking out YamlSwift at "3.3.1"
rome download --platform ios
Downloaded Nimble from: Nimble/iOS/Nimble.framework-v6.1.0.zip
Downloaded Nimble.dSYM from: Nimble/iOS/Nimble.framework.dSYM-v6.1.0.zip
Downloaded Quick from: Quick/iOS/Quick.framework-v1.1.0.zip
Downloaded Quick.dSYM from: Quick/iOS/Quick.framework.dSYM-v1.1.0.zip
Downloaded ReactiveCocoa from: ReactiveCocoa/iOS/ReactiveCocoa.framework-b27c62bf8743a6644219a862f2c51c4334f225d6.zip
Downloaded ReactiveCocoa.dSYM from: ReactiveCocoa/iOS/ReactiveCocoa.framework.dSYM-b27c62bf8743a6644219a862f2c51c4334f225d6.zip
Downloaded ReactiveSwift from: ReactiveSwift/iOS/ReactiveSwift.framework-1.1.1.zip
Downloaded ReactiveSwift.dSYM from: ReactiveSwift/iOS/ReactiveSwift.framework.dSYM-1.1.1.zip
Downloaded Result from: Result/iOS/Result.framework-3.2.1.zip
Downloaded Result.dSYM from: Result/iOS/Result.framework.dSYM-3.2.1.zip
Downloaded Yaml from: YamlSwift/iOS/Yaml.framework-3.3.1.zip
Downloaded Yaml.dSYM from: YamlSwift/iOS/Yaml.framework.dSYM-3.3.1.zip
Downloaded KeychainSwift from: keychain-swift/iOS/KeychainSwift.framework-7.0.1.zip
Downloaded KeychainSwift.dSYM from: keychain-swift/iOS/KeychainSwift.framework.dSYM-7.0.1.zip
Downloaded Tentacle from: Tentacle/iOS/Tentacle.framework-2141159b750a0cd170264445da01eb56f1b5a38c.zip
Downloaded Tentacle.dSYM from: Tentacle/iOS/Tentacle.framework.dSYM-2141159b750a0cd170264445da01eb56f1b5a38c.zip
Downloaded Argo from: Argo/iOS/Argo.framework-v4.1.2.zip
Downloaded Argo.dSYM from: Argo/iOS/Argo.framework.dSYM-v4.1.2.zip
Downloaded Curry from: Curry/iOS/Curry.framework-v3.0.0.zip
Downloaded Curry.dSYM from: Curry/iOS/Curry.framework.dSYM-v3.0.0.zip
Downloaded Runes from: Runes/iOS/Runes.framework-v4.0.1.zip
Downloaded Runes.dSYM from: Runes/iOS/Runes.framework.dSYM-v4.0.1.zip
Downloaded .Nimble.version from: Nimble/.Nimble.version-v6.1.0
Downloaded .Quick.version from: Quick/.Quick.version-v1.1.0
Downloaded .ReactiveCocoa.version from: ReactiveCocoa/.ReactiveCocoa.version-b27c62bf8743a6644219a862f2c51c4334f225d6
Downloaded .ReactiveSwift.version from: ReactiveSwift/.ReactiveSwift.version-1.1.1
Downloaded .Result.version from: Result/.Result.version-3.2.1
Downloaded .YamlSwift.version from: YamlSwift/.YamlSwift.version-3.3.1
Downloaded .keychain-swift.version from: keychain-swift/.keychain-swift.version-7.0.1
Downloaded .Tentacle.version from: Tentacle/.Tentacle.version-2141159b750a0cd170264445da01eb56f1b5a38c
Downloaded .Argo.version from: Argo/.Argo.version-v4.1.2
Downloaded .Curry.version from: Curry/.Curry.version-v3.0.0
Downloaded .Runes.version from: Runes/.Runes.version-v4.0.1
Again, the rome list --platform ios --missing
is empty and I'm able to compile my app.
When I run it though, I get the following error:
Referenced from: /Users/romain/Library/Developer/CoreSimulator/Devices/E78E9062-AF8A-4BE0-B6D0-4B6C9FB5737E/data/Containers/Bundle/Application/B2BB8941-D1B0-402F-A22D-35591451114B/Memoires.app/Frameworks/ReactiveCocoa.framework/ReactiveCocoa
Reason: image not found
Happy to dig a little more but in the meantime, have you seen something like this?
Refactor all the type Something = String
to newtype Something = Something String
Steps which explain the enhancement
type Location = String
type Version = String
to
newtype Location = Location String
newtype Version = Version String
type FrameworkName = String
type GitRepoName = String
to
newtype FrameworkName = FrameworkName String
newtype GitRepoName = GitRepoName String
Current and suggested behavior
Currently some functions are stirngly typed and this leads to quite some confusion around them.
For example in deriveFrameworkNameAndVersion the return type is [(FrameworkName, Version)]
which is just an alias for [(String, String)]
. This is not optimal as it's possible to mix up FrameworkName
with Version
later down then code.
If FrameworkName
and Version
were defined using newtype
instead of type
mixing FrameworkName
and Version
would not be possible.
Rome version: v.0.6.1.11
OS and version: MacOSX 10.11.6 - El Capitan
Reduce code duplication in some function.
Steps which explain the enhancement
Example of function which could use some slimming down
Current and suggested behavior
Currently there are tiny differences which lead to a ton of duplication. A more careful design can reduce code duplication.
Why would the enhancement be useful to most users
This enhancement does nothing for the users but a lot for code quality and maintenance
Hi there,
I'm looking for an option to build and upload different xcode-version builds of the same framework into the same bucket.
This might be possible by storing different buildtypes in subpaths in the same buckets and up-/downloading with a configuration path like:
rome upload --buildpath DEFINED_IN_ROMEFILE
rome download --buildpath DEFINED_IN_ROMEFILE
The buildpath could simply be defined within the romefile.
Does a similar feature already exist?
**Rome version:** latest
**OS and version:** macOS 10.12.5
Cheers
I want to use Rome as local cache only.
That is useful when I using git-worktree.
Currently if Rome does not find a dependency or bcsymbolmap
in the cache it outputs this as an error, and this is particularly ugly when used with Fastlane.
It is just noise in almost all cases I can think of, and is particularly noisy when you do your first Rome download and when you build only for one platform e.g. iOS where you get all the symbol map errors.
Steps which explain the enhancement
Current and suggested behavior
Running Rome for the first time on a project that has an empty cache chokes with an error on every single dependency. e.g:
[13:36:41]: ▸ Error: could not find OnePasswordExtension.6B87C426-B309-3394-A31D-8883DC6C4D6A.bcsymbolmap in local cache at : /Users/marc/Library/Caches/Rome/XXXXXX/onepassword-extension/iOS/6B87C426-B309-3394-A31D-8883DC6C4D6A.bcsymbolmap-04f146abd3bd5e6f7ba8b18b916c55f3b3c85aed.zip
[13:36:41]: ▸ Error: could not find OnePasswordExtension.F4A8C455-19F7-35F3-96D2-7DD4CAC5040A.bcsymbolmap in local cache at : /Users/marc/Library/Caches/Rome/XXXXXX/onepassword-extension/iOS/F4A8C455-19F7-35F3-96D2-7DD4CAC5040A.bcsymbolmap-04f146abd3bd5e6f7ba8b18b916c55f3b3c85aed.zip
[13:36:41]: ▸ Error: could not find OnePasswordExtension.7D3E2BB4-DE64-321B-994C-A69BD45877C7.bcsymbolmap in local cache at : /Users/marc/Library/Caches/Rome/XXXXXX/onepassword-extension/iOS/7D3E2BB4-DE64-321B-994C-A69BD45877C7.bcsymbolmap-04f146abd3bd5e6f7ba8b18b916c55f3b3c85aed.zip
[13:36:41]: ▸ Error: could not find OnePasswordExtension.5E20B41A-A888-3783-AE7B-F45278581988.bcsymbolmap in local cache at : /Users/marc/Library/Caches/Rome/XXXXXX/onepassword-extension/iOS/5E20B41A-A888-3783-AE7B-F45278581988.bcsymbolmap-04f146abd3bd5e6f7ba8b18b916c55f3b3c85aed.zip
and running Rome again later for a single platform after the cache has been populated for a single platform results in an error for every dependency for every platform that was not build, e.g. at least two error lines per dependency if you built dependencies only for iOS:
[13:48:41]: ▸ Error: could not find OnePasswordExtension.6B87C426-B309-3394-A31D-8883DC6C4D6A.bcsymbolmap in local cache at : /Users/marc/Library/Caches/Rome/XXXXXX/onepassword-extension/iOS/6B87C426-B309-3394-A31D-8883DC6C4D6A.bcsymbolmap-04f146abd3bd5e6f7ba8b18b916c55f3b3c85aed.zip
[13:48:41]: ▸ Error: could not find OnePasswordExtension.F4A8C455-19F7-35F3-96D2-7DD4CAC5040A.bcsymbolmap in local cache at : /Users/marc/Library/Caches/Rome/XXXXXX/onepassword-extension/iOS/F4A8C455-19F7-35F3-96D2-7DD4CAC5040A.bcsymbolmap-04f146abd3bd5e6f7ba8b18b916c55f3b3c85aed.zip
Why would the enhancement be useful to most users
This output is not generally useful to users. Items not being in a cache is not an error, it is expected behaviour for any cache that can be invalidated.
Rome version: 0.13.0.33
OS and version: maxOS 10.12.6
When the Cartfile.resolved contains a binary framework, e.g.:
binary "https://www.mapbox.com/ios-sdk/Mapbox-iOS-SDK.json" "3.5.0"
Rome emits the following error when attempting to perform a download:
$ rome download
Carfile.resolved parse error: "Cartfile.resolved" (line 1, column 1):
unexpected "b"
expecting white space, "git" or "github"
Additionally, when this occurs, Rome does not error with a non-zero exit status to indicate that an error occurred. This causes scripts to continue as if no error occurred.
This was tested with the latest Rome version:
$ rome --version
0.10.1.22 - Romam uno die non fuisse conditam.
Steps to reproduce
Cartfile
dependency: github "ashleymills/Reachability.swift" == 3.0
Cartfile.resolved
with the contents like github "ashleymills/Reachability.swift" "v3"
rome upload --platform iOS --cache-prefix 8E3004b
Expected
8E3004b/Reachability.swift/.Reachability.swift.version-v3
8E3004b/Reachability.swift/iOS/Reachability.framework-v3.zip
8E3004b/Reachability.swift/iOS/Reachability.framework-v3.dSYM.zip
Observed
8E3004b/Reachability.swift/.Reachability.swift.version-v3
Additional info
Before upload (and download as well) Rome should read contents of version file .Reachability.swift.version-v3
to get actual name of framework.
For instance contents of .Reachability.swift.version-v3
is the following:
{
"commitish" : "v3",
"iOS" : [
{
"hash" : "d2868e98c02222b2a7bff12397406e821bf48803c701a44fb0b4667f8efd5d94",
"name" : "Reachability"
}
]
}
From the contents of the version file we can see that real name of framework will be "Reachability.framework", but not "Reachability.swift.framework".
Similar problem observed with following dependencies:
github "BoltsFramework/Bolts-ObjC" "1.8.4"
github "airbnb/lottie-ios" "1.0.2"
github "ashleymills/Reachability.swift" "v3"
github "facebook/facebook-ios-sdk" "sdk-version-4.21.0"
github "facebook/facebook-sdk-swift" "a09021edf33417153fdf1066275f888c3b59fc6e"
github "yeahdongcn/UIColor-Hex-Swift" "3.0.2"
Rome version: 0.12.0.31 - Romam uno die non fuisse conditam.
OS and version: macOS 10.12.5 (16F73)
Thank you!
One of the carthage package that we use is XCGLogger and it contains two frameworks:
Rome is able to upload the XCGLogger.framework and symbol files but completely ignores the ObjcExceptionBridging.framework.
Rome version: 0.11.0.27
OS and version: macOS 10.12.5
Add support for using rsync
and/or scp
(or similar) to move files between the local Rome cache and a remote (privately hosted) file store.
For example:
[Cache]
local = ~/Library/Caches/Rome
remote_scp = [email protected]:/some/remote/rome-cache
# OR
remote_rsync = [email protected]:/some/remote/rome-cache
Currently only Amazon's S3 is supported as remote storage. Supporting rsync
and/or scp
would allow for using existing in-house storage infrastructure.
Supporting alternative remote copy mechanisms would allow users to get all the benefits of Rome's remote caching support without relying on a cloud storage provider.
Enable Rome to download & read an well know repository map
rome update-well-known [URL]
or default URL)list, download, upload
also check the well known map for framework name resolutionCurrent and suggested behavior
Currently there is no well-known map.
Why would the enhancement be useful to most users
The enhancement could enable users to write no [RepositoryMap] section of a Romefile
Current behavior
Cartfile:
github "bitstadium/HockeySDK-iOS"
Romefile:
[RepositoryMap]
HockeySDK-iOS = HockeySDK
When building with Carthage:
$ carthage build HockeySDK-iOS --platform ios
*** Building scheme "HockeySDK Framework" in HockeySDK.xcodeproj
The following is produced: Cartage/Build/iOS/HockeySDK.framework*
However, unlike Carthage, the repo name does not work when passed to rome upload
$ rome upload HockeySDK-iOS
(nothing uploaded)
If the Framework name is specified, it does work:
$ rome upload HockeySDK
Successfully uploaded HockeySDK to: HockeySDK/HockeySDK.framework-4.1.2.zip
Successfully uploaded HockeySDK.dSYM to: HockeySDK/HockeySDK.framework.dSYM-4.1.2.zip
As such, rome upload
doesn't function the same way as Carthage, which is what users would expect in this scenario (repository names are provided, not framework names)
Expected behavior
The upload
command should take repository names, not framework names.
Rome version: 0.6.0.10
OS macOS sierra 10.12
The following carthage command:
carthage build ReactiveCocoa --platform ios
produces the outputs:
Carthage/Build/iOS/ReactiveCocoa.framework
Carthage/Build/iOS/ReactiveCocoa.framework.dSYM
However, invoking:
rome upload ReactiveCocoa
Only uploads the framework, not the dSYM
.
It would seem that rome should cache not just .framework
files matching the framework name, but rather any files matching the prefix of the framework name.
Some of my frameworks are missing the platform directory and cannot be downloaded after a successful upload
For example, I execute the following command and get a success message:
rome upload --platform ios amp.ios
Uploaded .amp.ios.version to: amp.ios/.amp.ios.version-2.27
I can see on AWS, the file exists under /amp.ios/.amp.ios.version-2.27
When i attempt to download the file I get an error:
rome download amp.ios
The specified key does not exist.
But i have another framework that works just perfect:
rome upload --platform ios libzeus
Uploaded libzeus to: libzeus/iOS/libzeus.framework-2.70.zip
I can see on AWS, the file exists under /libzeus/iOS/.libzeus.version-2.70
rome download --platform ios libzeus
Downloaded libzeus from: libzeus/iOS/libzeus.framework-2.70.zip
Why would one of these use a different path structure and work?
**Rome version:** 0.10.0.21
**OS and version:** OSX Sierra - 10.12.3
Add a command to verify several properties of a framework
Steps which explain the enhancement
A verify command with several switches could be added to:
Current and suggested behavior
No verification is performed at the moment. Verification of some kind, along the lines of the suggestions above can be performed.
Why would the enhancement be useful to most users
Verifying properties of a framework present in the cache give the users move confidence in the cached frameworks.
Hey,
carthage 0.20.0 introduced the --cache-builds
flag, which generates .NAME.version
version files for each built framework. When building a dependency carthage will use these version files to check if a corresponding build artefact is already available and skip the build. Would be great if rome could store these version files alongside the frameworks in the S3 cache and restore them during a rome download
.
Thank you!
I recently updated from Rome v0.12.0.31 to v0.14.0.38 and can't properly download frameworks from cache anymore. Downloading directly from S3 works after clearing the cache. As soon as I try to get the frameworks from cache afterwards it fails again, though:
Found AFNetworking in local cache at: /Users/MYUSER/Library/Caches/Rome/AFNetworking/iOS/AFNetworking.framework-3.2.0.zip
rome: Carthage/Build/iOS/AFNetworking.framework/AFNetworking: fileAccess: does not exist (No such file or directory)
I checked the cache and there is no framework in the zip; only a bcsymbolmap file.
The files on S3 on the other hand look fine. From what I can tell the problem lies with the download process. It looks like the framework zips are correctly copied to the cache folder but then subsequently overwritten with the bcsymbolmap zips. As a result the next rome download will fail because there is no framework in the zip.
Here is copy of the output when downloading from S3:
Error: could not find AFNetworking in local cache at : /Users/MYUSER/Library/Caches/Rome/AFNetworking/iOS/AFNetworking.framework-3.2.0.zip
Downloaded AFNetworking from: AFNetworking/iOS/AFNetworking.framework-3.2.0.zip
Copied AFNetworking to: /Users/MYUSER/Library/Caches/Rome/AFNetworking/iOS/AFNetworking.framework-3.2.0.zip
Error: could not find AFNetworking.769B93C4-4CF0-39F1-A670-697E19307A7B.bcsymbolmap in local cache at : /Users/MYUSER/Library/Caches/Rome/AFNetworking/iOS/769B93C4-4CF0-39F1-A670-697E19307A7B.bcsymbolmap-3.2.0.zip
Error: could not find AFNetworking.24BC9060-0771-39F2-A635-E31BAF85A181.bcsymbolmap in local cache at : /Users/MYUSER/Library/Caches/Rome/AFNetworking/iOS/24BC9060-0771-39F2-A635-E31BAF85A181.bcsymbolmap-3.2.0.zip
Error: could not find AFNetworking.4B83EC95-A761-3986-96B1-79D7E1F84306.bcsymbolmap in local cache at : /Users/MYUSER/Library/Caches/Rome/AFNetworking/iOS/4B83EC95-A761-3986-96B1-79D7E1F84306.bcsymbolmap-3.2.0.zip
Error: could not find AFNetworking.7098A323-D3AB-3A81-9A37-CFBB5A725B75.bcsymbolmap in local cache at : /Users/kay/Library/Caches/Rome/AFNetworking/iOS/7098A323-D3AB-3A81-9A37-CFBB5A725B75.bcsymbolmap-3.2.0.zip
Error: could not download AFNetworking.769B93C4-4CF0-39F1-A670-697E19307A7B.bcsymbolmap : The specified key does not exist.
Error: could not download AFNetworking.24BC9060-0771-39F2-A635-E31BAF85A181.bcsymbolmap : The specified key does not exist.
Downloaded AFNetworking.4B83EC95-A761-3986-96B1-79D7E1F84306.bcsymbolmap from: AFNetworking/iOS/4B83EC95-A761-3986-96B1-79D7E1F84306.bcsymbolmap-3.2.0.zip
Copied AFNetworking to: /Users/kay/Library/Caches/Rome/AFNetworking/iOS/AFNetworking.framework-3.2.0.zip
Error: no such file Carthage/Build/iOS/4B83EC95-A761-3986-96B1-79D7E1F84306.bcsymbolmap
Downloaded AFNetworking.7098A323-D3AB-3A81-9A37-CFBB5A725B75.bcsymbolmap from: AFNetworking/iOS/7098A323-D3AB-3A81-9A37-CFBB5A725B75.bcsymbolmap-3.2.0.zip
Copied AFNetworking to: /Users/kay/Library/Caches/Rome/AFNetworking/iOS/AFNetworking.framework-3.2.0.zip
Error: no such file Carthage/Build/iOS/7098A323-D3AB-3A81-9A37-CFBB5A725B75.bcsymbolmap
Error: could not find AFNetworking.dSYM in local cache at : /Users/kay/Library/Caches/Rome/AFNetworking/iOS/AFNetworking.framework.dSYM-3.2.0.zip
Downloaded AFNetworking.dSYM from: AFNetworking/iOS/AFNetworking.framework.dSYM-3.2.0.zip
Copied AFNetworking.dSYM to: /Users/kay/Library/Caches/Rome/AFNetworking/iOS/AFNetworking.framework.dSYM-3.2.0.zip
Error: could not find .AFNetworking.version in local cache at : /Users/kay/Library/Caches/Rome/AFNetworking/.AFNetworking.version-3.2.0
Downloaded .AFNetworking.version from: AFNetworking/.AFNetworking.version-3.2.0
Copied .AFNetworking.version to: /Users/kay/Library/Caches/Rome/AFNetworking/.AFNetworking.version-3.2.0
Copied .AFNetworking.version to: Carthage/Build/.AFNetworking.version
Rome version: 0.14 and 0.13
OS and version: Tested in 10.13.2
Hope you can help :)
Add support for more platforms (watchOS, tvOS, macOS)
**Current and suggested behavior**
Currently only iOS is supported, this should be expanded to other platform too.
**Why would the enhancement be useful to most users**
This enhancement should be useful for frameworks that target more than 1 platform.
**Rome version:** v0.7.1.13
**OS and version:** Mac OS 10.11.6 - El Capitan
When using rome list --missing
, the projects listed under [IgnoreMap] are skipped thus it is not possible to use it to "checkout" or "bootstrap" dependencies that are not stored in Rome. For example:
"rome list --missing --platform ios | awk '{print $1}' | xargs carthage bootstrap --platform ios --cache-builds"
**Current and suggested behavior**
Add a parameter to allow returning ignored projects from "rome list"
[Short description of suggestion]
**Steps which explain the enhancement**
1. run `carthage update --platform ios && rome upload --platform ios`
**Current and suggested behavior**
output:
Copied Alamofire.framework to: /Users/xuemincai/Library/Caches/Rome/Alamofire/iOS/Alamofire.framework-4.4.0.zip
Error uploading Alamofire: Unexpected Error
**Why would the enhancement be useful to most users**
**Rome version: 0.11.0.27
**OS and version: macOS Sierra 10.12.5
Make Rome take binary
dependencies into account from Cartfile
.
Steps which explain the enhancement
binary
dependencies in Cartfile
/ Cartfile.resolved
.Current and suggested behavior
It appears that currently Rome does not take binary-only Carthage dependencies into account. For projects like ours that depend on large, externally-hosted binaries, this means we have to add an extra step that does carthage bootstrap <binarydeps>
to ensure those dependencies are available in the Carthage/Binaries
directory, which means always making a network request. (Even on a fast connection this might add 10-30 seconds to a build, whereas if the local cache could be used it would be < 1 second.)
Why would the enhancement be useful to most users
For any users relying on binary-only dependencies, it would no longer be necessary to add special workarounds, and builds would be faster by taking advantage of locally cached binaries instead of making network requests.
Rome version: 0.13.0.33
OS and version: macOS 10.12.6
This issue sounds similar to #106, but that ticket seemed to cover a different scenario, so I figured I'd file this as a separate issue.
I just updated our macOS build to use Rome and found that the build would break in code signing, but only in the case where binaries are copied in from Rome's local cache. (The build was successful when using artifacts produced directly by Carthage.) I discovered that the frameworks produced by Carthage contain symlinks, for example:
% ls -la Carthage/Build/Mac/CocoaLumberjack.framework
total 32
drwxr-xr-x 7 campbell staff 238 Jan 5 11:31 .
drwxr-xr-x 19 campbell staff 646 Jan 5 13:27 ..
lrwxr-xr-x 1 campbell staff 32 Jan 5 11:31 CocoaLumberjack -> Versions/Current/CocoaLumberjack
lrwxr-xr-x 1 campbell staff 24 Jan 5 11:31 Headers -> Versions/Current/Headers
lrwxr-xr-x 1 campbell staff 24 Jan 5 11:31 Modules -> Versions/Current/Modules
lrwxr-xr-x 1 campbell staff 26 Jan 5 11:31 Resources -> Versions/Current/Resources
drwxr-xr-x 4 campbell staff 136 Jan 5 11:31 Versions
After a rome upload
step, if I unzip the relevant file from the Rome cache, I see that the symlinks are no longer present:
% cd ~/Library/Caches/Rome/CocoaLumberjack/Mac/
% unzip CocoaLumberjack.framework-3.3.0.zip
% ls -la Carthage/Build/Mac/CocoaLumberjack.framework
total 504
drwxr-xr-x 7 campbell staff 238 Jan 5 2018 .
drwxr-xr-x 3 campbell staff 102 Jan 5 13:29 ..
-rwxr-xr-x 1 campbell staff 255500 Jan 5 2018 CocoaLumberjack
drwxr-xr-x 18 campbell staff 612 Jan 5 2018 Headers
drwxr-xr-x 3 campbell staff 102 Jan 5 2018 Modules
drwxr-xr-x 3 campbell staff 102 Jan 5 2018 Resources
drwxr-xr-x 4 campbell staff 136 Jan 5 2018 Versions
So if later I use rome download
, the Carthage/Build
directory will be populated with the framework(s) that don't contain symlinks.
This causes codesign
to fail later with a "bundle format is ambiguous" error, which is described here:
https://developer.apple.com/library/content/technotes/tn2206/_index.html#//apple_ref/doc/uid/DTS40007919-CH1-TNTAG313
"Perhaps a framework was copied incorrectly so the symlinks it contained were converted to normal files."
The relevant portion of our Cartfile.resolved
:
github "CocoaLumberjack/CocoaLumberjack" "3.3.0"
And our Romefile
:
[Cache]
local = ~/Library/Caches/Rome
[RepositoryMap]
CocoaLumberjack = CocoaLumberjack, CocoaLumberjackSwift
Rome version: 0.13.1.35
OS and version: macOS 10.12.6
Currently, when you download a framework that has less files than what's on disk, Rome will leave the old files in place within the framework's directory structure.
For example, in CocoaLumberjack 3.2.0, the DDOSLogger.h
file was added. If you have CocoaLumberjack 3.2.0 on disk, and perform a rome download CocoaLumberjack
(with your Cartfile.resolved containing version 3.1.0), the DDOSLogger.h
file will remain on disk. However, when you perform a build, you will get a failure with the following error:
<module-includes>:1:1: Umbrella header for module 'CocoaLumberjack' does not include header 'DDOSLogger.h'
Once you remove the DDOSLogger.h
file from disk, the build continues within errors.
Rome version: 0.10.2.24 - Romam uno die non fuisse conditam.
OS Version macOS 10.12.4 (16E195)
Rome should be able to automatically generate a Romefile
Steps which explain the enhancement
Cartfile.resolved
Carthage/Checkouts
folderIssue:
~/Library/Caches/Rome
Current and suggested behavior
Currently users have to generate by hand a Romefile. This can be an entry barrier to using Rome.
Why would the enhancement be useful to most users
Users would not need to write any Romefile to get a basic working configuration with a local cache and a [RepositoryMap]
section derived from the current contents of the Carthage/Checkouts
.
This will lower the friction when using Rome for the first time.
Rome version: N/A
OS and version: N/A
Enhancement Suggestion
Afternoon,
I'd love to use Rome but can't use S3 for big-silly-company reasons. I can, however set up a Minio (https://docs.minio.io/) instance locally which is basically an S3 compatible object store that runs within a docker container.
Getting rome to work with Minio would mean supporting an s3 url in .aws/config
rather than a region. Something like:
[default]
endpoint = https://play.minio.io:9000
Why would the enhancement be useful to most users
Supporting s3-compatible endpoints would open up other similar object storage systems and services. I know s3 can be super cheap, but some companies (esp. european ones) can't use AWS for compliance reasons.
Hi,
This solution is pretty cool. Setting up Amazon S3 looks scary though. Something like bintray might be cool?
Upload fails for certain types of frameworks where the binary inside the framework is a symlink to a version inside the framework.
For example, Starscream for Mac (https://github.com/daltoniam/Starscream). Here's a screenshot:
http://cloud.plutinosoft.com/oKcN
$ ls -la Carthage/Build/Mac/Starscream.framework
total 48
drwxr-xr-x 8 srebaud staff 272 Dec 14 10:57 .
drwxr-xr-x 21 srebaud staff 714 Dec 14 09:52 ..
-rw-r--r--@ 1 srebaud staff 6148 Dec 14 10:57 .DS_Store
lrwxr-xr-x 1 srebaud staff 24 Dec 14 09:52 Headers -> Versions/Current/Headers
lrwxr-xr-x 1 srebaud staff 24 Dec 14 09:52 Modules -> Versions/Current/Modules
lrwxr-xr-x 1 srebaud staff 26 Dec 14 09:52 Resources -> Versions/Current/Resources
lrwxr-xr-x 1 srebaud staff 27 Dec 14 09:52 Starscream -> Versions/Current/Starscream
drwxr-xr-x 5 srebaud staff 170 Dec 14 10:57 Versions
Steps which explain the enhancement
Current and suggested behavior
The Starscream.framework is not published in the local folder.
Rome version: 0.13.0.33
OS and version: ('Darwin', 'Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64', 'x86_64')
Add support for Google Cloud Storage
Current and suggested behavior
Currently only Amazon's S3 is supported as remote storage. Supporting Google Cloud Storage would allow users on the Google Cloud to use Rome too
Why would the enhancement be useful to most users
Supporting Google Cloud Storage would allow users on the Google Cloud to use rome too.
Automatically detect frameworks to upload/download from .version
-file.
Steps which explain the enhancement
Carthage.resolved
with the following contents github "facebook/facebook-ios-sdk" "sdk-version-4.21.0"
facebook-ios-sdk
produces 3 frameworks during build process: FBSDKLoginKit.framework, FBSDKShareKit.framework, FBSDKCoreKit.framework..facebook-ios-sdk.version
file with the following contents:{
"commitish" : "sdk-version-4.21.0",
"iOS" : [
{
"hash" : "c22ab9aaf5b3049b6a5ce2387c714525a163d8b942f6d3d93ce533d37353d282",
"name" : "FBSDKLoginKit"
},
{
"hash" : "ca63365bfc88e599656c74a3e44b233b97b0dbb845b6ef1bf5df87db05baf023",
"name" : "FBSDKShareKit"
},
{
"hash" : "47de2a67234abb2245692056fc633c88bb4704bb68a66592fc991f538aa62665",
"name" : "FBSDKCoreKit"
}
]
}
Current and suggested behavior
Currently we have to setup configuration setting RepositoryMap
to hint Rome
which files to use during download/upload.
[RepositoryMap]
facebook-ios-sdk = FBSDKCoreKit, FBSDKLoginKit, FBSDKShareKit
I am suggesting to update Rome
in the way so that it will read framework names from .version
-file. In this case we do not need to use RepositoryMap
configuration setting in order to hint Rome
which files to use during download/upload.
Why would the enhancement be useful to most users
Users will have better user experience and less configuration to setup. Reliability will be increased as well. For example: if in the future for some reason vendor will decide to change names and number of frameworks produced during Carthage build.
Rome version: 0.12.0.31 - Romam uno die non fuisse conditam.
OS and version: macOS 10.12.5 (16F73)
Thank you!
Add the possibility to output the results of the list command as JSON
Steps which explain the enhancement
list
(perhaps --print-format=JSON
)list
command outputCurrent and suggested behavior
Currently the only output format is a custom text representation
Why would the enhancement be useful to most users
This enhancement would allow for more flexibility when integrating Rome into other workflows
It would appear that the Romefile repository map doesn't work with github references—only with git references. It seems to have to do with the deriveFrameworkName
function at https://github.com/blender/Rome/blob/master/src/Lib.hs#L192, which doesn't appear to be using the romeMap
to do a repo to framework lookup. This results in the Romefile
mappings being unused when they are mapping from a github repo name to a framework name.
Rome looks like an awesome tool! Quick question about its usage:
In the example workflow, it would appear that there is an assumption that the entire contents of the current Cartfile.resolved
will either be fully available for download or not. However, in the case where you have internal frameworks that are constantly being updated, it could likely be the case that all frameworks but the one you just updated are available in your s3 cache.
Would it be possible to have a command like the following to allow building only the dependencies that are currently uncached? e.g.:
rome download && carthage build `rome unavailable`
Say for example that APIClient
was the only library that was uncached. This would evaluate to:
rome download && carthage build APIClient
Let me know if this is already possible or isn't needed for another reason. Thanks!
Rome should be able to also upload the framework build from the current project when
carthage build --platform iOS --no-skip-current
is used.
Steps which explain the enhancement
currentRepoName = current-framework-name
key value pairrome upload --no-skp-current
should look into [Current] to upload a framework.Current and suggested behavior
Why would the enhancement be useful to most users
Rome version: v0.6.1.10
OS and version: Mac OS 10.11.6 - El Capitan
When making changes to an existing framework, which is to be integrated into another framework or app, it is not possible to upload it as Rome only refers to the Cartfile.resolved and not the actual framework being built. This means that the framework in question must first be checked out and built in some other context.
**Steps which explain the enhancement**
1. Modify a framework.
2. Build the framework.
3. Call Rome upload
**Current and suggested behaviour**
The current behaviour is that only those frameworks referenced from the Cartfile.resolved are considered. The new behaviour would be that the framework being worked on would also be uploaded.
**Why would the enhancement be useful to most users**
The enhancement would facilitate interim modifications and their testing in the working environment. It would remove the redundant step of building again in the calling context for both interim and final versions.
**Rome version:** 0.6.0.9
**OS and version:** Mac OSX, 10.11.6
Would be great to have a local cache directory instead of AWS
Steps which explain the enhancement
1. Check if product is in local cache
2. Check if product is in AWS
3. Continue with normal behavior
**Current and suggested behavior**
sometimes, it takes too much time to fetch on aws
**Why would the enhancement be useful to most users**
sometimes, it takes too much time to fetch on aws
**Rome version:** -
**OS and version:** -
Add support for more platforms (watchOS, tvOS, macOS)
Current and suggested behavior
Currently only iOS is supported, this should be expanded to other platform too.
Why would the enhancement be useful to most users
This enhancement should be useful for frameworks that target more than 1 platform.
Rome version: v0.7.1.13
OS and version: Mac OS 10.11.6 - El Capitan
When downloading files from cache the Module.framework/Module file doesn't have executable permission, this leads to library not loaded crashes
Just adding the missing executable permission fix this issue.
Tests are currently lacking in the project. While some functions are harder to tests there are a few low hanging fruits.
Steps which explain the enhancement
By using QuickCheck add more tests
Tests should reside in the tests directory
Current and suggested behavior
No changes to the application's behaviour
Why would the enhancement be useful to most users
This change would enhance the confidence in the code base
Rome version: v0.6.0.10
OS and version: MacOSX 10.11.6 - El Capitan
Carthage allows one repository to have many framework targets. When a carthage build is performed, it builds them all. An example of this is CocoaLumberjack, which produces two iOS frameworks from a single repository:
carthage build CocoaLumberjack --platform ios
*** Building scheme "CocoaLumberjack-iOS" in Lumberjack.xcworkspace
*** Building scheme "CocoaLumberjackSwift-iOS" in Lumberjack.xcworkspace
this produces:
Carthage/Build/iOS/CocoaLumberjack.framework
Carthage/Build/iOS/CocoaLumberjack.framework.dSYM
Carthage/Build/iOS/CocoaLumberjackSwift.framework
Carthage/Build/iOS/CocoaLumberjackSwift.framework.dSYM
It seems as if the [RepositoryMap]
should allow for a single repository to map to many framework references. You could imagine it would look something like this:
[RepositoryMap]
CocoaLumberjack CocoaLumberjack CocoaLumberjackSwift
So it fails, given ~/.aws/credentials
doesn't exist. My build:
https://travis-ci.org/zerofinancial/relay/builds/269878346
**Rome version:** v0.12.0.31
**OS and version:** [Enter OS name and version here]
Hi,
Would it be possible to have more verbose info on the download of the frameworks?
I need that as Jenkins will kill the build if the job does not produce output log after a given time
Make Rome available via Homebrew
When attempting to try out the executable for fixing #2, I ran into the issue where attempting to upload binaries to AWS is failing. This is due to our carthage cache bucket being hosted in the us-west-2
region.
The upload
command fails with the following error:
Error uploading FrameworkName : The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
I have the region specified in my .aws/config
file, but this doesn't seem to be having any effect either.
It would appear that the issue is NorthVirginia
in this file https://github.com/blender/Rome/blob/master/app/Main.hs#L23
Cache bcsymbolmap files
Steps which explain the enhancement
Current and suggested behavior
Currently bcsymbolmap files seem to be ignored by Rome
There should be a zip archive also for bcsymbolmap files
Carthage archive command makes a zip file which contains also bcsymbolmap files, so that might be useful reference:
$ carthage archive Alamofire
*** Found Carthage/Build/iOS/Alamofire.framework
*** Found Carthage/Build/iOS/Alamofire.framework.dSYM
*** Found Carthage/Build/iOS/7246C4C8-54CB-36B4-814F-E6E8B595EA74.bcsymbolmap
*** Found Carthage/Build/iOS/0F93FE26-5586-3A7C-8449-2DB18861AF00.bcsymbolmap
*** Found Carthage/Build/iOS/17132633-DF81-32A0-B7FE-BB4101B1C246.bcsymbolmap
*** Found Carthage/Build/iOS/A48FAE2B-3B09-3F27-A4FB-BE81C0FFE75B.bcsymbolmap
*** Created Alamofire.framework.zip
Why would the enhancement be useful to most users
bcsymbolmap is required for uploading bitcode containing apps to App Store
Rome version: 0.12.0.31
OS and version: macOS 10.12.6
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.