Comments (12)
Thanks for this question, i will try to elaborate later tonight.
from rome.
Isn't it possible to parse the actual project files in order to extract the target names and avoid the manual mapping altogether?
from rome.
Fist off, if all your dependencies are of type "github ..." and follow the naming Owner/FrameworkName
where FrameworkName
is the actual name of the output binary (like Alamofire/Alamofire
that produces Alamofire.framework
) you don't need any mapping. It just works. Maybe this is not clear from the docs.
The mapping is needed for a couple of reasons.
- To resolve the name of the produced binary when it can't be derived (git repos and binary dependencies can be named whatever, and that is the only trace they leave in the Cartfile)
- To resolve multiple names in case a project produces multiple binaries
- To explicitly ignore some binaries
Now, to your question:
Isn't it possible to parse the actual project files in order to extract the target names and avoid the manual mapping altogether?
The mapping is needed because the project file (and thus the shared scheme) of the dependencies is not available the moment Rome runs.
The flow is:
- You have your app checked out .
- Along with the app the Cartfile and Cartfile.resolved are checked out too
- There is no
Carthage/Checkout
folder yet because the whole point is not to checkout or build - You run rome, get the binaries, compile and live happily ever after without waiting for the dependencies themselves to compile.
So it's a bit of a chicken and egg problem.
But ok, let's suppose you actually have the Checkout folder.
Then I suppose it's possible to read the names of the binaries and reverse match from the checkout path to the dependency in the Cartfile.
However I made rome to cover the first, more brutal, scenario.
I have an branch that has work in progress to try to auto generate the Romefile for you from a full carthage checkout if you want to pick it up.
from rome.
Thank you for the detailed response!
Indeed my question was upfront naive, but at the same time I still feel there is a place for improvement. This mapping looks to me quite a bummer to do manually given the overall idea of the tool to be transparent (as I would love to see it) to the whole build process. Looking at my own Cartfile I see at least realm-cocoa
which would generate Realm and RealmSwift, and also some indirect dependencies which could not match as well and that would be a tiral-and-error way of coming up with the mapping
Generating the rome file manually would probably solve the problem, although in this case I would put this mapping information in some kind of rome.resolved
file to separate this from user configurable settings, because as long as there is a way to get the mapping automatically I don't see any reason for the user to be involved into it at all
I'm thinking that even in the case when there is no Checkout folder and rome tries to get the needed binaries from the cache repository there could be some logic to get the mapping from the file previously pushed to the repository when it was originally created. Apparently rome has already run somewhere else to generate the cache, and if not then this will be the run which will have to build everything. So technically there is no chicken and the egg problem because there is always some rome process which runs initially together with the build (or after the build) and has access to all needed data to generate the clues for itself for later.
Hope this makes sense.
from rome.
@leonid-s-usov Thanks for the interest and for the suggestions.
I'm not trying to defend the current state of things and would be more than happy to make everything transparent if possible.
That said, let's discuss a couple of things.
Generating the rome file manually would probably solve the problem, although in this case I would put this mapping information in some kind of rome.resolved file to separate this from user configurable settings, because as long as there is a way to get the mapping automatically I don't see any reason for the user to be involved into it at all
The configuration file format is INI with sections.
The section [RepositoryMap]
is not required at all. So a minimal working Romefile looks like:
[Cache]
local = ~/Library/Chaces/Rome
If I understand correctly what you're saying, you're suggesting that the [RepositoryMap]
section should go in Romefile.resolved
. This is because in your vision the resolved file could be generated by Rome itself. Sure, makes sense if there is auto mapping.
there could be some logic to get the mapping from the file previously pushed to the repository when it was originally created.
You don't have to upload via Rome but let's ignore that for sake of simplicity. When rome uploads at the moment it still uses the map to figure out what to upload and where.
Let's take realm-cocoa
example.
During upload the map is used to tell it that BOTH Realm
and RealmSwift
have to go under the same cache path (i.e. <cache-prefix>/realm-cocoa/<platform>/[Realm |RealmSwift].framework
)
If I understand correctly, you're suggesting that since these binaries didn't appear out of thin air, at this point Rome could check the Carthage/Checkout
folder, reverse match and generate the Romefile.resolved
.
I didn't think about this flow. Seems like it could work and avoid the manual map.
As a candy, I think it would also be possible to upload the Romefile.resolved
to the remote cache at top level so that the cache auto-describes itself to someone that doesn't have the Romefile.resolved
.
This is a very appealing solution. My estimate of development time for this would be 2-5 days depending on your Haskell foo. At the moment I don't have this time but If you or anyone else wants to step in, be my guests. I have a small refactoring that I would like to push first though.
Now the actual suggestion in this ticket was for a simpler solution, where a [RepositoryMap]
would be stored somewhere remotely for all rome users that wish to use it. So that to them, it would appear like "magically" rome is aware of both Realm
and RealmSwift
.
from rome.
Good, so we are on the same page. Like I stated, I believe this information about mapping should be feasible to generate and consume without the user's intervention.
If uploaded to the remote cache I am thinking that maybe the fine should not be even created in the code repository at all, just as some version metadata at the known path in the repository. Could be even combined with the carthage version files into a single data file with the mapping and corresponding versions together.
The reason I posted on this issue is that although it does add this "magic" to the end user, it is laborious and error prone for the cases when the mapping is not up to date or the used carthage packages are proprietary. Someone would need to maintain this repository and it could grow pretty big over time.
Unfortunately, I have seen Haskell up close for the first time when browsing this repository source code, and to tell the truth I haven't felt this click inside to go and learn some Haskell. So I will not promise anything as of now
In any case, watching this project. Thank you
from rome.
@leonid-s-usov Have you seen this issue: #95
This suggests using the .version
file information but that would mean that Rome would only work if you use carthage with --cache-builds
so I'm not so adamant on the idea. Maybe I'm too cautious.
Can you elaborate on this? What do you mean by "just as some version metadata at the known path in the repository"
If uploaded to the remote cache I am thinking that maybe the file should not be even created in the code repository at all, just as some version metadata at the known path in the repository
from rome.
What you said,
As a candy, I think it would also be possible to upload the Romefile.resolved to the remote cache at top level so that the cache auto-describes itself to someone that doesn't have the Romefile.resolved
and what I said earlier, as well
I'm thinking that even in the case when there is no Checkout folder and rome tries to get the needed binaries from the cache repository there could be some logic to get the mapping from the file previously pushed to the repository when it was originally created
all this is about the fact that the mapping metadata may be uploaded to the repository by the process which initially pushes all data there (or updates), and then consumed by other clients. Not even necessary to have this mapping committed to the source code as the suggested rome.resolved
alongside the rome.ini
. I just used the naming convention to resemble Carthage layout and point out the fact that the mapping info is purely "technical" and shouldn't be manipulated directly by the user.
Regarding the --cache-builds
, looks like I was under wrong impression that the flag was required for rome to work in any case, wasn't it?
from rome.
yep, that issue #95 is exactly to the point!
what do you think of maybe making the rome tool call carthage on the user's behalf and make sure that the flags are set correctly? The local cache could actually be used directly be carthage. I did this in my project by creating a symlink of ./Carthage -> /path/to/local/carthage/cache/Carthage
from rome.
The problem with #95 is that .version
files are ALSO part of the cache. So can't use them for mapping.
.version
files tell carthage not to rebuit downstream depedencies found already built in Carthage/Build
, that's all it's does.
Rome is not supposed to call cathage because there is nothing to set correctly :\ It does not depend on --cache-builds
I'm not sure why you're doing /Carthage -> /path/to/local/carthage/cache
this if you're using rome. Rome will copy things at the right place for you.
from rome.
I'm doing this because i'm not using rome - yet 😄 .
And that's the reason why I'm here, because I was looking for some solution
The problem with #95 is that .version files are ALSO part of the cache. So can't use them for mapping.
Not sure I get it.. why can't the tool use files from the repository prior to making the decision? That is, download the small metadata file (maybe just the version files themselves, if --cache-builds is used) and use it to understand the mappings and then move on with its usual flow
from rome.
@leonid-s-usov You seem to be onto something better than I can think of at the moment so here is the challenge for you to solve:
- If there are no .version files, game over. No mapping possible atm and we all agree on that.
- If there are version files:
- How does the upload process to cache look like (think of git dependecies not on github and with messed up naming or multiple names)
- how does the download process look like
Here is the description of what the cache currently looks like for reference: https://github.com/blender/Rome#cache-structure
Ideally the change should not be breaking
from rome.
Related Issues (20)
- Xcode version specific upload/download HOT 4
- Concurrent downloads for remote cache HOT 1
- Nexus as example engine HOT 4
- [Question] Framework produces both static and dynamic output HOT 4
- Losing debug capability when using cached builds downloaded by Rome HOT 2
- Static folder isn't uploaded to remote server on Xcode 12 HOT 4
- Rome won't upload all downstream re-built frameworks HOT 3
- Rome sometimes printing partial output HOT 1
- Rome not failing when engine exit code is not zero HOT 1
- Issues running Rome with --no-skip-current HOT 6
- Needs to be updated to support XCFrameworks. HOT 13
- When using currentMap its not possible to reuse the generated binaries HOT 3
- Rome crashes when downloading dependencies HOT 1
- Skip bcsymbolmaps and dsyms when uploading/downloading a framework HOT 4
- Issues with Rome for xcframeworks HOT 24
- Failed to install latest ROME in MAC HOT 1
- using rome download --concurrently with --use-xcframeworks leads to io error: openBinaryFile: resource busy (file is locked)
- Add Rome arm64 release artifact HOT 3
- Error if carthage cache and rome cache have a file and directory with the same name
- Build failure with ghc 9.6.2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rome.