New SDK packing method on Linux (was: Outdated information?) about osxcross

[BioMike]

I was wrong. I ran into issue 29.

[BioMike]

I've looked into ways of extracting the SDK from newer XCode dmg archives on Linux and was able to to so with Xcode 5.1.1 (newer versions might also work) by using HFSExplorer version 0.23 (which build and runs on Linux): http://sourceforge.net/projects/catacombae/files/HFSExplorer/

I've manually extracted the data into a new directory (skipped all man page errors), and packed the directory into a tar.xz file. I used this in the build process. So far it seems it worked.

[tpoechtrager]

HFSExplorer seems to be a great tool, but to be honest, I am not really happy about the java / ant dependency.

However, thanks for posting this, I will update the packaging scripts soon.

[tpoechtrager]

HFSExplorer is totally broken, there are tons of uncaught exceptions and the command line utility doesn't even care about symlinks.

I am not sure if I really want to use such a tool.

[tpoechtrager]

I have added your method to the README (but plainly marked it as unsupported 😄).

[BioMike]

Thank you!

[BioMike]

I found a new and more elegant "unsupported" method. This one uses https://github.com/LubosD/darling-dmg, which allows compressed DMG files to be mounted through FUSE. This one doesn't add dependencies for java (but it does for FUSE though) and doesn't give nagging pop-ups about inaccessible files (there is however one bug report about inaccessable files/directories). I'm going to give this method a try.

[tpoechtrager]

darling-dmg seems to be the best method by far.

I will add a script for it once I am sure it really works the way it should.

I have marked the hfsexplorer method "unsupported" because the chance to end up with a working SDK isn't very high...

[geoff-nixon]

7z x *.dmg "*/*/*/*/*/Mac*/*/*/*.sdk"; mv $(find . -name \*.sdk) .; rm -rf Xcode *.sdk/*/*/man
# ... or whatnot.

With p7zip 9.38.1.
YMMV with 9.20.1 - it's a little clunkier, and chokes on larger DMGs for me.

[tpoechtrager]

p7zip 9.38.1 seems to have problems with xcode 6.3.2:

testing o32-clang ... ld: file too small (length=17) file '[...]/SDK/MacOSX10.10.sdk/usr/lib/libSystem.dylib' for architecture i386
clang-3.6: error: linker command failed with exit code 1 (use -v to see invocation)

[zarvox]

There's also libdmg-hfsplus which requires neither Java nor FUSE nor privilege escalation. My fork of libdmg-hfsplus supports bzip2-compressed DMGs, extracting symbolic links, and fixes some memory leaks, which most of the others in the libdmg-hfsplus repo graph don't.

I'll see if I can get a working SDK and write up the appropriate instructions.

[zarvox]

Hmm, I appear to have spoken too soon - looks like libdmg-hfsplus doesn't support HFS+ compression yet. I'll poke at making that work.

[BioMike]

I tried the p7zip option as well, didn't work for me as well.

@zarvox, might be good to collaborate with the darling team? (seems to be a one man show for now) to have a uniform lib and a FUSE FS access and the extraction tool that both use it. The FUSE FS access has currently still some problems (about accessing some files/dirs) that might be worthwile to investigate into.

[geoff-nixon]

@BioMike @tpoechtrager
Sorry, (I never realized this before) but it looks like p7zip doesn't create proper symlinks, it creates a text file with the target of the symlink. But it does handle HFS+ compression!

Try something like this script on for size:

#!/bin/sh
here=$PWD
7z x "$1" "*/*/*/*/*/Mac*/*/*/*.sdk"
mv $(find . -name \*.sdk) .
rm -rf Xcode */usr/share/man
find . -empty -delete

for symlink in $(find . -type f -size -1k); do
  if [ $(wc -l < "$symlink") = 0 ]; then
    target=$(cat "$symlink")

    link=$(basename "$symlink")
    cd "$(dirname "$symlink")"
    [ -e "$target" ] && rm "$(basename "$link")" && ln -s "$target" "$link"
  fi
  cd "$here"
done

[tpoechtrager]

@geoff-codes
I have tried a smiliar way yesterday and ran into lots of problems...

function fix_symlink() {
  dst=$(cat $1)
  [ $(echo "$dst" | wc -l | tr -d ' ') -ne 1 ] && return
  ([[ "$dst" == *\ * ]] || [[ "$dst" == \#* ]]) && return
  dst=$(cat $1)
  echo "fixing symlink: $1 -> $dst"
  rm $1 && ln -s $dst $1
}

export -f fix_symlink

find $TMP -type f -size -4096c -exec \
  /usr/bin/env bash -c 'fix_symlink "$0"' {} \;

I have now modified p7zip instead, not very elegant, but it seems to work...
Luckily p7zip doesn't depend on any libraries, so a C/C++ compiler is all you need (more or less).

@zarvox
I have already tried libdmg-hfsplus a while ago and it didn't work out so well.

However, hdutil indeed supports hfs+ (edit: never mind: overread "compression"),
but I am getting an Input/Output error rather quickly.

[zarvox]

@tpoechtrager I confirm success using your patched p7zip to produce working SDK tarballs, so I think that's probably the lowest-friction path forward for users.

git clone https://github.com/tpoechtrager/p7zip
cd p7zip && make 7z && cd ..
./p7zip/bin/7z x /path/to/Xcode_6.3.2.dmg
XCODEDIR=./Xcode ./tools/gen_sdk_package.sh

[maci0]

+1
Great to see this progressing!

[fdelapena]

Thank you for the patched p7zip, our building bot VPS uses OpenVZ and the FUSE module for the darling-dmg method is not available because the host exposed kernel does not have it enabled and can't be installed from the guest. The new method helped a lot 👍.

[Godin]

@tpoechtrager Thank you for the patched p7zip - works like a charm! Any chance that it will be accepted into upstream project eventually?

[tpoechtrager]

@Godin: You are welcome. I am not sure whether they would accept such a patch, my implementation is just a quick & dirty hack for OSXCross 😄. Try to create a feature request ticket @ http://sourceforge.net/p/p7zip/feature-requests/?source=navbar and maybe point them to my p7zip repository - so they don't have to start from zero.