[Request] Add Qt5 about extras

[tresf]

Some recommendations on cracking open the executable are welcome. 7zip only provides a Docs folder. Chocolatey has it marked as non-silent.

Update: https://sourceforge.net/p/sevenzip/bugs/1617/

[tresf]

Current work in progress for extraction:

• qt5.json
• extract-qt.ps1

Installing 'qt5' (5.8.0).
Loading qt-opensource-windows-x86-msvc2015_64-5.8.0.exe from cache
extract-qt.ps1 (895 B) [========================================] 100%
Checking hash of qt-opensource-windows-x86-msvc2015_64-5.8.0.exe... ok.
Warning: No hash in manifest. SHA256 is:
    bb3bf3540e3f4af3856abdbcea6d6461cec97f15889afc5da4b6d13e3346e9aa
Step 1: Extracting nested [0] file... done
Step 2: Extracting nested .7z archives... done
Step 3: Installing files (this could take a while)...

[lukesampson]

We might need some help with this one from someone who knows what a working installation would look like (I don't, sorry).

Would you be willing to submit a pull request? It would be much appreciated. You can use the manifest reference from Scoop.

One tip you might not be aware of is that you can sometimes extract files from an .exe using 7-zip, instead of running the .exe as an installer (preferable if possible). If you add a fragment like #/dl.7z to the end of a download URL, Scoop will download it using that filename and attempt to extract the contents.

It looks like the visualc manifest we have is version 2010—is that going to work with Qt5?

[tresf]

One tip you might not be aware of is that you can sometimes extract files from an .exe using 7-zip, instead of running the .exe as an installer

Well aware! 👍

(preferable if possible). If you add a fragment like #/dl.7z to the end of a download URL, Scoop will download it using that filename and attempt to extract the contents.

Oh! 🎉

It looks like the visualc manifest we have is version 2010—is that going to work with Qt5?

Probably but probably not on the machine I'm staging.

I was thoroughly impressed when cmake picked up cl.exe, so I think it's a matter of setting up the environment. I'm not a C++ developer by trade, but I need it for two projects.

The one thing I'm not sure about yet is how scoop stashes it's headers. POSIX does the whole /include thing, but I'm new to the Windows side.

QtCreator offers both mingw and MSVC++ versions and for two major MSVC++ releases and architectures (32/64) so the support is there, just a matter of configuring it.

We might need some help with this one from someone who knows what a working installation would look like (I don't, sorry).

This is where I hope I can help. https://github.com/LMMS/lmms/issues/3447#issuecomment-288430052 for context. (I'd rather not cross-link! 😈 )

[tresf]

It looks like the visualc manifest we have is version 2010—is that going to work with Qt5?

Probably but probably not on the machine I'm staging.

I stand corrected. From the downloads page... it appears only mingw, 2013 and 2015 are supported.

[tresf]

if you add a fragment like #/dl.7z to the end of a download URL, Scoop will download it using that filename and attempt to extract the contents.

Can we append this to wiki/App-Manifests#required-properties then?

[tresf]

Next question... I'm sure I'm doing some very simply wrong... any insight?

qt-opensource-windows-x86-msvc2015_64-5.8.0.exe (1.0 GB) [=========================] 100%
Warning: No hash in manifest. SHA256 is:
    f013e485b1e95cfce26a4d24f33c154b2f3289af6c3a73ca5a46dc010e5d0092
Extracting... Error moving directory:
 -------------------------------------------------------------------------------
ROBOCOPY     ::     Robust File Copy for Windows
-------------------------------------------------------------------------------    Started
: Wednesday, March 22, 2017 2:23:55 PM    Source :
C:\Users\Administrator\scoop\apps\qt\5.8.0\_scoop_extract\qt\      Dest :
C:\Users\Administrator\scoop\apps\qt\5.8.0\      Files : *.*           Options : *.* /S /E
/DCOPY:DA /COPY:DAT /MOVE /R:1000000 /W:30
------------------------------------------------------------------------------  2017/03/22
14:23:55 ERROR 2 (0x00000002) Accessing Source Directory
C:\Users\Administrator\scoop\apps\qt\5.8.0\_scoop_extract\qt\ The system cannot find the
file specified.
At C:\Users\Administrator\scoop\apps\scoop\current\lib\core.ps1:137 char:9
+         throw "Error moving directory: `n$out"
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Error moving di...ile specified. :String)
   [], RuntimeException
    + FullyQualifiedErrorId : Error moving directory:
     -------------------------------------------------------------------------------    ROB
   OCOPY     ::     Robust File Copy for Windows                               -----------
  --------------------------------------------------------------------    Started : Wedne
 sday, March 22, 2017 2:23:55 PM    Source : C:\Users\Administrator\scoop\apps\qt\5.8.0\
_scoop_extract\qt\      Dest : C:\Users\Administrator\scoop\apps\qt\5.8.0\      Files :
*.*            Options : *.* /S /E /DCOPY:DA /COPY:DAT /MOVE /R:1000000 /W:30   ---------
---------------------------------------------------------------------  2017/03/22 14:23
:55 ERROR 2 (0x00000002) Accessing Source Directory C:\Users\Administrator\scoop\apps\q
t\5.8.0\_scoop_extract\qt\ The system cannot find the file specified.

JSON:

{
   "version": "5.8.0",
   "url": "https://download.qt.io/archive/qt/5.8/5.8.0/qt-opensource-windows-x86-msvc2015_64-5.8.0.exe#/dl.7z",
   "hash": "f013e485b1e95cfce26a4d24f33c154b2f3289af6c3a73ca5a46dc010e5d0092",
   "extract_dir": "qt",
   "bin": "qt.ps1"
}

[lukesampson]

It looks like the "qt" directory doesn't exist in the archive. Maybe try without the "extract_dir" property?

[tresf]

@lukesampson thanks, trying that now. Is there a reason it didn't cache the 1GB download?

[lukesampson]

Ouch. It doesn't cache if the hash check fails — could that be it?

[tresf]

Acceptable. :). Added. Re-dling.

[lukesampson]

Also, I have added some documentation about the #/dl.7z technique to App Manifests.

[tresf]

Ok, I have the files extracting...

# Extract the [0] file from exe
7z x qt-installer.exe -t* -i![0] -y

# Extract only embedded 7z files from [0]
7z x [0] -t`# -i!*.7z -y

# Extract content from all 7z files
7z x 2.7z -ai!*.7z -y

# .. delete the files left behind, etc

This is a very (very) long process. I'm not sure if it's an inefficiency in 7zip or something related to a slowness in my terminal (stdout was on). Anyway, my next steps are wrapping this into a .json file so that I can have a working 5.8 directory.

In the case of this package, all executables are in 5.8\msvc2015_64\bin. Can someone steer me in the direction of:

1. Customize the extraction process via .json (embed a script, whatever)
2. Alias the bin executables so they're recognized commands
3. Write a file called qt.conf side-by-side qmake.exe with the following contents:

   [Paths]
   Prefix = C:\\Users\\Administrator\\scoop\\ <...> \\qt\\5.8\\msvc2015_64\\bin

My test bed is going to be qmake.exe which is used to direct Qt how to set up its build environment. Once that's working, we can determine how to handle the different architectures and MSVC++/mingw environments.

If configured properly, qmake -query should return something like this:

QT_SYSROOT:C:\Users\Administrator\scoop\<apps>\qt\5.8\msvc2015_64
QT_INSTALL_PREFIX:C:/Users/Administrator/scoop/<apps>/qt/5.8/msvc2015_64/bin
[...]
QMAKE_SPEC:win32-msvc2015
QT_VERSION:5.8.0

[lukesampson]

1. To do a custom extraction, you can provide PowerShell commands in either pre_install or post_install in the manifest (see reference).
2. Use bin in the manifest to target the executables you want to make available.
3. You can use post_install to output a qt.conf file. You can use the $dir variable in your post_install to get the path of the install directory.

If extracting with 7-zip turns out to be too slow, you can invoke the installer directly if that's faster. E.g. the python manifest does this, although it's not ideal.

[tresf]

To do a custom extraction, you can provide PowerShell commands in either pre_install or post_install in the manifest (see reference).

Would you be opposed to a decompress::extract_qtifw($path, $to) (qt installer framework format)?

Without this, each qt package (vs2013, vs2015, mingw) will need to have the 7z [...]; 7z [...]; 7z [...]; 7z [...] logic over and over in pre_install. I have a prototype 95% working. Will link a commit if OK.

Use bin in the manifest to target the executables you want to make available.

Got it. I found the "bin": [ "/foo/bar.exe", "/lorem/ipsum.exe"] perusing.

You can use post_install to output a qt.conf file. You can use the $dir variable in your post_install to get the path of the install directory.

Perfect.

[tresf]

If extracting with 7-zip turns out to be too slow, you can invoke the installer directly if that's faster.

If Qt provided an unattended method; struggling to find one.

[lukesampson]

Oh okay. Wow, that installer looks over-engineered.

I'd prefer not to add a custom extract method to Scoop if that's what you mean, but feel free to define this function in pre/post_install.

If it's a long function, you might also like to put it in a file and include it as a url in your manifest to download alongside the installer—linking to the raw version of GitHub gists is a quick way to do that, and you can share this file among the various QT manifests.

[tresf]

feel free to define this function in pre/post_install

Well, that still suffers the copy/paste on every manifest problem.

you might also like to put it in a file and include it as a url in your manifest to download alongside the installer

Sure, but I'd much rather not have direct ties to my personal gist. Sounds fragile.

My major concern is the extraction logic. Does the installer allow an extraction stub; let manifest take over?

• Here's the install shim: https://gist.github.com/tresf/9a3a476ef1edaadefbddb77db719224b
• Here's the function: https://gist.github.com/tresf/f884e40808d477ed044f0f616ab3b82a

Downloading a 3rd party script is OK too, but I think it will be slightly more difficult to shim in at install time. Do any other packages do exactly this?

[tresf]

@lukesampson my only argument for making this a bit more adoptable is that Qt is a very popular C++ development platform and the installer framework they use appears to be open source and recommended by them.

That said, I've never spotted it before in the wild outside of Qt's website. (Edit: Msys2 uses it.) I'm much more accustomed to the InstallShield|NSIS|WISE|MSI|etc's, but it may be something worthwhile of inclusion.

[lukesampson]

See visualc for an example of using a GitHub gist to support the install. Alternatively, you could include it in the scripts directory of scoop-extras with your pull request, but this is harder to test since the script won't be there until the PR is merged. I find Gists easy and reliable for these purposes, and infinitely more transparent to users than custom-made installers.

Yeah I haven't seen Qt's installer before either. If it does reach wider adoption then I'd be happy to include some specific functionality for it then. We're trying to solve the 80% case with Scoop, with workarounds for the programs that have complicated installs. Qt seems to be the latter. It's a hassle, but it can be done and it looks like you're making good progress.

[tresf]

Edit: This came in after the visualc link. Snippet removed, will attempt that method.

[tresf]

Yeah I haven't seen Qt's installer before either. If it does reach wider adoption then I'd be happy to include some specific functionality for it then.

FWIW, It appears the msys2 project has adopted the QtIFW as well.

In a lot of ways, msys2 is a direct "competitor" for scoop, since it ships its own shell and package manager, but since I spotted another in the wild, I wanted to make it known. If I see another large project using it, I'll open a separate bug report on it.

Qt5 + scoop support has stalled temporarily. Will update once more progress happens.

[tresf]

Side note.... the unattended method with QtIFW seems to be available, but it introduces other problems making 7z more attractive in the long run.

Have you guys read this http://doc.qt.io/qtinstallerframework/noninteractive.html

[tresf]

@nueko per https://github.com/msys2/msys2/issues/18, that won't work without OpenGL, which will break on Server Core 2016. This is probably an upstream bug with QtIFW.

This comment explains that I was unable to leverage the noninteractive installer on Server Core 2016: https://github.com/msys2/msys2/issues/18#issuecomment-298237877

[mingwandroid]

In a lot of ways, msys2 is a direct "competitor" for scoop

Not at all, msys2 is all about Open Source, software freedom and compiling from source in a repeatable way to build a community around those things on Windows.

Scoop is not.

[Milerius]

any news about that ?

[tresf]

@Milerius no, I gave up on the effort for now. Getting a proper C++ environment required quite a bit of work, especially handing the various Microsoft compilers. Anyone's free to take this objective over.

[Milerius]

@tresf Can we try with github actions ?

[tresf]

@tresf Can we try with github actions ?

I'm not sure what you mean by GitHub actions. If you're talking about using GitHub as a project management tool, that's up to the maintainers.

[Milerius]

You right. ^ Thank's for the information