3f / getnutool

Embeddable Package Manager (+core in .bat); 🕊 Lightweight tool to Create or Distribute using basic shell scripts (no powershell no dotnet-cli)

GetNuTool

Embeddable Package Manager (+core in .bat); 🕊 Lightweight tool to Create or Distribute using basic shell scripts (does not require powershell or dotnet-cli); NuGet / Chocolatey client;

Copyright (c) 2015-2024  Denis Kuzmin <[e-mail]> github/3F

「 ❤ 」

gnt Fnv1a128                                  # Get Fnv1a128 package
gnt /t:pack /p:ngin="bin\DllExport"           # Create new DllExport package
gnt "Conari;regXwild"                         # Get Conari & regXwild packages
msbuild gnt.core /p:ngpackages=LuNari/1.6.0   # Use msbuild to get LuNari 1.6.0 
gnt Huid/1.0.0:src.zip /t:grab                # Grab Huid 1.0 as zip without unpacking
gnt /p:ngconfig="dir\packages.config"         # Use specified packages.config
set ngpackages=Conari & gnt                   # shell scripts
gnt /p:ngpackages=putty.portable/0.69         # chocolatey
    /p:ngserver="https://chocolatey.org/api/v2/package/"

Download all editions: Core, Minified, Executable, C# version for .NET, ...

Official Direct Links:

• (Windows) Latest stable compiled batch-script [ gnt.bat ] https://3F.github.io/GetNuTool/releases/latest/gnt/

Projects based entirely on GetNuTool

• netfx4sdk - Developer Pack (SDK). NETFX 4: Visual Studio 2022+ / MSBuild 17+ / or other modern tools;
• hMSBuild - Compiled text-based embeddable pure batch-scripts for searching modern MSBuild instances;
• .NET DllExport Manager - Part of the DllExport tool that provides its completely independent management and distribution beyond the NuGet ecosystem. Relies on MvsSln as well;

Why GetNuTool

> gnt                                      # via single shell script
> msbuild gnt.core                         # via MSBuild engine
[NuGet gnt.raw("/t:pack /p:ngin=7z.Libs")] # via SobaScript
new GetNuTool()                            # .NET(netfx 4.0+, .NET 5+, .NET Core, Mono) C#

Lightweight Portable and completely Embeddable (+core in .bat) package tool to create or distribute everything from anything.

Back in those days it was originally developed as an alternative to solution level limitations, i.e. github.com/NuGet/Home/issues/1521. In attempt to provide a tool to easily maintain projects, libraries, and other related build processes; For all projects at once at the solution level and of course for each separately if necessary.

For example, it was designed to be more friendly to such NuGet packages:

• vsSolutionBuildEvent (before, aka vsSBE.CI.MSBuild)
• 7z.Libs
• DllExport
• ILAsm

However! GetNuTool has more powerful ways even for standard NuGet packages providing a wider range of use cases.

• Conari, Fnv1a128, MvsSln, regXwild, ...

Key Features

• Install .nupkg packages from remote NuGet (or like: chocolatey, ...) servers.
• Grab or Install any zipped packages from direct sources (local, remote http, https, ftp, ...).
• Controlled unpacking of all received packages. Modes: get or grab
• Hash values control using sha1 for receiving every package if used unsecured channels (~windows xp) etc.
• Creating new NuGet packages .nupkg from .nuspec.
• Two supported formats: xml packages.config (+extra: output, sha1) and inline records.
• Inline records and packages.config are fully compatible between, and config has backward compatibility with original packages.config
• Configurable custom folders for every receiving.
• Request to the server only if the package is not installed.
• Supports proxy with custom credential.
• Default settings are overridden through an environment variables: default .config files, NuGet server, etc.
• The ability to create one click ~8 KB .bat wrappers for any packages. Try for example vsSolutionBuildEvent.1.16.bat
• Easy integration into any scripts such as pure batch-script netfx4sdk, DllExport, hMSBuild
• C# projects support via GetNuTool.cs

Note:

• Dependencies are not considered.
• Doesn't manage projects or solutions files. You need MvsSln or anything else together with.
• NuGet events (Init.ps1, Install.ps1, Uninstall.ps1) from /tools will not be launched automatically.
  • To use events (from Visual Studio, MSBuild, NuGet, etc.), you can with vsSolutionBuildEvent, or vsCommandEvent, or configure it like DllExport project, or command it manually.

tModes and Commands

get

For receiving packages or zipped files from local or remote source, then installing / extracting.

The get is used by default but you can also specify /t:get

Property	Description	Default values
ngconfig	Define .config files.	1.9+ packages.config;.tools\packages.config
ngserver	Define server.	1.0+ https://www.nuget.org/api/v2/package/
ngpackages	List of packages. Disables ngconfig if specified.
ngpath	Common path for all packages.	1.0+ packages
wpath	1.4+ To change working directory.	1.4+ (The absolute path of the directory where the GetNuTool is located)
proxycfg	1.6.2+ To configure connection via proxy.
ssl3	1.9+ Do not drop legacy ssl3, tls1.0, tls1.1 if true.
break	1.9+ Disable the break on first package error if no

Package List Format

Attribute	Description	Example
id	Identifier of package.	Conari
version	(Optional) Package version.	1.5.0 or 1.5-beta2 or 1.5-RC etc.
output	(Optional) Where to store package data.	../tests/case1
sha1	(Optional) Expected sha1 for this package.	eead8f5c1fdff2abd4da7d799fbbe694d392c792

Note:

• As of version 1.2+, attributes are now case sensitive. Please use lowercase for id, version, output, sha1.
• It will link to the latest available version if version attribute is not defined.
• output attribute is relative to ngpath. You can also use absolute path.
• sha1 activates security check before every install / extracting. It useful when connection is not secure like on windows xp with obsolete ciphers. But please note: some servers (like official NuGet) may repackage .nupkg for some purposes, such as adding .signature.p7s etc. This of course changes sha1 hash value that you need to check.
• id allows only a-z A-Z 0-9 . - _ symbols without whitespaces.

ngpackages

id[/version][?sha1][:path];id2[/version][?sha1][:path];...

delimiters:

• ; 1.6+ Name1;Name2;Name3
• | 1.0+ Name1|Name2|Name3

/p:ngpackages=Name1
/p:ngpackages="Name1;Name2"

packages.config

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Name1" version="1.5.0" />
  <package id="Name2" output="path" />
  <package id="Name3" version="2.2-RC" sha1="eead8f5c1fdff2abd4da7d799fbbe694d392c792" />
</packages>

/p:ngconfig=".nuget/packages.config"
/p:ngconfig="debug.config;release.config;..."

Proxy Format

[usr[:pwd]@]host[:port]

/p:proxycfg="example.com:4321"

grab

1.9+

Grabs data without unpacking. The available parameters are similar to the get above.

For example:

gnt Huid/1.0.0:src.nupkg /t:grab

gnt :../netfx4sdk.cmd /t:grab /p:ngserver=https://server/netfx4sdk.cmd

msbuild gnt.core /t:grab /p:ngpackages=Fnv1a128:src.nupkg

gnt Huid/1.0.0?24ecda9a4fb8920067df31788d3dea708996ab08:src.nupkg /t:grab

pack

Creates the new .nupkg packages using .nuspec specification. Can also be found in the package folder after using get.

Property	Description	Default values
ngin	Path to directory where .nuspec to create package from it.
ngout	Optional path to save the final .nupkg package in other place.
wpath	1.4+ To change working directory.	1.4+ (The absolute path of the directory where the GetNuTool is located)

gnt /t:pack /p:ngin=path\to\dir
gnt /t:pack /p:ngin="path to\dir";ngout=..\dst.nupkg

Global Properties

Property	Value
debug	true to add extra info in stream.
logo	no to hide logo when processing starts.

For example:

gnt /t:pack /p:ngin=packages\LX4Cnh /p:debug=true

set debug=true & gnt /p:ngpackages="Conari;LX4cn;Fnv1a128";break=no

Extra Editions

Compiled. SobaScript component

GetNuTool now is part of NuGetComponent

#[NuGet gnt.raw("/t:pack /p:ngin=D:\7z.Libs")]

C# GetNuTool.cs

GetNuTool.cs includes a fully compatible gnt.core inside.

Add GetNuTool.cs to your project (.NET Framework 4.0+, .NET 5+, Mono, .NET Core, .NET Standard), then command like:

var gnt = new net.r_eg.GetNuTool();
gnt.Run(ngpackages: "Conari;regXwild");
...
gnt.Run(ngpackages: "Fnv1a128");

Batch (.bat) version

~8 KB gnt.bat includes a fully compatible gnt.core inside.

gnt.bat extra arguments

First key to gnt.bat	Description	Example
...	1.9+ alias to ngpackages=...	gnt Conari, gnt "regXwild;Fnv1a128"
-unpack	1.6+ To generate minified gnt.core from gnt.bat.	gnt -unpack
-msbuild path	1.6 - 1.8 To use specific msbuild. Removed in 1.9. Override engine instead	gnt -msbuild "D:\MSBuild\bin\amd64\msbuild" /p:ngpackages=Conari

Override engine

Since the -msbuild key was removed in 1.9 as obsolete, you have following ways to override engine search:

Either create msb.gnt.cmd in the directory from which the call gnt.bat is planned; with the following content, for example:

@echo msbuild.exe

Or place a full version of hMSBuild.bat (~19 KB) tool instead of msb.gnt.cmd stub.

Or -unpack command:

gnt -unpack & msbuild.exe gnt.core {args}

Or msb.gnt.cmd environment variable:

set msb.gnt.cmd=msbuild.exe & gnt {args}

Examples

gnt vsSolutionBuildEvent/1.16.1:../SDK & SDK\GUI

gnt Conari
gnt /p:ngpackages=Conari
msbuild gnt.core /p:ngpackages=Conari

gnt "Conari;regXwild;MvsSln"
gnt /p:ngpackages="Conari;regXwild;MvsSln"

gnt Fnv1a128
gnt /t:pack /p:ngin=packages/Fnv1a128

msbuild gnt.core /p:ngconfig=".nuget/packages.config";ngpath="../packages"

gnt Conari /p:proxycfg="guest:[email protected]:7428"

set ngpackages=Conari & call gnt || echo Failed

gnt "7z.Libs;vsSolutionBuildEvent/1.16.1:../packages/SDK"

Direct link to a remote package via https:

gnt :DllExport /p:ngserver=https://server/DllExport.1.7.4.nupkg

Direct link to a local package that will be stored at the top level next to the gnt

gnt : /p:ngserver=D:/local_dir/vsSolutionBuildEvent.SDK10.nupkg /p:ngpath=SDK10

gnt "Conari;Fnv1a12;LX4Cnh" /p:break=no /p:debug=true

Conari use D:\prg\projects\GetNuTool\GetNuTool\bin\Release\packages\Conari
Fnv1a12 ... The remote server returned an error: (404) Not Found.
LX4Cnh ... D:\prg\projects\GetNuTool\GetNuTool\bin\Release\packages\LX4Cnh
- /.version
- /build-info.txt
- /lib/net40/LX4Cnh.dll
- /lib/net40/LX4Cnh.xml
- /lib/net5.0/LX4Cnh.dll
- /lib/net5.0/LX4Cnh.xml
- /License.txt
- /LX4Cnh.nuspec
- /Readme.md
- /tools/gnt.bat
- /tools/hMSBuild.bat
...

gnt Huid/1.0.0:src.zip /t:grab

See also examples from tests/

Build and Use from source

git clone https://github.com/3F/GetNuTool.git src
cd src & build & bin\Release\gnt Conari

.sha1 official distribution

GetNuTool releases are now accompanied by a .sha1 file in the official distribution; At the same time, commits from which releases are published are signed with the committer's verified signature (GPG).

Make sure you are using official, unmodified, safe versions.

Note: .sha1 file is a text list of published files with checksums in the format:

40-hexadecimal-digits <space> file

eead8f5c1fdff2abd4da7d799fbbe694d392c792 path\file
...

Contributing

GetNuTool is waiting for your awesome contributions!

getnutool's Issues

Configuring proxy

We need some proxy settings.

~
keys for:

• server
• login/pass

Support direct links to packages

Basically, DllExport Manager is already contains the hack of this, like:

    :: TODO: hack for GNT v1.6.1
    if defined pkgLink (
        set pkgSrv=!pkgLink!
        set "_remoteUrl=:../!wPkgPath!"
    )
   ...

But we need to support this by our core for other related purpose.

MSB1001: Unknown switch -notamd64

For something like this - build.bat:

set msbuild=tools/msbuild


call gnt /p:ngconfig="packages.config" /nologo /v:m /m:4 || goto err
call %msbuild% -notamd64 "Conari.sln" /v:normal /l:"packages\vsSBE.CI.MSBuild\bin\CI.MSBuild.dll" /m:4 /t:Rebuild /p:Configuration=Release || goto err


goto exit

:err

echo. Build failed. 1>&2

:exit

>build

  GetNuTool v1.6 - github.com/3F
  =========

Getting `vsSBE.CI.MSBuild/1.6.12010` ... Extracting into `D:\prg\projects\Conari\Conari\packages\vsSBE.CI.MSBuild`

:: ! incorrect calling of original msbuild.exe:

>>> Microsoft (R) Build Engine version 14.0.25420.1            
>>> Copyright (C) Microsoft Corporation. All rights reserved.  
>>>                                                          
>>> MSBUILD : error MSB1001: Unknown switch.                 
>>> Switch: -notamd64                                        
>>>                                                          
>>> For switch syntax, type "MSBuild /help"                  
>>>  Build failed.                                           

the reason:

set msbuild="%msbuild%"

embedded/exec.tpl and msbuild-helper

TODO

msbuild & gnt.core wrappers

here, I added simple bat that will search installed MSBuild and will start build operations. So first, add same caller for this repo.

But also, I want some non-binary-wrapper of all this, i.e. gnt.core + msbuild to work directly like with one executable gnt tool.

powershell would be good, but... try new puzzle with native batch if it's possible at all -_-

Corrupted data when the compression method 0x08 -> 0x00 / compressed size info [02 00 00 00]

Possible bug with packages was described here:
3F/DllExport#38 (comment)

• Compression method: Deflate (0x08) -> Uncompressed (0x00)
• Actual uncompressed size == 0
• Compressed size == 2

GetNuTool processes packages through System.IO.Packaging.ZipPackage and for this data we can't use this at all:

source.CopyTo(Stream destination);
source.Read(byte[] buffer, int offset, int count);
(new StreamReader(source)).Read(char[] buffer, int index, int count)
(new StreamReader(source)).ReadToEnd();
...

all above will throw exception because of corrupted data and/or incorrect internal processing.

btw:

 // source.Length == 2 
 source.Seek(0, SeekOrigin.Begin);
 source.ReadByte(); // 0x03
 source.ReadByte(); // Exception

 source.Seek(-1, SeekOrigin.End);
 source.ReadByte(); // 0x00
 source.ReadByte(); // -1  end of the stream

 source.Seek(-2, SeekOrigin.End);
 source.ReadByte(); // 0x03
 source.ReadByte(); // Exception

Through ZipPackage implementation we also have no any information about position from file outside processed part. Or does it available ? o_O

Because for this case we can process this manually:

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00006850  55 9A DB D3 4F B0  FE 0F 50 4B  03 04 14 00  02 00    UљЫУO°ю.PK......
00006860 [00 00]21 B2 0C 4B [00 00 00 00][02 00 00 00] 00 00    ..!І.K..........
00006870  00 00 0E 00 1C 00  74 6F 6F 6C  73 2F 69 6E  69 74    ......tools/init
00006880  2E 70 73 31 20 A2  18 00 28 A0  14 00 00 00  00 00    .ps1 ў..( ......
00006890  00 00 00 00 00 00  00 00 00 00  00 00 00 00  00 00    ................

other header:

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00003F80  79 F0  E4 37 50 4B  03 04 14 00  02 00 [00 00] 21 B2  yрд7PK........!І
00003F90  0C 4B [00 00 00 00][02 00 00 00] 00 00 00 00 0D 00  .K..............
00003FA0  1C 00  6C 69 62 2F  6E 65 74 32  30 2F 5F 2E 5F 20  ..lib/net20/_._ 
00003FB0  A2 18  00 28 A0 14  00 00 00 00  00 00 00 00 00 00  ў..( ...........

that is, from headers above we have information about uncompressed type data, with compressed size == 2, and actual 0 bytes.

specific exception because of incorrect crc:

• MS.Internal.IO.Zip.ProgressiveCrcCalculatingStream.Read(Byte[] buffer, Int32 offset, Int32 count)

...
if (this._validateCrcWithExpectedCrc && this.CanValidateCrcWithoutRead() && this.CrcCalculator.Crc != this._expectedCrc)
{
	throw new FileFormatException(SR.Get("CorruptedData"));
}

well, seems try/catch is the most easy way to process this case, because it's bug of the System.IO.Packaging.ZipPackage and from nuget.org. Details here: 3F/DllExport#38

Any suggestions are welcome.

Compressed executable version

Try to prepare compressed executable version through DeflateStream/GZipStream.

• ~ compressed byte sequence -> base64 -> new bootloader for decoding data before executing from wrapper

Support for special symbols in path: ' &!~`@#$^(+)_=%-;[.]{,}

So... it continues 3F/DllExport#88 (comment)

The following possible chars in paths ' &!~`@#$^(+)_=%-;[.]{,} may cause some problems for use in third party components, however this is still possible to escape all of this. See this changes: 3F/DllExport@d4b46cd

But looks like not for ;

Initially GetNuTool (2015) uses same compilation of list of packages for different types:

1. packages.config file that was designed for compatibility with original NuGet clients. (today's nuget have priorities for PackageReference only). But anyway, I personally used mostly command-line arguments instead of this. So I'm not sure about the future of this file also for current project. But this is other story.
2. Env and msbuild properties via something like set ngpackages=... or /p:ngpackages="..." ...

Both of this encodes same syntax like:

• id[/version][:output]|id2[/version][:output]|...
• id[/version][:output];id2[/version][:output];... (; introduced in 1.6 because | requires escaping in some places for batch scripts)

But while | requires escaping in command-line usage, the truth is that this never be allowed in path in windows.

Okay, what we have today:

• id/1.6.0:output;id2;id3:output
• id/1.6.0:output|id2|id3:output

Where output may contain absolute path like id/1.6.0:D:\dir\name\;... (it's also ok for latest versions).

Thus, for current stage we need to support only ; without changing syntax as possible for both cases above.

The possible way

💡 Escaping ; -> ;;.

• This is not so good option when this is used through variables, e.g.:
• • id/1.6.0:out;;put;id2;id3:%dir:;=;;%

💡 Leave only |.

• It may require escaping like ^| as I said above. And for some usage it may also require escape ^ too, e.g.:
• • id/1.6.0:output^|id2^|id3:withcaret^^|
• • id/1.6.0:output^^|id2^^|id3:withcaret^^^^|

💡 Empty delimiter ;. This is vice versa logic to first variant. When is empty data ;; like nothing to be parsed:

• id/1.6.0:out;put;;id2;id3:%dir%;; a single ; after id2 because it should be only when ':' is presented.
• Too hard in implementing because of possible double semicolon in path, like "my;;name"

💡 Ignore ; only when | is found from ::

• id/1.6.0:output;id2;id3:%dir%
• id/1.6.0:out;put|id2;id3:%dir%|
• id/1.6.0:out;put^|id2;id3:%dir%^| ....

💡 All above.

Need to think.

feedback

Anyone who also uses this directly as a tool (i.e. not as dependencies from 3rd projects), please leave here your comment.