Support 'unar' instead of requiring various unpackers (e.g. xz) about zinit

[vladdoster]

I just stumbled upon this rust utility you might like.

https://github.com/ouch-org/ouch

[docwhat]

What I don't like about that idea though is that while you mention unar, there's a ton of alternatives. What $fancy_archive_extractor should we support?

It's true. There are a lot.

zi could just fetch unar (or some other unpacker) for its own use instead of depending on the system's unpacker.

borking the install and requiring a zi delete.

What exactly do you mean by that?

I don't remember exactly anymore.

I think I had unar but didn't have xz. The unpack failed, and zi didn't know since there was a directory present.

Fixing it required figuring out what happened, installing xz, and deleting the package so it could reinstall.

[alichtman]

Can you show me something that requires an unpacker?

[docwhat]

Note my has'xz' that I just added to prevent it borking the install and requiring a zi delete.

zinit wait lucid for \
  from'gh-r' \
  as'program' \
  has'xz' \
  mv'shellcheck* -> shellcheck' \
  pick'shellcheck/shellcheck' \
  @koalaman/shellcheck

[pschmitt]

I can see this within zinit's core.

If there's $fancy_archive_extractor, then use it.

ziextract should be relatively easy to patch:

zinit/zinit-install.zsh

Lines 1526 to 1827 in 5df5fb6

	ziextract() {
	emulate -LR zsh
	setopt extendedglob typesetsilent noshortloops # warncreateglobal
	local -a opt_move opt_move2 opt_norm opt_auto opt_nobkp
	zparseopts -D -E -move=opt_move -move2=opt_move2 -norm=opt_norm \
	-auto=opt_auto -nobkp=opt_nobkp || \
	{ +zinit-message "{error}ziextract:{msg2} Incorrect options given to" \
	"\`{pre}ziextract{msg2}' (available are: {meta}--auto{msg2}," \
	"{meta}--move{msg2}, {meta}--move2{msg2}, {meta}--norm{msg2}," \
	"{meta}--nobkp{msg2}).{rst}"; return 1; }
	local file="$1" ext="$2"
	integer move=${${${(M)${#opt_move}:#0}:+0}:-1} \
	move2=${${${(M)${#opt_move2}:#0}:+0}:-1} \
	norm=${${${(M)${#opt_norm}:#0}:+0}:-1} \
	auto=${${${(M)${#opt_auto}:#0}:+0}:-1} \
	nobkp=${${${(M)${#opt_nobkp}:#0}:+0}:-1}
	if (( auto )) {
	# First try known file extensions
	local -a files
	integer ret_val
	files=( (#i)**/*.(zip|rar|7z|tgz|tbz2|tar.gz|tar.bz2|tar.7z|txz|tar.xz|gz|xz|tar|dmg|exe)~(*/*|.(_backup|git))/*(-.DN) )
	for file ( $files ) {
	ziextract "$file" $opt_move $opt_move2 $opt_norm $opt_nobkp ${${${#files}:#1}:+--nobkp}
	ret_val+=$?
	}
	# Second, try to find the archive via `file' tool
	if (( !${#files} )) {
	local -aU output infiles stage2_processed archives
	infiles=( **/*~(._zinit*|._backup|.git)(|/*)~*/*/*(-.DN) )
	output=( ${(@f)"$(command file -- $infiles 2>&1)"} )
	archives=( ${(M)output[@]:#(#i)(* |(#s))(zip|rar|xz|7-zip|gzip|bzip2|tar|exe|PE32) *} )
	for file ( $archives ) {
	local fname=${(M)file#(${(~j:|:)infiles}): } desc=${file#(${(~j:|:)infiles}): } type
	fname=${fname%%??}
	[[ -z $fname || -n ${stage2_processed[(r)$fname]} ]] && continue
	type=${(L)desc/(#b)(#i)(* |(#s))(zip|rar|xz|7-zip|gzip|bzip2|tar|exe|PE32) */$match[2]}
	if [[ $type = (zip|rar|xz|7-zip|gzip|bzip2|tar|exe|pe32) ]] {
	(( !OPTS[opt_-q,--quiet] )) && \
	+zinit-message "{pre}ziextract:{info2} Note:{rst}" \
	"detected a {meta}$type{rst} archive in the file" \
	"{file}$fname{rst}."
	ziextract "$fname" "$type" $opt_move $opt_move2 $opt_norm --norm ${${${#archives}:#1}:+--nobkp}
	integer iret_val=$?
	ret_val+=iret_val
	(( iret_val )) && continue
	# Support nested tar.(bz2|gz|…) archives
	local infname=$fname
	[[ -f $fname.out ]] && fname=$fname.out
	files=( *.tar(ND) )
	if [[ -f $fname || -f ${fname:r} ]] {
	local -aU output2 archives2
	output2=( ${(@f)"$(command file -- "$fname"(N) "${fname:r}"(N) $files[1](N) 2>&1)"} )
	archives2=( ${(M)output2[@]:#(#i)(* |(#s))(zip|rar|xz|7-zip|gzip|bzip2|tar|exe|PE32) *} )
	local file2
	for file2 ( $archives2 ) {
	fname=${file2%:*} desc=${file2##*:}
	local type2=${(L)desc/(#b)(#i)(* |(#s))(zip|rar|xz|7-zip|gzip|bzip2|tar|exe|PE32) */$match[2]}
	if [[ $type != $type2 && \
	$type2 = (zip|rar|xz|7-zip|gzip|bzip2|tar)
	]] {
	# TODO: if multiple archives are really in the archive,
	# this might delete too soon… However, it's unusual case.
	[[ $fname != $infname && $norm -eq 0 ]] && command rm -f "$infname"
	(( !OPTS[opt_-q,--quiet] )) && \
	+zinit-message "{pre}ziextract:{info2} Note:{rst}" \
	"detected a {obj}${type2}{rst} archive in the" \
	" file {file}${fname}{rst}."
	ziextract "$fname" "$type2" $opt_move $opt_move2 $opt_norm ${${${#archives}:#1}:+--nobkp}
	ret_val+=$?
	stage2_processed+=( $fname )
	if [[ $fname == *.out ]] {
	[[ -f $fname ]] && command mv -f "$fname" "${fname%.out}"
	stage2_processed+=( ${fname%.out} )
	}
	}
	}
	}
	}
	}
	}
	return $ret_val
	}
	if [[ -z $file ]] {
	+zinit-message "{error}ziextract:{msg2} ERROR:{msg} argument" \
	"needed (the file to extract) or the {meta}--auto{msg} option."
	return 1
	}
	if [[ ! -e $file ]] {
	+zinit-message "{error}ziextract:{msg2} ERROR:{msg}" \
	"the file \`{meta}${file}{msg}' doesn't exist.{rst}"
	return 1
	}
	if (( !nobkp )) {
	command mkdir -p ._backup
	command rm -rf ._backup/*(DN)
	command mv -f *~(._zinit*|._backup|.git|.svn|.hg|$file)(DN) ._backup 2>/dev/null
	}
	.zinit-extract-wrapper() {
	local file="$1" fun="$2" retval
	(( !OPTS[opt_-q,--quiet] )) && \
	+zinit-message "{pre}ziextract:{msg} Unpacking the files from: \`{obj}$file{msg}'{…}{rst}"
	$fun; retval=$?
	if (( retval == 0 )) {
	local -a files
	files=( *~(._zinit*|._backup|.git|.svn|.hg|$file)(DN) )
	(( ${#files} && !norm )) && command rm -f "$file"
	}
	return $retval
	}
	→zinit-check() { (( ${+commands[$1]} )) || \
	+zinit-message "{error}ziextract:{msg2} Error:{msg} No command {data}$1{msg}," \
	"it is required to unpack {file}$2{rst}."
	}
	case "${${ext:+.$ext}:-$file}" in
	((#i)*.zip)
	→zinit-extract() { →zinit-check unzip "$file" || return 1; command unzip -o "$file"; }
	;;
	((#i)*.rar)
	→zinit-extract() { →zinit-check unrar "$file" || return 1; command unrar x "$file"; }
	;;
	((#i)*.tar.bz2|(#i)*.tbz2)
	→zinit-extract() { →zinit-check bzip2 "$file" || return 1; command bzip2 -dc "$file" | command tar -xf -; }
	;;
	((#i)*.tar.gz|(#i)*.tgz)
	→zinit-extract() { →zinit-check gzip "$file" || return 1; command gzip -dc "$file" | command tar -xf -; }
	;;
	((#i)*.tar.xz|(#i)*.txz)
	→zinit-extract() { →zinit-check xz "$file" || return 1; command xz -dc "$file" | command tar -xf -; }
	;;
	((#i)*.tar.7z|(#i)*.t7z)
	→zinit-extract() { →zinit-check 7z "$file" || return 1; command 7z x -so "$file" | command tar -xf -; }
	;;
	((#i)*.tar)
	→zinit-extract() { →zinit-check tar "$file" || return 1; command tar -xf "$file"; }
	;;
	((#i)*.gz|(#i)*.gzip)
	if [[ $file != (#i)*.gz ]] {
	command mv $file $file.gz
	file=$file.gz
	integer zi_was_renamed=1
	}
	→zinit-extract() {
	→zinit-check gunzip "$file" || return 1
	.zinit-get-mtime-into "$file" 'ZINIT[tmp]'
	command gunzip "$file" |& command egrep -v '.out$'
	integer ret=$pipestatus[1]
	command touch -t "$(strftime %Y%m%d%H%M.%S $ZINIT[tmp])" "$file"
	return ret
	}
	;;
	((#i)*.bz2|(#i)*.bzip2)
	→zinit-extract() { →zinit-check bunzip2 "$file" || return 1
	.zinit-get-mtime-into "$file" 'ZINIT[tmp]'
	command bunzip2 "$file" |& command egrep -v '.out$'
	integer ret=$pipestatus[1]
	command touch -t "$(strftime %Y%m%d%H%M.%S $ZINIT[tmp])" "$file"
	return ret
	}
	;;
	((#i)*.xz)
	if [[ $file != (#i)*.xz ]] {
	command mv $file $file.xz
	file=$file.xz
	}
	→zinit-extract() { →zinit-check xz "$file" || return 1
	.zinit-get-mtime-into "$file" 'ZINIT[tmp]'
	command xz -d "$file"
	integer ret=$?
	command touch -t "$(strftime %Y%m%d%H%M.%S $ZINIT[tmp])" "$file"
	return ret
	}
	;;
	((#i)*.7z|(#i)*.7-zip)
	→zinit-extract() { →zinit-check 7z "$file" || return 1; command 7z x "$file" >/dev/null; }
	;;
	((#i)*.dmg)
	→zinit-extract() {
	local prog
	for prog ( hdiutil cp ) { →zinit-check $prog "$file" || return 1; }
	integer retval
	local attached_vol="$( command hdiutil attach "$file" | \
	command tail -n1 | command cut -f 3 )"
	command cp -Rf ${attached_vol:-${TMPDIR:-/tmp}/acb321GEF}/*(D) .
	retval=$?
	command hdiutil detach $attached_vol
	if (( retval )) {
	+zinit-message "{error}ziextract:{msg2} WARNING:{msg}" \
	"problem occurred when attempted to copy the files" \
	"from the mounted image: \`{obj}${file}{msg}'.{rst}"
	}
	return $retval
	}
	;;
	((#i)*.deb)
	→zinit-extract() { →zinit-check dpkg-deb "$file" || return 1; command dpkg-deb -R "$file" .; }
	;;
	((#i)*.rpm)
	→zinit-extract() { →zinit-check cpio "$file" || return 1; $ZINIT[BIN_DIR]/share/rpm2cpio.zsh "$file" | command cpio -imd --no-absolute-filenames; }
	;;
	((#i)*.exe|(#i)*.pe32)
	→zinit-extract() {
	command chmod a+x -- ./$file
	./$file /S /D="`cygpath -w $PWD`"
	}
	;;
	esac
	if [[ $(typeset -f + →zinit-extract) == "→zinit-extract" ]] {
	.zinit-extract-wrapper "$file" →zinit-extract || {
	+zinit-message -n "{error}ziextract:{msg2} WARNING:{msg}" \
	"extraction of the archive \`{file}${file}{msg}' had problems"
	local -a bfiles
	bfiles=( ._backup/*(DN) )
	if (( ${#bfiles} && !nobkp )) {
	+zinit-message -n ", restoring the previous version of the plugin/snippet"
	command mv ._backup/*(DN) . 2>/dev/null
	}
	+zinit-message ".{rst}"
	unfunction -- →zinit-extract →zinit-check 2>/dev/null
	return 1
	}
	unfunction -- →zinit-extract →zinit-check
	} else {
	integer warning=1
	}
	unfunction -- .zinit-extract-wrapper
	local -a execs
	execs=( **/*~(._zinit(|/*)|.git(|/*)|.svn(|/*)|.hg(|/*)|._backup(|/*))(DN-.) )
	if [[ ${#execs} -gt 0 && -n $execs ]] {
	execs=( ${(@f)"$( file ${execs[@]} )"} )
	execs=( "${(M)execs[@]:#[^:]##:*executable*}" )
	execs=( "${execs[@]/(#b)([^:]##):*/${match[1]}}" )
	}
	builtin print -rl -- ${execs[@]} >! ${TMPDIR:-/tmp}/zinit-execs.$$.lst
	if [[ ${#execs} -gt 0 ]] {
	command chmod a+x "${execs[@]}"
	if (( !OPTS[opt_-q,--quiet] )) {
	if (( ${#execs} == 1 )); then
	+zinit-message "{pre}ziextract:{rst}" \
	"Successfully extracted and assigned +x chmod to the file:" \
	"\`{obj}${execs[1]}{rst}'."
	else
	local sep="$ZINIT[col-rst],$ZINIT[col-obj] "
	if (( ${#execs} > 7 )) {
	+zinit-message "{pre}ziextract:{rst} Successfully" \
	"extracted and marked executable the appropriate files" \
	"({obj}${(pj:$sep:)${(@)execs[1,5]:t}},…{rst}) contained" \
	"in \`{file}$file{rst}'. All the extracted" \
	"{obj}${#execs}{rst} executables are" \
	"available in the {msg2}INSTALLED_EXECS{rst}" \
	"array."
	} else {
	+zinit-message "{pre}ziextract:{rst} Successfully" \
	"extracted and marked executable the appropriate files" \
	"({obj}${(pj:$sep:)${execs[@]:t}}{rst}) contained" \
	"in \`{file}$file{rst}'."
	}
	fi
	}
	} elif (( warning )) {
	+zinit-message "{pre}ziextract:" \
	"{error}WARNING: {msg}didn't recognize the archive" \
	"type of \`{obj}${file}{msg}'" \
	"${ext:+/ {obj2}${ext}{msg} }"\
	"(no extraction has been done).%f%b"
	}
	if (( move | move2 )) {
	local -a files
	files=( *~(._zinit|.git|._backup|.tmp231ABC)(DN/) )
	if (( ${#files} )) {
	command mkdir -p .tmp231ABC
	command mv -f *~(._zinit|.git|._backup|.tmp231ABC)(D) .tmp231ABC
	if (( !move2 )) {
	command mv -f **/*~(*/*~*/*/*|*/*/*/*|^*/*|._zinit(|/*)|.git(|/*)|._backup(|/*))(DN) .
	} else {
	command mv -f **/*~(*/*~*/*/*/*|*/*/*/*/*|^*/*|._zinit(|/*)|.git(|/*)|._backup(|/*))(DN) .
	}
	command mv .tmp231ABC/$file . &>/dev/null
	command rm -rf .tmp231ABC
	}
	REPLY="${${execs[1]:h}:h}/${execs[1]:t}"
	} else {
	REPLY="${execs[1]}"
	}
	return 0
	}

What I don't like about that idea though is that while you mention unar, there's a ton of alternatives.
What $fancy_archive_extractor should we support?

borking the install and requiring a zi delete.

What exactly do you mean by that?

[alichtman]

If we support installing packages from GH Releases, we must support decompressing them. I think it's reasonable to add support like this.

This is essentially what unar does, to the best of my knowledge. We can reimplement it, or just declare unar as a dependency.

[pschmitt]

If we support installing packages from GH Releases, we must support decompressing them. I think it's reasonable to add support like this.

That is supported already. I've literally linked to the implementation above :)
zinit can already extract archives.
This issue is about whether ziextract should try unpacking with unar before falling back to the individual tar, unzip, unrar etc commands.

[alichtman]

Hah, right. I see that now. Ignore my comment above. I was thinking we only supported shell scripts and executables.

I don't use this feature, so I'm not super knowledgeable about it.

[vladdoster]

@docwhat,

Can this be closed?

[docwhat]

This issue is about whether ziextract should try unpacking with unar before falling back to the individual tar, unzip, unrar etc commands.

At least, unar should be tried. I had unar available but didn't have xz.

[docwhat]

Can this be closed?

I'd prefer if unar was added to ziextract.

[vladdoster]

@docwhat,

I, too, have been on systems or in containers that do not have xz (xz-utils on Ubuntu), file, or unzip.

The most portable solution is to try using python3 modules (e.g., lzma, zip, etc.) and fallback to install programs. I think Python3 will solve most issues of missing archive tools.

Thoughts?