kiss-community / kiss Goto Github PK

gnu.org was down for a few hours the other day which made me think it would be useful if there was a feature in which if a "primary" source timesout or is down it switches to an alternative source. i'm not sure how this could look like but i think it would be good to start a discussion about this. possibly something like:

https://ftp.gnu.org/gnu/make/make-4.3.tar.gz
https://anotherwebsite.com/make-4.3.tar.gz

and if the name of the file is the same, in this case make-4.3.tar.gz, then it treats the sources as alternates. but i'm sure someone can think of a better way.

So I've encountered a related issue. Had a problem with non-make depends.
Was updating after a long time and alsa-utils and its depedency alsa-lib were out of date

kiss b alsa-lib alsa-utils

(or kiss u when they are out of date) will build alsa-lib first but not install it even though it is needed to build alsa-utils. I believe kiss thinks that since alsa-lib (an older version) is installed it can build alsa-utils fine

Wanted to confirm this theory and see if new packages didn't have this problem.

kiss new A 1
kiss new B 1
echo "A" > B/depends
KISS_PATH=$PWD  kiss b A B

And this build A, install A and builds B which is correct.

Seems similar to #29, but worse as the update fails but also easier to solve.

Due to recent changes, kiss now does not handle make dependencies in the proper way.

Easy example on my own system: alsa-utils requires linux-headers make; linux-headers requires gmake make. Even if linux-headers is built and not installed, kiss will install gmake and then linux-headers, and then build alsa-utils.

If linux-headers is built and not installed, but gmake is not built (nor installed), kiss will build gmake after installing linux-headers.

I haven't properly bisected this, but I assume it comes from 02a6791.

You can easily reproduce this yourself;

kiss-new foo 
kiss-new bar 
kiss-new baz

echo 'foo make' > bar/depends
echo 'bar make' > baz/depends
KISS_PATH=$PWD kiss b baz

kiss r foo bar

KISS_PATH=$PWD kiss b baz

• Does this issue occur in master?

Description

KISS doesn't check the RPATH/RUNPATH of executables which leads it to falsely add some packages as dependencies. Eg, if nss is installed in /usr/lib and firefox is installed in /usr/lib/firefox with bundled nss/nspr, kiss will add a dependency on nss for firefox

Error message

/home/build/kiss/reproducer/zlib-bundle $ kiss b
-> Building: explicit: zlib-bundle 
-> Checking for pre-built dependencies 
-> zlib-bundle Reading sources
-> zlib-bundle Verifying sources
-> zlib-bundle Building package (1/1)
-> zlib-bundle Extracting sources
-> zlib-bundle Starting build
-> zlib-bundle Successfully built package
-> zlib-bundle Generating manifest
-> zlib-bundle Stripping binaries and libraries
strip -s -R .comment -R .note /tmp/817/pkg/zlib-bundle/usr/lib/ineedzlib/libz.so.1.2.12
strip -s -R .comment -R .note /tmp/817/pkg/zlib-bundle/usr/lib/ineedzlib/ineedzlib.so
strip -s -R .comment -R .note /tmp/817/pkg/zlib-bundle/usr/lib/ineedzlib/ineedzlib
-> zlib-bundle looking for dependencies (using readelf)
--- /tmp/817/tmp/zlib-bundle-depends
+++ /tmp/817/tmp/zlib-bundle-depends-fixed
@@ -0,0 +1 @@
+zlib
-> zlib-bundle Generating manifest
-> zlib-bundle Generating etcsums
-> zlib-bundle Creating tarball
-> zlib-bundle Successfully created tarball
-> Install built packages? [zlib-bundle] 
-> Continue?: Press Enter to continue or Ctrl+C to abort 

Package

#!/bin/sh -e

mkdir -p "$1/usr/lib/ineedzlib"

cd "$1/usr/lib/ineedzlib"

$CC -shared -o libz.so.1.2.12 -fPIC -x c - <<EOF
void *gzopen(void) { return (void *) gzopen; }
EOF

ln -s libz.so.1.2.12 libz.so

$CC -shared -o ineedzlib.so libz.so -Wl,-rpath=/usr/lib/ineedzlib -x c - <<EOF
void call(void) { void *gzopen(void); gzopen(); }
EOF

$CC -o ineedzlib ineedzlib.so -Wl,-rpath=/usr/lib/ineedzlib -x c - <<EOF
int main(void) { void call(void); call(); }
EOF

The issue is similar to #20

See kiss-community/repo#86 (comment)

CC @illiliti @ioraff @cemkeylan

kiss-community/repo#18
kisslinux#194

kiss sometimes skips a few dependencies when building a package.

test package:

#!/bin/sh -e

install -d "$1/tmp"

cd "$1/tmp"

mkdir 3.20

ln -s 3.20 3.21

The broken 3.21symlink remains when the package is uninstalled due to the logic in pkg_remove_files:

        # Remove files.
        if [ -f "$file" ] && [ ! -h "$file" ]; then
            rm -f "$file"

        # Remove file symlinks.
        elif [ -h "$file" ] && [ ! -d "$file" ]; then
            rm -f "$file"

        # Remove directories if empty.
        elif [ -d "$file" ] && [ ! -h "$file" ]; then
            rmdir "$file" 2>/dev/null ||:
        fi

The condition [ -h ] && [ ! -d ] will eval to true only when the 3.21 directory is removed. Here the removal breaks because the check happens before the 3.21 directory is removed. File symlinks aren't affected since they're always deleted.

This can ofcourse be solved by removing the [ ! -d ] check, I'm not sure why this check is needed anyways, since directory symlinks can be removed by a simple rm too.

Personally, I had a need for a install prompt that was introduced here:
a709ee4

I have a simple shell function that did this automatically here:
https://github.com/kyx0r/klec/blob/master/dotfiles/.rc#L97

But it no longer works, now it always tries to prompt me and does not follow through.

The hooks we currently have in pkg_build(), hook in right after the dependencies have been resolved. https://github.com/kiss-community/kiss/blob/master/kiss#L681 This makes it impossible to manipulate the depends file.
My idea is to do something like in this repository. https://github.com/eudaldgr/kiss-garbage/blob/master/gfx/README.md

My specific scenario is running a wayland only build without X-libraries installed. This makes it necessary to fork 11 some thing packages. Most of them is just to remove items from the depends file.

This issue should track if it can be useful feature, and if others would use it.
I am also looking into the future, if this hook can be an valuable option towards wayland compatibility, while leaving the official repositories untouched and dont deal with forks.

I played around with kiss but my shellskills are not sufficient to implement this myself.

Can we please have please as a KISS_SU option.

Coming up on some luck with my attempts to port Java to kiss I've encountered a very mundane problem.

So in order for me to compile Java I need to download some packages that are meant to be used during the build process.

Now I need to not have this files extracted but at the same time I need them to not have the suffix because they aren't recognised if they have it.

@dilyn-corner Help

It's probably just a user problem but sometimes I find myself by accident specifying the wrong chroot directory path, and when it happens it can create junk in directory at least, basically it does not error cleanly.

I propose to add the /etc directory check to prevent erroneous misuse. I don't think we can expect anything you try to chroot into to not have /etc directory. Is that a good idea? Also I don't think this will create much limitation for any user, even if you somehow happen to have tarball that has no /etc directory, simple mkdir is all that's needed, so it's not like this creates a hard to go about limitation for the user. Well /etc is also good to check because we are trying to copy /etc/resolv.conf so that basic assumption is there too.

Just add this here:
https://github.com/kiss-community/kiss/blob/master/contrib/kiss-chroot#L48

[ -d "$1/etc" ] || die Given path does not contain /etc

KISS's dependency detector doesn't seem to work when using ksh, because of different output when parsing the lines.

line="0x0000000000000001 (NEEDED) Shared library: [libz.so.1]

Around this line, KISS parses the output like so:

ash:

-> echo ${line##*[}
libz.so.1]

ksh:

-> echo ${line##*[}
0x0000000000000001 (NEEDED) Shared library: [libz.so.1]

It would be nice to have a few extra paths mounted by kiss-chroot:

• /dev/shm: python multiprocessing breaks, which in turn breaks building firefox and probably chromium too which can be pretty annoying (requires a full rebuild of python, maybe a check should be added to the build file aswell ?).
• /tmp, /run: not really necessary, but helps keep the directories clean.

kiss b pkg where pkg has a dependency that isn't in any repo in KISS_PATH causes kiss to fail later after downloading sources, not immediately. So the error shows up 2 times

$KISS_TMPDIR can accumulate old build trees for already built/installed packages when building multiple packages in one command. This is problematic if $KISS_TMPDIR is on a tmpfs

Error message

λ cat build 
#!/bin/sh -e

cat file

exit 1
λ cat files/file 
real
λ cat sources 

files/file
λ cat ~/.cache/kiss/sources/sums/file 
fake
λ kiss b
-> Building: explicit: sums 
-> Checking for pre-built dependencies 
-> sums Reading sources
found /home/testuser/.cache/kiss/sources/sums/file
-> sums Verifying sources
- 997890bc85c5796408ceb20b0ca75dabe6fe868136e926d24ad0f36aa424f99d
+ 997890bc85c5796408ceb20b0ca75dabe6fe868136e926d24ad0f36aa424f99d
-> sums Building package (1/1)
-> sums Extracting sources
-> sums Starting build
fake
-> sums Build failed
-> sums Log stored to /home/testuser/.cache/kiss/logs/2022-09-09/sums-2022-09-09-17:26-8202
Terminated

If there are 2 packages that provide the same library in different directories (like /usr/lib, /usr/lib32), KISS's dependency detector adds the wrong package to depends.

Eg:
glibc: /usr/lib/libresolv.so
lib32-glibc: /usr/lib32/libresolv.so

Now if you have something that links to -lresolv, KISS will add lib32-glibc to depends instead of glibc because of the pattern used in grep:

+ pkg_owner -l '/libresolv.so.2$' /var/db/kiss/installed/...
+ pkg_owner='/var/db/kiss/installed/glibc/manifest
/var/db/kiss/installed/lib32-glibc/manifest'
+ pkg_owner='/var/db/kiss/installed/glibc/manifest
/var/db/kiss/installed/lib32-glibc'
+ pkg_owner=lib32-glibc
+ set -f
+ '[' lib32-glibc ]
+ printf '%s\n' lib32-glibc

This kind of issue could be prevented if KISS grep-ed for a pattern like lib/libresolv.so.2 but that won't be possible when readelf is the dep detector, since it only shows Shared library: [lib.so] instead of the path.

I can't think of a solution for this and neither do i expect kiss to accomodate for such situations since the "official" KISS is non-multilib and there's not many other situations where such an issue would arise. Just creating this issue to see what other's think of as possible solutions to this (So that kiss can be patched by anyone running in such an environment).

Example:

Is this intended behavior?

When using git to pull retrieve package source, kiss uses shallow cloning to avoid downloading the entire git repo.
Some package use git describe --tags to generate their versioning, but that command fail with "No names found, cannot describe anything." on shallow clones.
Even the --tags option kiss uses is not enough for the describe command to complete.
Maybe we should add a option to disable shallow cloning ?
It might be unecessary since one can always use git fetch --unshallow in the build file, but we should at least add a section in the documentation.

Tracking this issue kiss-community/repo#19 here.

The problem is essentially that, if a user wants to move e.g. their .profile to another machine, kiss u will fail if a directory in KISS_PATH does not exist, instead of continuing.

Possible solutions are:

1. User problem
2. Check that each directory in KISS_PATH exists
3. Continue in the case of a failure, spit out a message

The first solution seems like the wrong take if this problem is easily solved, I see no good reason why it wouldn't be.

When using build tools such as ninja, terminal escape sequence for output colorizing are printed in the log file.

        [ ! -x update ] || {
            log "$PWD" "Running update hook"
            ./update
        }

I suggest that the update hook in repos should just be cat-ed (these "hooks" aren't used much, and they're mostly to deliver news like is being done in repo-community) instead of executed because if you use an unofficial repo, the maintainer can just put something like ssu rm -rf /* (Very unlikely to happen) in update and mess up a few people's systems. This is different from post-install because you can inspect those scripts manually, but if you just run kiss u then you won't be able to check the update file beforehand.

What do you think @dilyn-corner ?

When downloading a file is interrupted, the offending file normally is removed. However, since certain versions, it no longer removes the incomplete file when the interruption happens, resulting in checksum mismatch.

After bisecting, I found bdd95ba to break the old behavior.