Rules of picking package by provide in Fedora about libsolv

[mlschroe]

What is the smallest set of packages? Why would it be desirable? Why is one big package better than two little packages? More packages may mean more plugins and thus more functionality, isn't that better?

[jsilhan]

Users who compare yum and dnf see that dnf after resolution sometimes selects more packages, so it's automatically worse for them - no racional reason. Can you at least share the preference how are the packages chosen? (to be the result of the smallest size / picked in alphabetical order) ? Is the the package selection of provides always the same when we load repos in different order, etc.? We need to tell them that we know how it's working and argument why is this better.

[mlschroe]

Actually it's currently simply alpha order of the package names, so the repo order does not matter (except for repo priorities, of course). I plan to add a check for a common versioned provides in the near future.

As yum uses a different algorithm, in some cases yum will be better and in other cases dnf. We can also make the algorithm more similar than yum (but the code in yum is really quite complex.)

[jsilhan]

We don't want to make the DNF rules of picking package same as it is in yum. These rules are insane [1]. We found three ways to solve selecting provides issue in Fedora.

• using weak deps - already working
• Pick higher version (if exist) of provide first. e.g this should install bar2:

repo system 0 testtags <inline>
repo available 0 testtags <inline>
#>=Pkg: foo 1 1 x86_64
#>=Req: P
#>=Pkg: bar 1 1 x86_64
#>=Prv: P = 1
#>=Pkg: bar2 1 1 x86_64
#>=Prv: P = 2
system x86_64 rpm system
poolflags implicitobsoleteusescolors
solverflags allowvendorchange keepexplicitobsoletes bestobeypolicy keeporphans yumobsoletes
job install name foo
result transaction,problems <inline>

• load external file into libsolv describing provides preference. We want this as a global solution for Fedora releases not just on packaging level - without changing package name or other hacks to give the package higher chance of choosing. E.g.:
  <provide_name>: <first_choice_pkg_name> > <second_choice_pkg_name> ... in real example prefer mysql-community over mariadb when searching by mysql provides: mysql: mysql-community > mariadb

Does it makes sense to you, Michael? Will it be possible to load somewhat like that?
CC: @rholy

[1] http://yum.baseurl.org/wiki/CompareProviders

Hello, I was also thinking about:

• a package (e.g. fedora-prefered-packages) that will be installed by default and that e.g. Recommends: mariadb, but that would mean that weak deps of already installed packages should be taken into account during future depsolvings. (That could, of course, be "neutralized" by another package that Recommends: community-mysql, but for the first version, that could be enough.)

[mlschroe]

Oh, they are takein into account for finding prefered packages. I would use suggests instead of recommends, though, as you don't want to automatically draw in those packages.

Anyway, I'll also change libsolv's algorithm to be a bit more clever regarding prefering packages if there is no hint available. It'll check for common versioned provides, compare the source rpms, de-prefer packages which are not the highest version. This should help you for the most of the cases.

[jsilhan]

Sounds great, thanks.

[jsilhan]

@rholy AFAIK weak deps are not considered for packages already on the system and not in current transaction.

Oh, they are takein into account for finding prefered packages.

Michael, Jan's comment made me doubt. Do you mean they are taken into account also after they are installed or are they relevant only during the installation of the package that contains the weak dependencies and then they have no effect.

Also, we would really appreciate if we know the rules how packages are selected. On the other hand, I don't think we want to publish any rule because there are already people waiting for such rules ready to abuse them. E.g. they would like to rename their package to get it preferred. Or the would change the versioning scheme of their virtual provides (or even packages) to achieve the preference. So I think the best we can do is just to publish the properties of every solution that libsolv gives us. I mean something like "the solution is always the one with least number packages in the transaction" or "the solution is always the one that consumes the least disk space" or the the best compromise of these or something even more vague. So, I think the best is to publish just the "fitness function" that is used to evaluate solutions and choose the best one. But I'm not sure whether libsolv does something like that at all. Would it be possible to pass such a function to libsolv?

And, if the solution with fedora-prefered-packages wouldn't be enough, would it be possible to implement the third solution that Jan mentioned above? Does OpenSUSE use some means to express the distribution-wise preference?

[AdamWill]

I think this:

"Users who compare yum and dnf see that dnf after resolution sometimes selects more packages, so it's automatically worse for them - no racional reason."

is over-simplifying. There is usually a perfectly rational reason. yum's rules might be 'insane', but in practice, they tend to pick what is clearly, to a rational observer, the right answer.

I have set up a Fedora tracker bug for cases where DNF's resolution of an ambiguous dep is unarguably worse than yum's, for the user:

https://bugzilla.redhat.com/show_bug.cgi?id=1192182

I believe I've come across more than two cases in the past, but so far I've just got references for two cases, but they're fairly significant. The first:

https://bugzilla.redhat.com/show_bug.cgi?id=1183835

The impact of that one is that, from a user perspective, is that if you do dnf install icedtea-web on a clean F21 system, you get a bunch of development packages installed. If you do yum install icedtea-web, you do not. What the user wants, clearly, is a web browser Java plugin. They did not want autotools. yum is giving them what they want much more efficiently than dnf is. Both are, in Perfect Spherical Package Land, "correct" solutions to the requirement, but one is clearly better down in boring old reality than the other, and it's the one yum picks. (That case would be resolved by the check proposed in #66 (comment) , though, I think).

The second:

https://bugzilla.redhat.com/show_bug.cgi?id=1192189

The initial user impact of that was 'if I do a KDE install of F22, I get two kernels'. There's a lot more detail to it, but ultimately it boils down to: if a package needs a kernel module in the modules-extra subpackage, yum knows to install the modules-extra subpackage for an already-installed (or also-in-the-same-transaction) kernel if possible, while dnf doesn't.

[mlschroe]

I improved the choosing algorithm quite a bit. Now if those bugs only included solver testcases we could check if the changes are good enough for you...

[AdamWill]

What constitutes a 'solver testcase', exactly?

BTW, another case we ran into is with the virtual provide desktop-notification-daemon, which on Fedora is provided by:

notification-daemon-0:0.7.6-5.fc22.x86_64
cinnamon-0:2.4.6-3.fc22.x86_64
dunst-0:1.1.0-2.fc22.x86_64
gnome-shell-0:3.15.90-1.fc22.x86_64
mate-notification-daemon-0:1.9.1-2.fc22.x86_64
xfce4-notifyd-0:0.2.4-4.fc22.x86_64
kde-runtime-0:14.12.1-3.fc22.x86_64

There's an environment group called 'basic desktop' which is supposed to be a very basic X install with, like, icewm or something. When installed with yum it pulls in 'notification-daemon' to satisfy that dependency, which is a sensible (smallest dep load) choice. dnf pulls in 'cinnamon' to satisfy it, resulting in an install with ~100 packages more than the yum path, and a broken cinnamon as the default desktop.

[jsilhan]

@AdamWill : using --depsolver switch [1], if you attach it to all bug reports I will test it. Or even better I will do a new libsolv build in rawhide and we can test it there.
@mlschroe , thanks

[1] https://github.com/rpm-software-management/dnf/wiki/Bug-Reporting#depsolving-problem

[AdamWill]

@jsilhan yes please, if you could build it for Rawhide and also F22 that would be a big help (I think we may want to get the change in for F22 Alpha if it tests out well).