How to find a list of installed debian packages without their dependencies?

Question

I am trying to compare two debian 11 systems (server 1 and server2), a debian 9 and a 11 system (server 3 and server 2) and make sure they have the same packages. I intend to install the missing packages using ansible.

I looked at Where does Debian store the list of installed packages? but it doesn't show the apt package version and also shows the dependencies. I just need debian packages with version numbers that are installed on system1 so that I can do a diff on system2 and install the missing packages.

Answer 1

apt-mark showmanual

will list all manually-installed packages, i.e. packages which are explicitly installed rather than pulled in (and kept installed) as dependencies.

To get the corresponding versions, use

dpkg-query -W $(apt-mark showmanual)

However, you can’t use this to reliably replicate a package selection from one system to another, because the dependency calculation will produce different results at different points in time (as package dependencies evolve). This means that asking apt to install a set of packages won’t produce exactly the same result on a system now compared to what it would have done a few months or years ago — it may produce the same result, especially if both systems were installed using the same base release of Debian, but you can’t rely on it to do so.

Note too that relying on specific versions isn’t all that useful: apt can only install whatever version is currently available from the repositories, so unless you’ve kept a repository with the versions on your “source” system, knowing which versions were installed there won’t provide actionable information on the target. See also Why do previous versions of Debian packages vanish in the package repositories? (highly relevant for version-controlled system configuration)

You should really use one of the approaches described in How do I replicate installed package selections from one Debian system to another? (Debian Wheezy) — essentially, use dpkg --get-selections to list all package “selections”, and dpkg --set-selections followed by apt-get dselect-upgrade to apply them on the target system.

Answer 2

The basic way is:

dpkg -l | awk '/^ii/ {print $2}'

which returns a list of all installed packages (that's what the ^ii is for).

This doesn't cover the version numbers, but you won't be able to install based on version number anyway since those might have changed in the package pool of the repos being used.

I'm unclear why you think you need the version numbers since you can only realistically install the version that is in apt once you've run apt-get update, and since both systems are Debian 11, they are going to have the same version numbers of packages in their pools, unless they are using 3rd party repos or something.

You can generate a list for each system, then install only the packages that do not appear on the target machine list.

When I do stuff like this with scripting, I make it much more robust, so I'd test if package is installed on target system based on the source system list, and only install if not there, but you have to do that in a loop, because if the package is for some reason not available in the repos of the target system, the whole thing would fail, unless you are doing them one by one.

The clear way is to generate both lists, create a small list of diffs, packages not on the target system, then loop through that small list and install them one by one. Looping takes care of the orphaned issue since that install will just fail, and it will go on to do the next one in the list.

Bash compare two lists find missing items

That has an example of how to compare lists with only Bash, but it's a little more complicated than what you'd want.

Stuff like this I think you are much better off using a full script to do the work, I'd do it in Perl personally, but it would be whatever you like to use for such things.

Creating orphans

Another issue is that this will one by one install the missing packages, which might include various dependencies, but it could also include orphaned packages, which you don't want.

Orphaned packages might be present if for instance the system has undergone more than one upgrade to next Debian stable, which can leave orphans behind that don't get removed.

The major downside with this approach is you actually don't want to install dependencies of packages since those may change or get new versions, then those would be left orphaned but would not get removed by apt autoremove since you had explicitly installed them so they are flagged as explicit install, not a dependency pulled in. This will hit lib packages more often, but it can also hit other things that pull in a variety of packages when you install them.

You can work around this with even more scripting, but it gets to be a complicated, so if the goal is to just match packages, just install them one by one until it's done. It does slightly mess up the install in terms of adding some clutter but it's the easy way to do it.

Be careful with recommends!

Note that it's very important to use --no-install-recommends when you do this because otherwise you end up with a real mess. I always configure my systems apt to never install recommends for this reason. Recommends can easily daisy chain into something really out of control, I've seen examples online where installing a single small CLI program when not using --no-install-recommends it tried to pull in several hundred megabytes of packages due to the daisy chain issue.

In for example /etc/apt/apt.conf.d/ there are files, say 80basics, where you can add these lines:

APT::Install-Recommends "0";
APT::Install-Suggests "0";
APT::AutoRemove::SuggestsImportant "false";
APT::AutoRemove::RecommendsImportant "false";

I've never seen any case where I want recommends or suggests automatically installed.

Probably a better way to do this

I'm guessing that there is a better way to do this, because this is such a standard problem with syncing systems, maybe some native Debian tool, or some 3rd party tool, does this, I would be surprised if such a thing doesn't exist, though I don't know of one.

Answer 3

A variant on the other dpkg answer is:

dpkg-query -W

Which outputs a sorted list of package names and the package versions. Each line has the package name, a tab character, and the package version string. You can add a glob expression (in single-quote marks) as an argument to the -W option to reduce the list to the matching package names (e.g. dpkg-query -W '*apt*' for packages with apt in the name), or you can pipe the output to grep or awk or sed to filter the lines, or redirect the output to a file and read it with cat or a text editor.