Introduction

If you used Void Linux for some amount of time you probably ran into these types of errors when updating packages:

foo-1.0.0_1: broken, unresolvable shlib 'libbar.so.2'

Annoying as it is, it is part of a very important part of Void Linux and its QA, today we will look at how xbps-src cooperates with xbps. to make shlibs work.

Shared Libraries

If you're already familiar with shared libraries you can skip to the next section

Shared libraries are a collection of code that is loaded by programs when they start, after a program starts all other programs that also need that library will also use it automatically without needing to load it.

Knowing what libraries a programs needs is simple, ldd from glibc can be used. In this case we will use lddtree from pax-utils since it provides us with a nice tree output:

$ lddtree /usr/bin/cat
cat => /usr/bin/cat (interpreter => /lib/ld-linux-x86-64.so.2)
    libc.so.6 => /usr/lib/libc.so.6
        ld-linux-x86-64.so.2 => /lib/ld-linux-x86-64.so.2

The files after the arrow (=>) are the libraries to be loaded when the program starts.

Let's take a look at python3:

$ lddtree /usr/bin/python3
python3 => /usr/bin/python3 (interpreter => /lib/ld-linux-x86-64.so.2)
    libpython3.6m.so.1.0 => /usr/lib/libpython3.6m.so.1.0
        libdl.so.2 => /usr/lib/libdl.so.2
            ld-linux-x86-64.so.2 => /lib/ld-linux-x86-64.so.2
        libutil.so.1 => /usr/lib/libutil.so.1
        libm.so.6 => /usr/lib/libm.so.6
    libpthread.so.0 => /usr/lib/libpthread.so.0
    libc.so.6 => /usr/lib/libc.so.6

Interesting, it loads /usr/lib/libpython3.6m.so.1.0, Let's remove it to see how it breaks!

\# rm /usr/lib/libpython3.6m.so.1.0
$ python3
/usr/bin/python3: error while loading shared libraries: libpython3.6m.so.1.0: cannot open shared object file: No such file or directory

Oh, it can't load the library and thus it can't start. Well now we know the importance of shared libraries and that you shouldn't go around deleting them.

xbps-src shlibs

Shared libraries are of great interest to Void Linux and the grand majority of the software shipped makes use of those. So a great deal of time and manpower has been invested into xbps-src and the surrounding infrastructure to make sure that it works.

xbps-src has the shlibs mechanism, it tries to detect all libraries that are used by every executable on the package and will add it to the dependencies of the package. To know what library belongs to what package it makes use of the ever-expanding, and rarely retracting common/shlibs file.

Since the file is too big to throw down here lets grab only a few entries:

libreadline.so.8 libreadline8-8.0_1
libudev.so.1 eudev-libudev-1.6_1
libLLVM-7.so libllvm7-7.0.0_1

The first field tells the name of the library file and the second the package which provides it, this information is used to generate runtime dependencies of packages automatically.

Example: a packages that has an executable that depends on libudev.so.1 will have a dependency on eudev-libudev-1.6_1 or greater.

Let's see it in action, the 04-generate-runtime-deps.sh hook that is on common/hooks/pre-pkg is the one that runs all the logic for producing it, let's build a package that has library dependencies and see it work!

$ xbps-src pkg rlwrap
=> rlwrap-0.43_2: running pre-pkg hook: 04-generate-runtime-deps ...
   SONAME: libutil.so.1 <-> glibc>=2.28_1
   SONAME: libreadline.so.8 <-> libreadline8>=8.0_1
   SONAME: libncursesw.so.6 <-> ncurses-libs>=5.8_1
   SONAME: libc.so.6 <-> glibc>=2.28_1

Let's take a look at the dependencies!

$ xbps-query --repository=hostdir/binpkgs -x rlwrap
perl>=0
glibc>=2.28_1
libreadline8>=8.0_1
ncurses-libs>=5.8_1

Neat, we guarantee that all dependencies of a package that are libraries are fulfilled.

shlib bumps!

But the world of FOSS libraries is not static, on the contrary it moves quite a lot when compared to proprietary vendors.

So when a library writer makes an incompatible change, what happens ? The soname is bumped!

As an example, recently readline-8.0 was released and changed its soname from libreadline.so.7 to libreadline.so.8.

What happens when that file change the name ? Same result as if we remove the library the executables won't be able to start anymore since they can't load the required shared library.

When that happens it is up for distributions like Void Linux to rebuild all packages that use that library to use the new one.

Thankfully xbps-src is always ever helpful with that, let's take a look at what it print outs when it detects a soname bump.

$ xbps-src pkg libtirpc
=> ERROR: libtirpc-1.1.4_1: SONAME bump detected: libtirpc.so.2 -> libtirpc.so.3
=> ERROR: libtirpc-1.1.4_1: please update common/shlibs with this line: "libtirpc.so.3 libtirpc-1.1.4_1"
=> ERROR: libtirpc-1.1.4_1: all reverse dependencies should also be revbumped to be rebuilt against libtirpc.so.3:
=> ERROR:    autofs-5.1.5_2
=> ERROR:    nfs-utils-1.3.4_6
=> ERROR:    quota-4.04_3
=> ERROR:    rpcbind-1.2.5_1
=> ERROR:    zfs-0.7.12_1
=> ERROR: libtirpc-1.1.4_1: cannot continue with installation!
=> ERROR: libtirpc-1.1.4_1: pre-pkg_99-pkglint: 'grep -E "${_pattern}" $mapshlibs' exited with 1
=> ERROR:   in hook() at common/hooks/pre-pkg/99-pkglint.sh:108
=> ERROR:   in run_func() at common/xbps-src/shutils/common.sh:21
=> ERROR:   in run_pkg_hooks() at common/xbps-src/shutils/common.sh:251
=> ERROR:   in main() at common/xbps-src/libexec/xbps-src-prepkg.sh:47

Neatly helpful, it tells us to update the common/shlibs entry, and even tells us the packages we need to revbump.

revbump means bumping the revision= field by 1 in the template, so the package is rebuilt and is considered an update to users.

For doing revbumps we can make use of xtools, a package of shell script utilities for dealing with xbps-src and Void Linux systems in general, today we are going to use xrevbump.

$ xrevbump 'rebuild against libtirpc.so.3' autofs fs-utils quota rpcbind zfs
srcpkgs/autofs/template: bump to revision 3
[revbump-libtirpc 866c9c8c80] autofs: rebuild against libtirpc.so.3
 1 file changed, 1 insertion(+), 1 deletion(-)
srcpkgs/fs-utils/template: bump to revision 2
[revbump-libtirpc 4fd6e3e9be] fs-utils: rebuild against libtirpc.so.3
 1 file changed, 1 insertion(+), 1 deletion(-)
srcpkgs/quota/template: bump to revision 4
[revbump-libtirpc b6ce2d7c02] quota: rebuild against libtirpc.so.3
 1 file changed, 1 insertion(+), 1 deletion(-)
srcpkgs/rpcbind/template: bump to revision 2
[revbump-libtirpc 9a8fe973cb] rpcbind: rebuild against libtirpc.so.3
 1 file changed, 1 insertion(+), 1 deletion(-)
srcpkgs/zfs/template: bump to revision 2
[revbump-libtirpc 52104e587f] zfs: rebuild against libtirpc.so.3
 1 file changed, 1 insertion(+), 1 deletion(-)

Done!, we just need to build the packages and test them, and after that the commits can be pushed to the master branch, after which the Void Linux builders will pick them up and build them and make them available for the end users.