Troubleshooting: Rocky Linux 9 on i586

$Id$


System clock not properly set when RTC is set in future

Symptom: Had set the RTC clock in far future, such as 2038/02/01 (Y2038 overflowed date). When rebooted, the system time is 2022-04-07. timedatectl shows that RTC is in correct future date, but system time is not.

Cause: Not kernel, but systemd, resets the clock to "sane" value when it detects the system clock is far from the systemd build date.

Apr 07 00:00:00 systemd[1]: System time is further ahead than 15y after build time, resetting clock to build time.

How far?
#define CLOCK_VALID_RANGE_USEC_MAX 473364000000000, 15 years.
From when (the epoch)?
systemd src/shared/clock-util.c:clock_apply_epoch() does
  1. if /usr/lib/clock-epoch file exists, use the mtime of the file for the epoch. Usually you do not have this file.
  2. TIME_EPOCH, which is a value dynamically generated when systemd was built. meson.build tries
    1. $SOURCE_DATE_EPOCH environment on build. RHEL 9 rpmbuild picks this up from %changelog of RPM *.spec file. For systemd, currently it sets this to 1649289600 (2022-04-07 00:00:00 UTC)
    2. Date of the latest git tag, if the build directory is a git workspace.
    3. Date of NEWS file in the distribution.

Workaround: None.
Placing a /usr/lib/clock-epoch file, and timestamping it near the desired system date, makes the system boot time to that date (doesn't honour RTC), when experimenting with future system time.

Workaround: Place a new systemd unit file /etc/systemd/system/hwclock-hctosys.service as following:

[Unit]
Description Set date from RTC
After=basic.target
Wants=basic.target

[Service]
Type=oneshot
ExecStart=/usr/sbin/hwclock --hctosys
RemainAfterExit=yes

[Install]
WantedBy=basic.target
Then, invoke systemctl enable hwclock-hctosys to enable it.
Caveats: This setup still makes system time jump a couple of times during boot, so timestamps of early system logs are not linear.


dnf/rpm refuses to open rpmdb.sqlite

Synptom: rpm and/or dnf command refuses to open /var/lib/rpm/rpmdb.sqlite file, which is clear in sight, no weird ACLs, no SELinux problems.

[kabe@i686 ix86.repo]$ sudo dnf --installroot=/mnt --disablerepo=\* --enablerepo=ix86\* reinstall kbd
error: Unable to open sqlite database /mnt/var/lib/rpm/rpmdb.sqlite: unable to open database file
error: cannot open Packages index using sqlite - Operation not permitted (1)
error: cannot open Packages database in /mnt/var/lib/rpm
Error: Error: rpmdb open failed
[kabe@i686 ix86.repo]$ ls -l /mnt/var/lib/rpm/
total 16056
-rw-r--r--. 1 root root 16408576 Sep 22 20:45 rpmdb.sqlite
-rw-r--r--. 1 root root    32768 Sep 23 06:08 rpmdb.sqlite-shm
-rw-r--r--. 1 root root        0 Sep 22 20:45 rpmdb.sqlite-wal

Cause: plain stat(2) on the file was failing, due to rpmdb.sqlite file having Y2038-overflow timestamp.

[kabe@i686 ix86.repo]$ stat /mnt/var/lib/rpm/rpmdb.sqlite
  File: /mnt/var/lib/rpm/rpmdb.sqlite
  Size: 16408576        Blocks: 32048      IO Block: 4096   regular file
Device: 812h/2066d      Inode: 12583042    Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2038-02-02 13:28:25.030000000 +0900
Modify: 2022-09-22 20:45:55.593398092 +0900
Change: 2022-09-22 20:45:55.593398092 +0900
 Birth: 2022-09-22 17:00:03.843346104 +0900

Rocky Linux supplied sqlite-libs-3.34.1-5.el9.i686 is not compiled with -D_TIME_BITS=64, so it ceases to work when any of the file it touches has Y2038-overflow time attributes.

Workaround: Use sqlite-libs package supplied by this site.


anaconda early death: No library named udev

Symptom: anaconda installer dies at text mode stage:

  File "/usr/lib/python3.9/site-packages/pyanaconda/core/configuration/storage_constraints.py", line 22, in <module>
    from pyanaconda.core.storage import DEVICE_TYPE_LVM, DEVICE_TYPE_MD, DEVICE_TYPE_PARTITION, \
  File "/usr/lib/python3.9/site-packages/pyanaconda/core/storage.py", line 22, in <module>
    from blivet import udev
  File "/usr/lib/python3.9/site-packages/blivet/udev.py", line 39, in <module>
    global_udev = pyudev.Context()
  File "/usr/lib/python3.9/site-packages/pyudev/core.py", line 61, in __init__
    self._libudev = load_ctypes_library('udev', SIGNATURES, ERROR_CHECKERS)
  File "/usr/lib/python3.9/site-packages/pyudev/_ctypeslib/utils.py", line 55, in load_ctypes_library
    raise ImportError('No library named %s' % name)
ImportError: No library named udev
Exception ignored in: <function Context.__del__ at 0xb4edb418>
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/pyudev/core.py", line 65, in __del__
    self._libudev.udev_unref(self)
AttributeError: 'Context' object has no attribute '_libudev'

Pane is dead (status 1, Sun Oct 16 12:35:14 2022)

Cause: For unknown reason, python3.i586 tries to load /usr/lib/libudev.so, not /usr/lib/libudev.so.1, and fails.

Workaround: Add /usr/lib/libudev.so symlink into the installer medium. Adding lorax --installpkgs=systemd-devel will do it.


DNF error: Error in POSTTRANS scriptlet in rpm package kernel-core

Symptom: When installing the system with time set beyond Y2038, anaconda installer halts during package install:

The following error occured while installing the payload. This is fatal error and installation will be aborted.

DNF error: Error in POSTTRANS scriptlet in rpm package kernel-core


/tmp/packaging.log has log:

20:09:41,788 INF packaging: Configuring (running scriptlet for): kernel-core-5.14.0-70.26.1.el9.v1.i686 1664514571 71c15d5731de2e5c4a824e989abb8981e08e8b8148dc9c08a919218fd7d3a50c
20:12:23,810 INF dnf.rpm: grub2-probe: error: cannot find a GRUB drive for /dev/sda2.  Check your device.map.
sed: couldn't edit /var/tmp/dracut.eQcVl8/initramfs/usr/lib/systemd/system/dbus.socket: not a regular file
sed: couldn't edit /var/tmp/dracut.eQcVl8/initramfs/usr/lib/systemd/system/systemd-tmpfiles-setup.service: not a regular file
sed: couldn't edit /var/tmp/dracut.1VlkEL/initramfs/usr/lib/systemd/system/dbus.socket: not a regular file
sed: couldn't edit /var/tmp/dracut.1VlkEL/initramfs/usr/lib/systemd/system/systemd-tmpfiles-setup.service: not a regular file
sed: couldn't edit /boot/loader/entries/ac6ed28bcbe9456b977b68236ce04be4-0-rescue.conf: not a regular file
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
grep: /boot/grub2/grubenv: No such file or directory
warning: %posttrans(kernel-core-5.14.0-70.26.1.el9.v1.i686) scriptlet failed, exit status 4

20:12:23,864 ERR dnf.rpm: Error in POSTTRANS scriptlet in rpm package kernel-core

Cause 1: grub-probe command is not compiled with -D_TIME_BITS=64. Without it, it fails to probe /dev/sda2 and fails with message grub2-probe: error: cannot find a GRUB drive for /dev/sda2. Check your device.map.

Cause 2: sed is not compiled with -D_TIME_BITS=64. Without it, it fails to sed -i a file and returns error.

exit status 4 is a sum of errors during the posttrans scriptlet. It may vary across installations.

Resolution: Use the grub and sed package provided from this site.


error: Could not open %files file /home/kabe/r9builds/kernel-r9/BUILD/kernel-5.14.0-427.16.1.el9_4/debugfiles.list: No such file or directory

Symptom: Compiling kernel with rpmbuild --with debuginfo (default) . Compile does not finish:

...
/usr/lib/rpm/find-debuginfo.sh: line 329: -q/debuglinks.list: No such file or directory
/usr/lib/rpm/find-debuginfo.sh: line 594: -q/debugfiles.list: No such file or directory
/usr/lib/rpm/find-debuginfo.sh: line 599: -q/debugfiles.list: No such file or directory
...
error: Could not open %files file /home/kabe/r9builds/kernel-r9/BUILD/kernel-5.14.0-427.16.1.el9_4/debugfiles.list: No such file or directory

Resolution: Update find-debuginfo.sh (rpm-build package) to rpm-build-4.16.1.3-29.el9 or newer.


kernel fails to compile: *** Error during writing of the configuration.

Symptom: Kernel fails to compile in early stage:

+ rm kernel-5.14.0-x86_64.config
+ RHJOBS=4
+ PACKAGE_NAME=kernel
+ ./process_configs.sh -w -n -c 5.14.0
Processing /home/kabe/r9builds/kernel-r9/BUILD/kernel-5.14.0-427.16.1.el9_4/linux-5.14.0-427.16.1.el9_4.v1.i586/configs/kernel-5.14.0-i586.config ...

*** Error during writing of the configuration.

make[2]: *** [scripts/kconfig/Makefile:77: olddefconfig] Error 1
make[1]: *** [Makefile:657: olddefconfig] Error 2

Cause: You have non-stat(2)able directory somewhere in *.config path, namely /home/kabe/r9builds/kernel-r9/BUILD/kernel-5.14.0-427.16.1.el9_4/linux-5.14.0-427.16.1.el9_4.v1.i586/configs/kernel-5.14.0-i586.config in the above example.

Non-stat(2)ability means in -U_TIME_BITS context. If you had set clock beyond Y2038, and reverted back, some of the directories is assigned an Y2038 atime, and 32bit stat(2) will fail.

Resolution: pass -D_TIME_BITS=64 to ./process_config.sh in SPECS/kernel.spec, i.e

...
## You need -D_TIME_BITS=64 when parent dirs of new.config has
## Y2038-overflowed atime
RHJOBS=$RPM_BUILD_NCPUS PACKAGE_NAME=kernel HOST_EXTRACFLAGS="-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64 -fcf-protection=none" V=1 VERBOSE=1 ./process_configs.sh $OPTS %{specversion}
...

postgresql compile fails with missing header

Symptom: postgresql.src compile fails with:

+ /usr/bin/make -O -j4 V=1 VERBOSE=1 world
...
In file included from ../../src/include/postgres.h:47,
                 from hashfn.c:24:
../../src/include/utils/elog.h:71:10: fatal error: utils/errcodes.h: No such file or directory
   71 | #include "utils/errcodes.h"
      |          ^~~~~~~~~~~~~~~~~~
compilation terminated.
or:
PATH="/home/kabe/r9builds/postgresql-r9/BUILD/postgresql-13.7/tmp_install/usr/bin:$PATH" LD_LIBRARY_PATH="/home/kabe/r9builds/postgresql-r9/BUILD/postgresql-13.7/tmp_install/usr/lib"  ../../src/test/regress/pg_regress --temp-instance=./tmp_check --inputdir=. --bindir=     --dbname=contrib_regression --load-extension=ltree python3/ltree_plpython
============== creating temporary instance            ==============
============== initializing database system           ==============

pg_regress: initdb failed
Examine /home/kabe/r9builds/postgresql-r9/BUILD/postgresql-13.7/contrib/ltree_plpython/log/initdb.log for the reason.
Command was: "initdb" -D "/home/kabe/r9builds/postgresql-r9/BUILD/postgresql-13.7/contrib/ltree_plpython/./tmp_check/data" --no-clean --no-sync > "/home/kabe/r9builds/postgresql-r9/BUILD/postgresql-13.7/contrib/ltree_plpython/log/initdb.log" 2>&1

initdb.log shows:
sh: line 1: initdb: command not found

Cause: postgresql Makefile is touchy on MAKELEVEL environment variable. If ./rpmbuild is invoked via make, MAKELEVEL is set to MAKELEVEL=1 for make invoked inside the *.spec, and confuses postgresql Makefiles.

Resolution: Set to MAKELEVEL=0 in *.spec, if the variable is set.


python3.9 package fails to build, excess nis.cpython-39-i386-linux-gnu.so

Symptom: When building python3.9 package, rpmbuild fails:

...
2024-06-06 20:43:05 building 'nis' extension
...
2024-06-06 21:04:05 error: Installed (but unpackaged) file(s) found:
2024-06-06 21:04:05    /usr/lib/python3.9/lib-dynload/nis.cpython-39-i386-linux-gnu.so
2024-06-06 21:04:05    /usr/lib/python3.9/lib-dynload/nis.cpython-39d-i386-linux-gnu.so

Cause: setup.py prepares to build NIS subsystem when /usr/include/rpcsvc/yp_prot.h and /usr/include/rpc/rpc.h is found, but by default on %{rhel} >= 9, rpmbuild --without nis is default, and does not pick up the resulting NIS subcomponent.

Workaround: Uninstall libnsl2-devel package, which provides /usr/include/rpcsvc/yp_prot.h , before building python3.9 paclage.


python3.11 package fails to build, test fails, ModuleNotFoundError: No module named 'distutils.tests'

Symptom: Building python3.11 (needed for cargo2rpm (EPEL)) fails in %check:

======================================================================
ERROR: test_advanced_left_recursive (test.test_peg_generator
.test_c_parser.TestCParser.test_advanced_left_recursive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/kabe/r9builds/python3.11-r9/BUILD/Python-3.11.
7/Lib/test/test_peg_generator/test_c_parser.py", line 236, in test_advanced_left
_recursive
    self.run_test(grammar_source, test_source)
  File "/home/kabe/r9builds/python3.11-r9/BUILD/Python-3.11.7/Lib/test/test_peg_generator/test_c_parser.py", line 115, in run_test
    self.build_extension(grammar_source)
  File "/home/kabe/r9builds/python3.11-r9/BUILD/Python-3.11.7/Lib/test/test_peg_generator/test_c_parser.py", line 112, in build_extension
    generate_parser_c_extension(grammar, Path('.'), library_dir=self.library_dir)
  File "/home/kabe/r9builds/python3.11-r9/BUILD/Python-3.11.7/Tools/peg_generator/pegen/testutil.py", line 105, in generate_parser_c_extension
    compile_c_extension(
  File "/home/kabe/r9builds/python3.11-r9/BUILD/Python-3.11.7/Tools/peg_generator/pegen/build.py", line 54, in compile_c_extension
    from distutils.tests.support import fixup_build_ext  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'distutils.tests'

Workaround: Define SETUPTOOLS_USE_DISTUTILS=stdlib environment variable during build.


cmake %check fails, CMP0019-WARN - FAILED: and others

Symptom: When building cmake, %check fails:

333: ESC[31mCMake Error at /home/kabe/r9builds/cmake-r9/BUILD/cmake-3.26.5/Tests/RunCMake/RunCMake.cmake:231 (message):
333:   CMP0019-WARN - FAILED:
333:
333:   stderr does not match that expected.
333:
333:   Command was:
333:
333:    command> "/home/kabe/r9builds/cmake-r9/BUILD/cmake-3.26.5/redhat-linux-build/bin/cmake" "/home/kabe/r9builds/cmake-r9/BUILD/cmake-3. 26.5/Tests/RunCMake/CMP0019" "-G" "Unix Makefiles" "-DRunCMake_TEST=CMP0019-WARN" "--no-warn-unused-cli" "-DCMAKE_MAKE_PROGRAM=/usr/bin/gmake"
...
333:    expect-stderr> ^CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
333:    expect-stderr>   Compatibility with CMake < 2.8.12 will be removed from a future version of
333:    expect-stderr>   CMake.
...
333:   Actual stderr:
333:
333:    actual-stderr> ESC[0mCMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
333:    actual-stderr>   Compatibility with CMake < 2.8.12 will be removed from a future version of
333:    actual-stderr>   CMake.

Cause: cmake color output makes stderr maching of test to fail.

Workaround: Define environment CLICOLOR=0 .

CLICOLOR=0 \
./rpmbuild --target=i586 -v -bb .... SPECS/cmake.spec

libsolv fails to configure, Could NOT find PythonInterp: (found /usr/bin/python3)

Symptom: When compiling libsolv, CMake configure fails:

...
-- Found PythonLibs: /usr/lib/libpython3.11.so (found version "3.11.7")
CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find PythonInterp: (found /usr/bin/python3)
Call Stack (most recent call first):
  /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:598 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake/Modules/FindPythonInterp.cmake:169 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  bindings/python/CMakeLists.txt:14 (FIND_PACKAGE)

Cause: Found /usr/bin/python3 is a system default python3.9, which is not for the detected /usr/lib/libpython3.11.so .

Workaround: Uninstall python3.11-devel package to get rid of /usr/lib/libpython3.11.so .

...
-- Found PythonLibs: /usr/lib/libpython3.9.so (found version "3.9.18")
-- Found PythonInterp: /usr/bin/python3
...

lld %check fails, /usr/bin/python3.11 /usr/bin/lit

Symptom: during lld compile, %check fails:

Executing(%check): /bin/sh -e /var/tmp/rpm-tmp.yMNSv0
+ umask 022
+ cd /home/kabe/r9builds/lld-r9/BUILD
+ cd lld-17.0.6.src
+ /usr/bin/cmake --build redhat-linux-build -j4 --verbose --target check-lld
[0/1] cd /home/kabe/r9builds/lld-r9/BUILD/lld-17.0.6.src/redhat-linux-build/test && /usr/bin/python3.11 /usr/bin/lit -sv    --path /usr/lib/llvm /home/kabe/r9builds/lld-r9/BUILD/lld-17.0.6.src/redhat-linux-build/test
Traceback (most recent call last):
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
    return next(cls.discover(name=name))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
StopIteration

Cause: /usr/bin/lit (python3-lit package) is not ready for Python 3.11.

Workaroud: Temporary uninstall python3.11: rpm -rv python3.11 --nodeps


mingw-gcc compile fails, missing process.h

Symptom: Compile fails during build of mingw-gcc:

...
checking for process.h... yes
...
checking for process.h... no
...
if [ x"-fpic" != x ]; then   gcc -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m32 -march=i586 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -c -DHAVE_CONFIG_H -g -O2      -I. -I../../libiberty/../include-W -Wall -Wwrite-strings -Wc++-compat -Wstrict-prototypes -Wshadow=local -pedantic  -D_GNU_SOURCE  -fpic ../../libiberty/pex-unix.c -o pic/pex-unix.o; else true; fi
../../libiberty/pex-unix.c:59:10: fatal error: process.h: No such file or directory
   59 | #include 
      |    
process.h is not under /usr/include/ but at /usr/i686-w64-mingw32/sys-root/mingw/include/process.h .

Workaround: Pretend process.h is unavilable: env ac_cv_header_process_h=no ./rpmbuild --target=i586 ...


pcp compile fails, Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.

Symptom: pcp package compile fails:

Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead.
If this causes problems, reconfigure your locale. See the locale(1) manual
for more information.
Info: creating stash file /home/kabe/r9builds/pcp-r9/BUILD/pcp-6.2.0/src/libpcp_qed/src/.qmake.stash
Project ERROR: Unknown module(s) in QT: svg
make[5]: Makefile: No such file or directory
make[5]: *** No rule to make target 'Makefile'.  Stop.

Cause: qmake from QT6 (package qt6-qtbase-devel-6.x.x) is invoked, while system default QT is still QT 5. The error messages comes from /usr/lib/libQt6Core.so.6 .

When this ocuurs, plain qmake emits similar error:

[kabe@i686 pcp-6.2.0]$ qmake --version
Detected locale "C" with character encoding "ANSI_X3.4-1968", which is not UTF-8.
Qt depends on a UTF-8 locale, and has switched to "C.UTF-8" instead.
If this causes problems, reconfigure your locale. See the locale(1) manual
for more information.
QMake version 3.1
Using Qt version 6.5.2 in /usr/lib
[kabe@i686 pcp-6.2.0]$ _

Workaround: Define QMAKE=/usr/bin/qmake-qt5 , to select QT5 explicitly, i.e env QMAKE=/usr/bin/qmake-qt5 ./rpmbuild --target=i586 ... .


openssh fails to compile: /usr/include/sha2.h:63:16: error: redefinition of 'struct _SHA2_CTX'

Symptom: openssh fails to compile:

In file included from sk-usbhid.c:31:
/usr/include/sha2.h:63:16: error: redefinition of 'struct _SHA2_CTX'
   63 | typedef struct _SHA2_CTX {
      |                ^~~~~~~~~
In file included from openbsd-compat/openbsd-compat.h:46,
                 from includes.h:174,
                 from sk-usbhid.c:19:
openbsd-compat/sha2.h:66:16: note: originally defined here
   66 | typedef struct _SHA2_CTX {
      |                ^~~~~~~~~
In file included from sk-usbhid.c:31:
/usr/include/sha2.h:70:3: error: conflicting types for 'SHA2_CTX'; have 'struct _SHA2_CTX'
   70 | } SHA2_CTX;
      |   ^~~~~~~~

Cause: /usr/include/sha2.h, which is a member of EPEL libmd-devel, conflicts with OpenSSH counterpart.

Workaround: Uninstall libmd-devel package. Also, you do not want to depend on libbsd supplied by EPEL, so invoking ac_cv_search_SHA256Update=no ./rpmbuild ... is recommended.


I want the text console to be on lower resolution

Symptom: Booting into text console. The KMS console tries to use full-screen resolution, but I want it to be 800x600.

Resolution: add video=800x600 to kernel command line.

RHEL 9 derivatives will have Hyper-V console as a KMS driver, so video= kernel option will work. (Info: for RHEL 8, video=hyperv_fb:800x600 was needed)

Making change permanent

Edit /boot/loader/entries/*.conf to be

...
options root=UUID=244150bf-638d-423a-8ba6-a7e424fcad37 ro crashkernel=1G-4G:192M,4G-64G:256M,64G-:512M resume=UUID=c4d7fb5b-125b-4b2e-8bdb-eeebb7c5d8de video=800x600
...


I want the Xorg resolution to be lower, not 1024x768

Symptom: When I start Xorg by xinit, the Xorg modesettings driver ignores the current KMS resolution, and uses ex. 1024x768.

Workaround: Drop-in the following file as /etc/X11/xorg.conf.d/40-monitor800.conf:

Section "Monitor"
        Identifier "Monitor800x600"
        Modeline "m800x600"   40.00  800 840 968 1056  600 601 605 628 +hsync +vsync
        Option "PreferredMode" "m800x600"
EndSection

Section "Screen"
        Identifier "Screen800x600"
        Monitor "Monitor800x600"
EndSection
Option "PreferredMode" "m800x600" is the important part.


I do not want KDE to change the screen resolution

Symptom: I am satisfied with Xorg resolution by xinit, but when with KDE 5 installed, startx resets the screen resolution to 1024x768. I do not want KDE to change resolution.

Workaround: Stop the "KScreen 2" background service. Invoke "System Setting", navigate to "Startup and Shutdown" -> "Background Services", disable and stop "KScreen 2" service.


libnl compile fails under chroot

Symptom: libnl make check fails:
FAIL: tests/check-all
=====================

Running suite(s): main
 Abstract addresses
 Netlink attributes
 Clone ematch tree
 netns
37%: Checks: 29, Failures: 18, Errors: 0
tests/nl-test-util.c:127:S:Core:cache_and_clone:0: assert(r == 0) failed (errno=1, Operation not permitted)
tests/nl-test-util.c:127:S:Core:test_create_iface:0: assert(r == 0) failed (errno=1, Operation not permitted)
tests/nl-test-util.c:127:S:Core:test_create_iface:1: assert(r == 0) failed (errno=1, Operation not permitted)
...

Cause: unshare(CLONE_NEWUSER) seems to be not available under setarch i686 chroot /chroot/i686 environment. unshare -U ls also fails on commandline.


Compiling ruby fails on %check

Symptom: Compiling ruby. %check fails with:

  1) Failure:
TestGem#test_default_path [/home/kabe/r9builds/ruby-r9/BUILD/ruby-3.0.4/test/rubygems/test_gem.rb:616]:
<["/usr/share/gems"]> expected but was
<["/usr/share/gems", "/usr/local/share/gems"]>.
...

Workaround: You must temporary disable ruby on the compilation environment.

sudo sh -c ' \
if [ -d /usr/share/rubygems ]; then mv /usr/share/rubygems{,-}; fi; \
if [ -d /usr/share/ruby ]; then mv /usr/share/ruby{,-}; fi; \
if [ -d /usr/lib/ruby ]; then mv /usr/lib/ruby{,-}; fi; \
if [ -d /usr/share/gems ]; then mv /usr/share/gems{,-}; fi; \
if [ -e /usr/bin/ruby ]; then mv /usr/bin/ruby{,-}; fi; \
'
Do the opposite when you finished compiling.

Compile of librttopo fails with missing mingw32-debugfiles.list

Symptom: Compile of librttopo (EPEL) fails:

Processing files: mingw32-librttopo-debuginfo-1.1.0-9.el9.noarch
error: Could not open %files file /home/kabe/r9builds/librttopo-epel-r9/BUILD/librttopo/mingw32-debugfiles.list: No such file or directory

Workaround: Invoke rpmbuild with following options:

./rpmbuild --target=i586 -v -bb \
--without mingw \
-D 'mingw_build_win32 0' \
-D 'mingw_build_win64 0' \
-D 'mingw_build_ucrt64 0' \
...

clang %check fails: %clang_dxc: command not found

Symptom: clang compile fails in %check:

+ : 'RUN: at line 1'
+ %clang_dxc -Tlib_6_7 -fcgl -Fo - /home/kabe/r9builds/clang-r9/BUILD/clang-16.0.6.src/test/Driver/hlsl_no_stdinc.hlsl -###
+ /usr/bin/FileCheck /home/kabe/r9builds/clang-r9/BUILD/clang-16.0.6.src/test/Driver/hlsl_no_stdinc.hlsl --check-prefix=STDINC
/home/kabe/r9builds/clang-r9/BUILD/clang-16.0.6.src/test/Driver/hlsl_no_stdinc.hlsl:9:11: error: STDINC: expected string not found in input
// STDINC:"-finclude-default-header"
          ^
<stdin>:1:1: note: scanning from here
/home/kabe/r9builds/clang-r9/BUILD/clang-16.0.6.src/redhat-linux-build/test/Driver/Output/hlsl_no_stdinc.hlsl.script: line 1: %clang_dxc: command not found
^
<stdin>:1:58: note: possible intended match here
/home/kabe/r9builds/clang-r9/BUILD/clang-16.0.6.src/redhat-linux-build/test/Driver/Output/hlsl_no_stdinc.hlsl.script: line 1: %clang_dxc: command not found

Cause: python3-lit-15.0.7-1.el9.noarch.rpm package is too old. Newer python3-lit-16.0.6-2.el9.noarch.rpm has definitions for ToolSubst('%clang_dxc', command=self.config.clang, .


lorax fails with missing libavahi-common.so.3

Symptom: lorax fails with:

2023-02-23 18:30:02,225: libavahi-common.so.3, needed by /usr/bin/ostree, not found
libavahi-common.so.3, needed by /usr/bin/ostree, not found
2023-02-23 18:30:02,226: libavahi-client.so.3, needed by /usr/bin/ostree, not found
libavahi-client.so.3, needed by /usr/bin/ostree, not found

Cause: lorax erases avahi-glib package during run to save space:

pylorax.log:template line 257: removepkg opus libtheora libvisual flac-libs gsm avahi-glib avahi-libs ModemManager-glib
...
pylorax.log:removepkg avahi-glib: 58kb
pylorax.log:removed /var/tmp/lorax/lorax.o1jrqzmp/installtree/usr/lib/libavahi-glib.so.1.0.2
pylorax.log:removed /var/tmp/lorax/lorax.o1jrqzmp/installtree/usr/lib/libavahi-glib.so.1
pylorax.log:removepkg avahi-libs: 489kb

Resolution: Compile ostree package without avahi dependency: with_avahi=no CPPFLAGS="-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" ./rpmbuild


Spinned DVD1.iso exceeds 4.7GB

Symptom: Spinned DVD1.iso exceeds 4.7GB limit, like 5.2GB.

Workaround: Exclude texlive, mingw and gcc-toolset-12 from the media. You are not likely to need it immediately.

xorrisofs -o ./DVD1.iso \
-b isolinux/isolinux.bin -c isolinux/boot.cat \
-no-emul-boot -boot-load-size 4 -boot-info-table \
-R -T -J \
-v \
-V "Rocky-Linux-9-1-i386" \
-m upgrade.img -m boot.iso \
-m 'texlive-*' -m 'mingw*' -m 'gcc-toolset-12-*' \
-no-pad -iso-level 3 -D --hardlinks -joliet-long \
./img9

Cannot login on console under heavy load

Symptom: Under heavy load, login on vt console is rejected by

Login timed out after 60 seconds

Resolution: Add login(1) LOGIN_TIMEOUT in /etc/login.defs:

## default 60secs is short under heavy load
LOGIN_TIMEOUT	300

You may have to boot into single user mode to make this change, since multi-user mode keeps CPU load > 5 for several minutes.


When I add multiple input methods to ibus, the tray icon disappears

Symptom: Using the "Breeze dark" scheme, which is default in RHEL 9.1 era EPEL. When I add multiple input methods, the tray icon disappears. It is still there; hovering the mouse cursor will show the icon in thin color.

Workaround: Use "Breeze" scheme, from Settings - Appearance - Plasma Style. The tray icon coloring seems to be scheme dependent.


KDE does not start, systemd shows timeout

Symptom: Starting KDE by startkde (startx) does not work. journalctl shows plasma related services repeatedly timeout:

Mar 19 17:32:21 cygnus.five.ten systemd[598]: plasma-polkit-agent.service: Failed with result 'timeout'.
?? Subject: Unit failed
?? Defined-By: systemd
?? Support: https://access.redhat.com/support
??
?? The unit UNIT has entered the 'failed' state with result 'timeout'.
Mar 19 17:32:21 cygnus.five.ten systemd[598]: Failed to start KDE PolicyKit Authentication Agent.

Mar 19 17:32:16 cygnus.five.ten systemd[598]: plasma-powerdevil.service: start operation timed out. Terminating.
Mar 19 17:32:16 cygnus.five.ten systemd[598]: plasma-powerdevil.service: Failed with result 'timeout'.
?? Subject: Unit failed
?? Defined-By: systemd
?? Support: https://access.redhat.com/support
??
?? The unit UNIT has entered the 'failed' state with result 'timeout'.
Mar 19 17:32:16 cygnus.five.ten systemd[598]: Failed to start Powerdevil.
?? Subject: A start job for unit UNIT has failed
?? Defined-By: systemd
?? Support: https://access.redhat.com/support
??
?? A start job for unit UNIT has finished with a failure.
??
?? The job identifier is 13121 and the job result is failed.
Mar 19 17:32:17 cygnus.five.ten systemd[598]: plasma-powerdevil.service: Scheduled restart job, restart counter is at 372.

Workaround: Enlenghen timeout for systemctl --user services. Use a systemd drop-in facility. Create the following directories:

mkdir /etc/systemd/user/gmenudbusmenuproxy.service.d
mkdir /etc/systemd/user/plasma-baloorunner.service.d
mkdir /etc/systemd/user/plasma-kactivitymanagerd.service.d
mkdir /etc/systemd/user/plasma-ksmserver.service.d
mkdir /etc/systemd/user/plasma-kwin_x11.service.d
mkdir /etc/systemd/user/plasma-plasmashell.service.d
mkdir /etc/systemd/user/plasma-polkit-agent.service.d
mkdir /etc/systemd/user/plasma-powerdevil.service.d
mkdir /etc/systemd/user/plasma-xembedsniproxy.service.d
and drop-in the following as timeout.conf :
[Service]
TimeoutSec=600sec

Scripted as:

for i in \
gmenudbusmenuproxy.service \
plasma-baloorunner.service \
plasma-kactivitymanagerd.service \
plasma-kcminit.service \
plasma-ksmserver.service \
plasma-kscreen.service \
plasma-kded.service \
plasma-kwin_x11.service \
plasma-plasmashell.service \
plasma-polkit-agent.service \
plasma-powerdevil.service \
plasma-xembedsniproxy.service \
plasma-xdg-desktop-portal-kde.service \
xdg-desktop-portal.service \
; do mkdir /etc/systemd/user/$i.d; cat << 'EOF' > /etc/systemd/user/$i.d/timeout.conf;
[Service]
TimeoutSec=600sec
EOF
done

cat << 'EOF'
do:
systemctl --no-reload --user mask plasma-kscreen
systemctl --no-reload --user mask plasma-powerdevil
systemctl --user daemon-reload
EOF

Do systemctl --user daemon-reload, and check that setting is valid by systemctl --user show plasma-plasmashell and search for TimeoutStartUSec=10min .

By default, plasma-powerdevil.service and plasma-polkit-agent.service has TimeoutSec=5sec, which is way too short for slow machines.


Wi-Fi adapter was usable during install, but not after reboot

Symptom: Wi-Fi adapter was recognised and could connect during anaconda installation, but not after HDD reboot.

Cause: Probably NetworkManager-wifi package is missing. It is not included in "minimum" install.

Resolution: Install NetworkManager-wifi package, then systemctl restart NetworkManager .


/usr/libexec/dconf-service: CPU ISA level is lower than required

Resolution: Recompile the named binary in "setarch i686" environment with glibc.i586 .

How to know "CPU ISA level" restriction is embedded

The restriction is checked by glibc.

$ objdump -j .note.gnu.property -s /usr/bin/dconf

/usr/bin/dconf:     file format elf32-i386

Contents of section .note.gnu.property:
01ec 04000000 18000000 05000000 474e5500  ............GNU.
01fc 020000c0 04000000 03000000 028000c0  ................
020c 04000000 01000000            |
     |        |                   |
     |        flag(0x01)        [swab]
     4 bytes size follows         |
                                  |
GNU_PROPERTY_X86_ISA_1_NEEDED   0xc0008002

[KDE5] Properties menu of keyboard icon tasktray does not open

Symptom: Selecting "Properties" in the keyboard icon in the task tray does not invoke anything. From command line, it dies with error:

$ python3 /usr/share/ibus/setup/main.py ibus-setup

(ibus-setup:1720): Gtk-WARNING **: 20:53:09.622: Could not load a pixbuf from /org/gtk/libgtk/theme/Adwaita/assets/check-symbolic.svg.
This may indicate that pixbuf loaders or the mime database could not be found.
**
Gtk:ERROR:../gtk/gtkiconhelper.c:494:ensure_surface_for_gicon: assertion failed (error == NULL): Failed to load /usr/share/icons/Adwaita/16x16/status/image-missing.png: Unrecognized image file format (gdk-pixbuf-error-quark, 3)
Bail out! Gtk:ERROR:../gtk/gtkiconhelper.c:494:ensure_surface_for_gicon: assertion failed (error == NULL): Failed to load /usr/share/icons/Adwaita/16x16/status/image-missing.png: Unrecognized image file format (gdk-pixbuf-error-quark, 3)

Points to check:


Cannot connect to WPA2-only (no WPA) access point using ath5k driver

Symptom: Using Wi-Fi adapter using ath5k kernel driver. SSID scan works, but cannot connect to access points with WPA disabled (only WPA2-PSK).

Cause: Some ath5k hardware seems to not capable of WPA2 hardware offloading.

Workaround: use nohwcrypt=1 ath5k option. Add ath5k.nohwcrypt=1 to kernel boot options, or place file below as /etc/modprobe.d/ath5k.conf (see modprobe.d(5))

options ath5k nohwcrypt=1

cannot change input method

Symptom:Trying to switch ibus input method. Click-selecting the keyboard icon, or using [Super]-Space, does show the alternate input method (after terrible delay), but keyboard icon doesn't change and input method doesn't change.

Cause: Default timeout of 5 seconds may be too short for slow machines

Resolution: set a longer timeout on ibus-daemon.

% pkill ibus-daemon
% IBUS_TIMEOUT=120000 /usr/bin/ibus-daemon -d -r --xim --timeout=120000

There are two configurable timeouts for ibus-daemon; ibus timeout (IBUS_TIMEOUT), which defaults to 6 seconds, and gdbus timeout (--timeout) which defaults to 5 seconds. These may be too short for slow machines. Above example sets them to 120 seconds. (--timeout seems to be enough)

Making change permanent

Edit /etc/X11/xinit/xinput.d/ibus.conf as:

XIM=ibus
XIM_PROGRAM="/usr/bin/ibus-daemon"
ICON="ibus"
XIM_ARGS="-r --xim --timeout=120000"
PREFERENCE_PROGRAM=/usr/bin/ibus-setup
...


I do not want bash find nonexistent repository for mistyped command

Symptom: It is very annoying when bash goes to find into noexistent repository for mistyped command:

$ aa
bash: aa: command not found...
Long pause
Failed to search for file: cannot update repo 'extras': Cannot prepare internal mirrorlist: Status code: 404 for https://mirrors.rockylinux.org/mirrorlist?arch=i386&repo=extras-9 (IP: 151.101.42.132); Last error: Status code: 404 for https://mirrors.rockylinux.org/mirrorlist?arch=i386&repo=extras-9 (IP: 146.75.94.132)

Resolution: Uninstall PackageKit-command-not-found package. The find hook is defined in /etc/profile.d/PackageKit.sh .

rpm -ev PackageKit-command-not-found


Is xorg-x11-drv-nv worth over nouveau.ko kernel driver for NVidia Geforce cards?

xbench result of nouveau.ko modeset and xorg-x11-drv-nv does not much differ. xorg-x11-drv-nv has about 20% overall performance gain on X11 primitive operations, but since nouveau.ko kernel module is very stable comapred to Rocky Linux 8 counterpart, just sticking to nouveau.ko is not unwise.


Cannot start Xorg with radeon.ko driver blacklisted

Symptom: Blacklisted radeon.ko KMS driver by kernel option module_blacklist=radeon, nomodeset or /etc/modprobe.d/radeon.conf with blacklist radeon, since KMS radeon.ko driver has horrible performance when scrolling the console. Installed xorg-x11-drv-ati driver. When I try starting Xorg, it fails:

[  5552.173] (II) [KMS] drm report modesetting isn't supported.
[  5552.175] (EE) Screen 0 deleted because of no matching config section.

Cause: xorg-x11-drv-ati needs KMS driver to work. Do not blacklist radeon.ko kernel driver.


Cannot start Xorg with mgag200.ko driver blacklisted

Symptom: Blacklisted mgag200.ko kernel module, since KMS (DRM) console has horrible scrollong performance. Xorg does not start, with /var/log/Xorg.0.log saying:

[   560.593] (II) LoadModule: "mga"
[   560.621] (II) Loading /usr/lib/xorg/modules/drivers/mga_drv.so
[   560.640] (II) Module mga: vendor="X.Org Foundation"
[   560.640] 	compiled for 1.20.11, module version = 2.0.1
[   560.641] 	Module class: X.Org Video Driver
[   560.642] 	ABI class: X.Org Video Driver, version 24.1
[   560.656] (II) LoadModule: "modesetting"
[   560.677] (II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
[   560.809] (II) Module modesetting: vendor="X.Org Foundation"
[   560.810] 	compiled for 1.20.11, module version = 1.20.11
[   560.811] 	Module class: X.Org Video Driver
[   560.811] 	ABI class: X.Org Video Driver, version 24.1
[   560.812] (II) LoadModule: "fbdev"
[   560.829] (II) Loading /usr/lib/xorg/modules/drivers/fbdev_drv.so
[   560.850] (II) Module fbdev: vendor="X.Org Foundation"
[   560.851] 	compiled for 1.20.11, module version = 0.5.0
[   560.852] 	Module class: X.Org Video Driver
[   560.852] 	ABI class: X.Org Video Driver, version 24.1
[   560.853] (II) LoadModule: "vesa"
[   560.887] (WW) Warning, couldn't open module vesa
[   560.888] (EE) Failed to load module "vesa" (module does not exist, 0)
[   560.889] (II) MGA: driver for Matrox chipsets: mga2064w, mga1064sg, mga2164w,
	mga2164w AGP, mgag100, mgag100 PCI, mgag200, mgag200 PCI,
	mgag200 SE A PCI, mgag200 SE B PCI, mgag200 EV Maxim,
	mgag200 ER SH7757, mgag200 eW Nuvoton, mgag200 eW3 Nuvoton,
	mgag200eH, mgag200eH3, mgag400, mgag550
[   560.914] (II) modesetting: Driver for Modesetting Kernel Drivers: kms
[   560.915] (II) FBDEV: driver for framebuffer: fbdev
[   560.963] (WW) Falling back to old probe method for modesetting
[   560.965] (EE) open /dev/dri/card0: No such file or directory
[   560.965] (WW) Falling back to old probe method for fbdev
[   560.986] (II) Loading sub module "fbdevhw"
[   560.987] (II) LoadModule: "fbdevhw"
[   560.993] (II) Loading /usr/lib/xorg/modules/libfbdevhw.so
[   561.023] (II) Module fbdevhw: vendor="X.Org Foundation"
[   561.023] 	compiled for 1.20.11, module version = 0.0.2
[   561.024] 	ABI class: X.Org Video Driver, version 24.1
[   561.045] (EE) open /dev/fb0: No such file or directory
[   561.099] (EE) No devices detected.
[   561.100] (EE) 
Fatal server error:
[   561.102] (EE) no screens found(EE) 
[   561.104] (EE) 

Cause: RHEL 9 does not support video cards without DRM drivers, as mentioned in patch of xorg-x11-server package, 0001-mustard-xfree86-Disable-the-PCI-probe-path.patch . Un-applying this patch makes Xorg to not work on Hyper-V.

Workaround: Do not disable mgag200.ko kernel module.


Color is weird on ATI Radeon graphics card

Symptom: xinit -- /usr/bin/Xorg -retro causes weird color for root window stipple, when using xorg-x11-drv-ati Xorg driver. Does not occur without xorg-x11-drv-ati and using modeset driver.

Resolution: Add Option "AccelMethod" "glamor" (The newer AccelMethod) to the Xorg setting. Add following as /etc/X11/xorg.conf.d/30-radeon.conf:

Section "Device"
	Identifier "radeon-dev"
	Driver "radeon"
	Option "AccelMethod" "glamor"
EndSection

Cannot get 1360x768 widescreen for X on Matrox MGA G200 video card

Symptom: Using Matrox MGA G200 video card. On boot, mgag200.ko KMS module is loaded, and video is limited to 1024x768, although the monitor is 1360x768.

Workaround:

  1. Prepare /usr/X11/xorg.conf.d/40-mgamonitor.conf as below:
    Section "Monitor"
    	Identifier "Monitor0"
    	#Option "ReducedBlanking"
    	VertRefresh 55-65
    	#Option "MaxClock" "67000"
    	ModeLine "m1360x768"  85.5  1360 1424 1536 1792  768 771 777 795 +hsync +vsync
    	#Option "PreferredMode" "m1360x768"
    	#Option "DefaultMode" "m1360x768"
    EndSection
    
    Section "Screen"
    	Identifier "Screen0"
    	Monitor "Monitor0"
    	SubSection "Display"
    		Virtual 1360 768
    		Viewport 1360 768
    		#Modes "m1360x768"
    	EndSubSection
    EndSection
    
    "Virtual 1360 768" "Viewport 1360 768" is the important part. xorg-x11-drv-mga seems to use 1368x768 for unknown reason.

You still need this 40-mgamonitor.conf with #Modes "m1360x768" uncommented, when using modesetting driver (without xorg-x11-drv-mga) and setting console to 1360x768 by kernel option video=1360x768 .


Scroll speed slow on DRM KMS fbdev console

Symptom: Scrolling in DRM KMS fbdev console is slow.

Workaround: There is Scroll speed impovement patch, which is merged into kernel-6.x series (commit 6f29e049,0d030118,61bfcb6a,7438f52b ) but not backported for kernel-5.x line, which is the base for RHEL 9.

The kernel provided by this site has this acceleration patch backported, and additional acceleration by assembly. It makes scrolling faster by x2 for modern hardware, +30% gain in i586.


AttributeError: 'LitConfig' object has no attribute 'substitute' when compiling clang

Symptom: When compiling clang, following error occurs:
+ SOURCE_DATA_EPOCH=1629181597
+ LD_LIBRARY_PATH=/home/kabe/r9builds/clang-r9/BUILDROOT/clang-15.0.7-2.el9_1.v1.i386//usr/lib
+ /usr/bin/ninja check-all -C redhat-linux-build
ninja: Entering directory `redhat-linux-build'
[0/1] Running all regression tests
lit: /usr/lib/python3.9/site-packages/lit/TestingConfig.py:102: fatal: unable to parse config file '/home/kabe/r9builds/clang-r9/BUILD/clang-15.0.7.src/redhat-linux-build/tools/extra/include-cleaner/test/lit.site.cfg.py', traceback: Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/lit/TestingConfig.py", line 91, in load_from_path
    exec(compile(data, path, 'exec'), cfg_globals, None)
  File "/home/kabe/r9builds/clang-r9/BUILD/clang-15.0.7.src/redhat-linux-build/tools/extra/include-cleaner/test/lit.site.cfg.py", line 14, in 
    config.llvm_tools_dir = path(lit_config.substitute("/usr/bin"))
AttributeError: 'LitConfig' object has no attribute 'substitute'

FAILED: CMakeFiles/check-all
Resolution:python3-lit.noarch package is old. Replace with Rocky Linux 9.2 version, python3-lit-15.0.7-1.el9.noarch.rpm . This version has substitute method defined.

kernel compile fails at last stage: error: Illegal sequence ".." in: 0x........

Symptom: When compiling the kernel, it fails at very last stage with following errors:

error: Illegal sequence ".." in: 0x........
error: Dependency tokens must begin with alpha-numeric, '_' or '/': .
error: Illegal char '<' (0x3c) in: 0x......<.

Cause: /usr/lib/rpm/redhat/find-provides.ksyms assumes output of readelf -R .rodata module.ko is always 16-byte aligned.

When .rodata is not aliged at 16 bytes, e.g

$ readelf -R .rodata /tmp/module.ko

Hex dump of section '.rodata':
  0x00000000 00000000 12001000 13000000 13080800 ................
  0x00000010 00000000 13000800 29000000 14080800 ........).......
  0x00000020 3c000000 14000800                   <.......
/usr/lib/rpm/redhat/find-provides.ksyms assembles an hex string like:
$ readelf -R .rodata /tmp/module.ko | awk '/0x/{printf $2$3$4$5}'
00000000120010001300000013080800000000001300080029000000140808003c00000014000800<.......
which has ASCII dump part appended as a garbage.

Resolution: Patch /usr/lib/rpm/redhat/find-provides.ksyms .

--- /usr/lib/rpm/redhat/find-provides.ksyms.dist	2022-05-16 21:07:11.000000000 +0900
+++ /usr/lib/rpm/redhat/find-provides.ksyms	2023-06-29 18:20:52.545271482 +0900
@@ -39,7 +39,7 @@ for module in $(grep -E '/lib/modules/.+
         | awk --non-decimal-data '{printf("'"${dep_pfx}"'(%s) = 0x%08x\n", $2, $1)}' \
         | LC_ALL=C sort -u
     else
-        ELFRODATA=$(readelf -R .rodata $module | awk '/0x/{printf $2$3$4$5}')
+        ELFRODATA=$(readelf -R .rodata $module | cut -b 1-48 | awk '/0x/{printf $2$3$4$5}')
         if [[ -n $(readelf -h $module | grep "little endian") ]]; then
             RODATA=$(echo $ELFRODATA | sed 's/\(..\)\(..\)\(..\)\(..\)/\4\3\2\1/g')
         else

libssh2 compile fails in %check

Symptom: libssh2 (from EPEL) fails in %check stage.

FAIL: ssh2.sh
============================================================================
Testsuite summary for libssh2 -
============================================================================
# TOTAL: 3
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to libssh2-devel@cool.haxx.se
============================================================================

tests/test-suite.log:

command-line line 0: Deprecated option UsePrivilegeSeparation
main: sshd: ssh-rsa algorithm is disabled
command-line line 0: Deprecated option UsePrivilegeSeparation
Failure establishing SSH session
FAIL ssh2.sh (exit status: 1)

Cause: ssh-rsa cipher used in tests/etc/{host,user} is not supported by DEFAULT policy of update-crypto-policies --show.

Workaround: do one of following:


Orphan file /usr/lib/qt5/plugins/generic/libqtslibplugin.so during qt5-qtbase compile

Symptom: During qt5-qtbase package build, at the end, orphaned files are reported and build fails:

    Installed (but unpackaged) file(s) found:
   /usr/lib/cmake/Qt5Gui/Qt5Gui_QTsLibPlugin.cmake
   /usr/lib/debug/usr/lib/qt5/plugins/generic/libqtslibplugin.so-5.15.3-1.el9.v1.i386.debug
   /usr/lib/qt5/plugins/generic/libqtslibplugin.so

Cause: tslib functionality is automatically turned on when tslib-devel is installed, but qt5-qtbase.spec does not count it.

Resolution: in qt5-qtbase.spec, do ./configure ... -no-tslib .


rust-* compile fails in %install, no /usr/share/cargo/registry/*

Symptom: When compiling rust-* packages in EPEL, %install fails to find /usr/share/cargo/registry/*:

Processing files: rust-cargo-c-devel-0.9.12-4.el9.noarch
error: File not found: /home/kabe/r9builds/rust-cargo-c-epel-r9/BUILDROOT/rust-cargo-c-0.9.12-4.el9.i386/usr/share/cargo/registry/cargo-c-0.9.12/LICENSE
error: File not found: /home/kabe/r9builds/rust-cargo-c-epel-r9/BUILDROOT/rust-cargo-c-0.9.12-4.el9.i386/usr/share/cargo/registry/cargo-c-0.9.12/README.md
error: Directory not found: /home/kabe/r9builds/rust-cargo-c-epel-r9/BUILDROOT/rust-cargo-c-0.9.12-4.el9.i386/usr/share/cargo/registry/cargo-c-0.9.12

Cause: %__cargo_install rpmbuild macro is redefined to bogus one by rust-toolset package.

Workaround: Uninstall rust-toolset (and python3-setuptools-rust dependency) before compiling.


lorax bombs out by missing perl(Exporter)

Symptom: lorax bombs out by:

 Problem 1: package perl-interpreter-4:5.32.1-480.el9.v1.i586 requires libperl.so.5.32, but none of the providers can be installed
  - package perl-interpreter-4:5.32.1-480.el9.v1.i586 requires perl(:MODULE_COMPAT_5.32.1), but none of the providers can be installed
  - package perl-interpreter-4:5.32.1-480.el9.v1.i586 requires perl-libs, but none of the providers can be installed
  - package perl-interpreter-4:5.32.1-480.el9.v1.i586 requires perl-libs(x86-32) = 4:5.32.1-480.el9.v1, but none of the providers can be installed
  - conflicting requests
  - nothing provides perl(Exporter) needed by perl-libs-4:5.32.1-480.el9.v1.i586
  - nothing provides perl(Carp) needed by perl-libs-4:5.32.1-480.el9.v1.i586
  - nothing provides perl(Encode) needed by perl-libs-4:5.32.1-480.el9.v1.i586
  - nothing provides perl(File::Spec) needed by perl-libs-4:5.32.1-480.el9.v1.i586

Cause: perl-Exporter package was not built during perl.src build.

Resolution: Add a flag to rpmbuild of perl: ./rpmbuild -D 'perl_bootstrap 1' ... SPECS/perl.spec This will build perl-Exporter package.


pipewire-jack-audio-connection-kit conflicts with jack-audio-connection-kit

Symptom: Conflict error reported during dnf group install kde-desktop-environment:

Error:
 Problem: problem with installed package jack-audio-connection-kit-1.9.21-1.el9.v1.i586
  - package pipewire-jack-audio-connection-kit-0.3.47-3.el9_2.v1.i586 conflicts with jack-audio-connection-kit provided by jack-audio-connection-kit-1.9.21-1.el9.v1.i586
  - conflicting requests

Resolution: You should not have been compiling jack-audio-connection-kit (EPEL) package in first place. pipewire-jack-audio-connection-kit-libs package compiled from pipewire.src package will provide libjack.so.0 dependency. Check rpmbuild --with jack is in effect during pipewire.src build.

Wrong Resolution: Rebuild pipewire.src package with ./rpmbuild --without jack. This will make it not build the conflicting pipewire-jack-audio-connection-kit package.


No sound available, pipewire device not working

Symptom: aplay -L shows pipewire device,

$ aplay -L
null
    Discard all samples (playback) or generate zero samples (capture)
sysdefault
    Default Audio Device
pipewire
    PipeWire Sound Server
default
    Default ALSA Output (currently PipeWire Media Server)
sysdefault:CARD=I82801DBICH4
    Intel 82801DB-ICH4, Intel 82801DB-ICH4
    Default Audio Device
...
but
aplay /usr/share/sounds/alsa/Front_Center.wav
playes no sound and does not return. (after checking volume is up by alsamixer -D hw:0)

Cause: RHEL 9.2 supplied pipewire-0.3.47-3.el9_2 and wireplumber-0.4.8-1.el9 seems to have a bug.

Resolution: Compile and install pipewire and wireplumber package with Fedora 39 supplied ones, that is, pipewire-0.3.80-1.fc39 and wireplumber-0.4.14-2.fc39 .


pw-play says "stream node 32 error: no node available"

Symptom: pw-play says:

$ pw-play /usr/share/sounds/alsa/Front_Center.wav
stream node 32 error: no node available
remote error: id=2 seq:7 res:-2 (No such file or directory): no node available
$ _

Workaround: The error is intermittent, if you had already replaced pipewire and wireplumber packages to Fedora 39 version. The pipewire.service did not woke up fast enough in response to pipewire.socket access.

$ pw-play /usr/share/sounds/alsa/Front_Center.wav
stream node 32 error: no node available
remote error: id=2 seq:7 res:-2 (No such file or directory): no node available
$ pw-play /usr/share/sounds/alsa/Front_Center.wav
the sound plays
$ _

You could reset the state (first pw-play fails) by systemctl --user stop pipewire.service .


Wireplumber freezes the entire system when used with snd-sbawe

Symptom: Using Creative Sound Blaster AWE32 sound card. This card is an ISA-PNP card, so manual module load is required. When the following sequence is executed, entire system freezes.

$ sudo modprobe snd-sbawe seq_ports=0
$ systemctl --user start pipewire wireplumber

Cause: Unknown.

Workaround: Invoke pipewire and wireplumber one by one:

$ systemctl --user mask wireplumber
$ sudo modprobe snd-sbawe seq_ports=0
$ systemctl --user start pipewire
$ systemctl --user unmask wireplumber
$ systemctl --user start wireplumber
(This is not a bulletproof solution; some kernels still freeze by above sequence)

Above sequence will keep CPU usage in "top" around "50.0 sy", so Invoke pw-play command to revive CPU usage back to sane value:

$ pw-play /usr/share/sounds/alsa/Front_Center.wav
Wait for couple of minutes. Sound plays, and CPU usage goes down to normal. (Reason that pw-play fixes CPU usage is unknown)

No sound available from xinit/startx

Symptom: Starting Xorg by adding args to Xorg:

$ xinit -- /usr/bin/Xorg -retro
$ startx /usr/bin/xterm -- /usr/bin/Xorg -retro
Inside the X session, aplay aborts:
$ aplay  /usr/share/sounds/alsa/Front_Center.wav
Playing WAVE '/usr/share/sounds/alsa/Front_Center.wav' : Signed 16 bit Little Endian, Rate 48000 Hz, Mono
aplay: set_params:1416: Unable to install hw params:
ACCESS:  RW_INTERLEAVED
FORMAT:  S16_LE
SUBFORMAT:  STD
SAMPLE_BITS: 16
FRAME_BITS: 16
CHANNELS: 1
RATE: 48000
PERIOD_TIME: 125000
PERIOD_SIZE: 6000
PERIOD_BYTES: 12000
PERIODS: 4
BUFFER_TIME: 500000
BUFFER_SIZE: 24000
BUFFER_BYTES: 48000
TICK_TIME: [0 0]
$ _

Cause: When Xorg is invoked without vt1 parameter, it tries to run on the next vt, thus vt2, but systemd-logind does not recognise that vt2 is owned by the user and does not add write permission to /dev/snd/controlC0.

Proper permission of the sound device file should be:

$ getfacl /dev/snd/controlC0
getfacl: Removing leading '/' from absolute path names
# file: dev/snd/controlC0
# owner: root
# group: audio
user::rw-
user:kabe:rw-
group::rw-
mask::rw-
other::---
When audio is not working, the emphasised part is not present. You can switch back to tty1 (Ctrl-Alt-F1) and test aplay, which works.

Workaround:


[KDE5] Starting KDE invokes the screenlocker and cannot reach desktop on slow machine

Symptom: Started KDE by startx on a slow machine. Since it's slow, it invokes the default screenlocker during startup, and the machine is slow enough to be screenlocker reinvoked after disarming the screenlocker. I can't access the desktop. systemsettings can't be invoked to change the setting.

Workaround: Manually disable the spash screen and screenlocker by creating the following files:

mkdir -p ~/.config
cat > ~/.config/ksplashrc << 'EOF'
# ~/.config/ksplashrc
[KSplash]
Engine=none
Theme=None
EOF
mkdir -p ~/.config/
cat > ~/.config/kscreenlockerrc << 'EOF'
# ~/.config/kscreenlockerrc
[Daemon]
Autolock=false
EOF


[KDE5] Black background/wallpaper on KDE desktop

Symptom: Initial user, started KDE desktop. The wallpaper is black.
journalctl -xe reveals:

Oct 22 15:51:18 rocky9i686.five.ten plasmashell[1757]: qml: The backend got an unknown wallpaper provider type. The wallpaper will now fall back to the default. Please check your wallpaper configuration!

Cause: The plugin for reading *.webp background/wallpaper is missing.

Resolution (Compile and) install qt5-qtimageformats package. This will install /usr/lib/qt5/plugins/imageformats/libqwebp.so, the plugin for reading images written in *.webp format.


[KDE5] Black background/black wallpaper on KDE desktop, no taskbar

Symptom: Started KDE desktop, but plasmashell seems to not working. Black background/wallpaper, no task bar. journalctl is recording:

Oct 30 20:01:03 cygnus.five.ten kded5[3121]: QProcess: Destroyed while process ("/usr/libexec/kf5/kconf_update") is still running.
...
Oct 31 19:43:42 cygnus.five.ten kconf_update[4849]: QProcess: Destroyed while process ("/usr/lib/kconf_update_bin/spectacle-migrate-shortcuts") is still running

Cause: Hardcoded default timeout in /usr/include/qt5/QtCore/qprocess.h:

    bool waitForFinished(int msecs = 30000);
(30 secs) is too short for slow machines.

Workaround: Patch that line in qt5-qtbase.src to be:

    bool waitForFinished(int msecs = 300000); //5 minutes
and, install it on the compile machine and target machine, then recompile kf5-kconfig and plasma-workspace, and install them on target machine.


[KDE5] No "Audio" settings in systemsettings

Resolution: Install plasma-pa (EPEL) package. Be sure you have pipewire-pulseaudio (not pulseaudio) package to satisfy pulseaudio-daemon dependency.


BCM43xx based WiFi does not work

Symptom: On insertion of Broadcom BCM43xx based (Cardbus) WiFi adapter, dmesg or /var/log/messages says as below and wlan0 interface does not appear under ip link:

[ 2835.272754] pcmcia_socket pcmcia_socket0: pccard: CardBus card inserted into slot 0
[ 2835.272803] pci 0000:03:00.0: [14e4:4320] type 00 class 0x028000
[ 2835.272828] pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x00001fff]
[ 2835.273081] pci 0000:03:00.0: BAR 0: assigned [mem 0xd4000000-0xd4001fff]
[ 2836.492280] b43-pci-bridge 0000:03:00.0: enabling device (0000 -> 0002)
[ 2836.492418] ssb: Found chip with id 0x4306, rev 0x03 and package 0x00
[ 2836.497229] b43-pci-bridge 0000:03:00.0: Sonics Silicon Backplane found on PCI device 0000:03:00.0
[ 2836.742010] b43-phy1: Broadcom 4306 WLAN found (core revision 5)
[ 2836.767037] b43-phy1: Found PHY: Analog 2, Type 2 (G), Revision 2
[ 2836.767064] b43-phy1: Found Radio: Manuf 0x17F, ID 0x2050, Revision 2, Version 0
[ 2836.797769] Broadcom 43xx driver loaded [ Features: PNLS ]
[ 2836.902466] b43 ssb0:0: Direct firmware load for b43/ucode5.fw failed with error -2
[ 2836.902543] b43 ssb0:0: Direct firmware load for b43/ucode5.fw failed with error -2
[ 2836.902636] b43 ssb0:0: Direct firmware load for b43-open/ucode5.fw failed with error -2
[ 2836.903994] b43 ssb0:0: Direct firmware load for b43-open/ucode5.fw failed with error -2
[ 2836.904004] b43-phy1 ERROR: Firmware file "b43/ucode5.fw" not found
[ 2836.904009] b43-phy1 ERROR: Firmware file "b43-open/ucode5.fw" not found
[ 2836.904013] b43-phy1 ERROR: You must go to https://wireless.wiki.kernel.org/en/users/Drivers/b43#devicefirmware and download the correct firmware for this driver version. Please carefully read all instructions on this website.

Resolution: Install a firmware package. b43-openfwwf-5.2-10.el6.noarch.rpm supplied with CentOS 6 works. If it doesn't, you need to use b43-fwcutter by yourself to extract a firmware.

Note: b43.ko driver is not supplied in stock Rocky Linux 9 kernel.

To make the driver work in installation/rescue disc, you must add b43-openfwwf-5.2-10.el6.noarch.rpm in ix86.repo/BaseOS/Packages/, createrepo, and pass --installpkgs=b43-openfwwf to lorax.


systemd-coredump does not generate core, although it seems to be invoked

Symptom: systemd-coredump seems to be running on occation, but no core is generated. coredumpctl shows no core.

Cause: Default RuntimeMaxUSec=5min was not enough for slow machine to dump the core.

[kabe@rocky9i686 system]$ systemctl show systemd-coredump@0.service | grep RuntimeMax
RuntimeMaxUSec=5min

Resolution: Enlenghten the RuntimeMaxSec for the service:

# mkdir /etc/systemd/system/systemd-coredump@.service.d
# cat << 'EOF' > /etc/systemd/system/systemd-coredump@.service.d/timeout.conf
[Service]
TimeoutSec=1200sec
RuntimeMaxSec=1200sec
EOF
# systemctl daemon-reload

Do invistigate the coredump by coredumpctl debug pid and report the bug to the appropriate place.


kabe$sra-tohoku.co.jp