Creating CentOS 8.3 for 32bit i686/i586

[Screenshot of CentOS 8.3 desktop on Pentium 120MHz]

This page is an attempt to create a CentOS 8 for 32bit i686/i586.

CentOS 8.x does not have 32bit i686 distribution, and low on probability provided, since many essential packages are not side-by-side compiled for i686. (Note that CentOS 7.x i686 distribution was based on i686 artifacts compiled for 32bit compatibility)

Warning: This doesn't work on MMX-only processors. Processors with MMX but no SSE will execute some SSE instructions as MMX instruction, which lacks #UD exception for emulating it.
Note: This is just a technical challenge. It is insanely impractical to deploy CentOS 8 on sub-GHz CPUs in any way.

It works fairly well on Pentium M 1.8GHz. KDE 5 is somewhat sluggish, but works.

Some figures deployed on Pentium 120MHz:

(Above figures improved by about 1.5 by eliminating emulated endbr32 opcode embedded everywhere)


$Keywords: CentOS 8, 32bit, i686, i586, installation media respin, anaconda, Pentium, CMOV emulation, SSE emulation, SSE2 emulation, QtWebEngine without SSE2 $

See the Latest release info for latest information.

$Id$ (2020/11)

Does this work on my machine?

Your CPU has capability of
cmov mmx sse sse2 works? Example CPUs
yesplain Pentium
mmx noPentium MMX, K6-2, WinChip
cmovmmx noPentium II, Cyrix MII
cmovmmxsse noPentium III
cmovmmxssesse2 yesPentium M, Pentium 4

The reason for all this fuss is that not all binaries are re-compiled. CentOS provided userland .i686 RPMs includes CMOV and SSE2 instructions which needs emulation, but MMX/SSE only processors cannot emulate them properly.

Things you need

Preparing i686 compilation environment

Compiling for .i686.rpm packages should be done in setarch i686'ed chroot environment. Many packages just doesn't cross-compile properly in x86_64 environment.

On the work machine, disable SELinux, or make it at least permissive. This is a lorax(1) requirement. You need a reboot for this.

Bind-mounting a chroot filesystems

Suppose you are prepring a chroot environment in /chroot/i686/ of the work machine. After sudo mkdir -p /chroot/i686/, add the following entries in /etc/fstab : 

# /chroot/i686
/proc		/chroot/i686/proc	none auto,bind 0 0
/dev		/chroot/i686/dev	none auto,bind 0 0
/dev/pts	/chroot/i686/dev/pts	none auto,bind 0 0
tmpfs		/chroot/i686/dev/shm	tmpfs auto,nosuid,nodev 0 0
sysfs		/chroot/i686/sys	sysfs auto,rw,nosuid,nodev,noexec 0 0
/sys/fs/cgroup	/chroot/i686/sys/fs/cgroup none auto,bind 0 0
## to let systemd build test see a /sys/fs/cgroup/systemd fs_type
/sys/fs/cgroup/systemd	/chroot/i686/sys/fs/cgroup/systemd none auto,bind 0 0
## to let systemd build test see /run/systemd/session/<#>
/run		/chroot/i686/run	none auto,bind 0 0
## to let sed build test see selinux
selinuxfs	/chroot/i686/sys/fs/selinux	selinuxfs auto 0 0

/home		/chroot/i686/home	none auto,bind 0 0

Do: 

base$ sudo mkdir -p /chroot/i686/{proc,dev,sys,run} /chroot/i686/home

Then, manually mount the entries above, or just reboot to mount them all.

Bootstrapping 32bit development environment in the chroot

The goal is to prepare enough 32bit packages into /chroot/i686/ to run rpmbuild(8).

Drop-in ix86.repo as /chroot/i686/etc/yum.repos.d/ix86.repo . 

base$ sudo mkdir -p /chroot/i686/etc/yum.repos.d/
base$ sudo cp ix86.repo /chroot/i686/etc/yum.repos.d/ix86.repo

Install rpm-build package and mass dependencies using dnf --installroot . Use precompiled packages provided by this site for bootstrapping. 

base$ sudo setarch i686 --32bit \
    dnf --installroot=/chroot/i686 --releasever=8 \
    --disablerepo=\* \
    --enablerepo=ix86repo-BaseOS --enablerepo=ix86repo-AppStream \
    --verbose install rpm-build sudo

Pull in other packages needed for rpmbuild: 

base$ sudo setarch i386 --32bit \
    dnf --installroot=/chroot/i686 --releasever=8 \
    --disablerepo=\* \
    --enablerepo=ix86repo-BaseOS --enablerepo=ix86repo-AppStream \
    --verbose install dnf gcc make

chrooting to the i686 environment

With bash(1) installed in the chroot, it should be able to chroot inside. Check uname -a to see if it is an i686 environment. 

base$ uname -a
Linux centos8.localdomain 4.18.0-80.11.2.el8_0.x86_64 #1 SMP Tue Sep 24 11:32:19 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

base$ sudo cp /etc/passwd /chroot/i686/etc/passwd
base$ sudo setarch i686 --32bit chroot /chroot/i686 sudo -u $LOGNAME -s
i686$ uname -a
Linux centos8.localdomain 4.18.0-80.11.2.el8_0.x86_64 #1 SMP Tue Sep 24 11:32:19 UTC 2019 i686 i686 i386 GNU/Linux
This is tedious, so you want to add the following to ~/.bashrc in the work machine's base x86_64 system: 
if [ `uname -m` != i686 ]; then
	alias i686env="sudo setarch i686 --32bit chroot /chroot/i686 sudo -u $LOGNAME -s"
fi

Compiling a Package

Now, you would compile the individual packages.

To compile a package,

mkdir a dedicated directory

Since you are mass-building various packages, directly building under ~/rpmbuild/ is not recommended. The example below uses ~/c8builds/coreutils-c8/ for coreutils package build. 

base$ sudo setarch i686 --32bit chroot /chroot/i686
i686# sudo -u user -s
user@i686$ cd
user@i686$ mkdir c8builds
user@i686$ cd c8builds
user@i686$ mkdir coreutils-c8
user@i686$ cd coreutils-c8
user@i686$ pwd
/home/user/c8builds/coreutils-c8

Prepare the directory for rpmbuild

You would like to run a following mkrpmdir shellscript inside the directory: 

#!/bin/sh

### mkdir
for d in RPMS/i686 RPMS/i486 RPMS/i586 RPMS/noarch RPMS/i386 BUILD SPECS SRPMS SOURCES; do mkdir -p $d; done

### ./rpmbin
cat > ./rpmbin << 'EOF'
#!/bin/sh

D=${0%/*}
test x"$D" = x"." && D="`pwd`"

#--target=i586
exec ${0##*/} -D "_topdir $D" "$@"

EOF
chmod +x ./rpmbin

### symlink ./rpmbuild, ./rpm -> rpmbin
for i in rpmbuild rpm; do rm -f ./$i; ln -s rpmbin ./$i; done
This script prepares ./rpm and ./rpmbuild to install/compile the package.

Download a source rpm

Source RPMs are available under ix86.repo/Source/, or if not, under http://vault.centos.org/. For example, source RPM for coreutils package is at http://ftp.iij.ad.jp/pub/linux/centos-vault/8.3.2011/BaseOS/Source/SPackages/coreutils-8.30-8.el8.src.rpm . Download it: 

user@i686$ curl -R -O http://ftp.iij.ad.jp/pub/linux/centos-vault/8.3.2011/BaseOS/Source/SPackages/coreutils-8.30-8.el8.src.rpm
You would like to take note what directory the package goes eventually: 
user@i686$ echo BaseOS > reponame

Expand the source rpm in the directory

"Install" the source rpm in the current directory. You do not need a root privilege for source installation and compile. 

user@i686$ ./rpm -ivh coreutils-8.30-8.el8.src.rpm
Note that you use ./rpm . It installs the source RPM in current directory and populates ./SPECS/ and ./SOURCE/ .

Invoke rpmbuild to build the binary RPM

You will invoke ./rpmbuild multiple times until it compiles, so prepare a Makefile like the following: 

PKG=coreutils
DIST=.el8
binary:
	./rpmbuild --target=i586 -v -bb \
	-D 'dist $(DIST)' \
	SPECS/$(PKG).spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done | \
	tee log

src:
	./rpmbuild --target=i586 -v -bs \
	-D 'dist $(DIST)' \
	SPECS/$(PKG).spec
Change the emphasized part.

Invoke rpmbuild to build the binary RPM

You have made a Makefile, so invoke 

user@i686$ make
./rpmbuild --target=i586 -v -bb \
-D 'dist .el8' \
SPECS/coreutils.spec 2>&1 | \
while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$line"; done| \
tee log
2020-11-03 07:17:53 error: Failed build dependencies:
2020-11-03 07:17:53     autoconf is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     automake is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     gettext-devel is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     glibc-langpack-en is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     gmp-devel is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     hostname is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     libacl-devel is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     libattr-devel is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     libcap-devel is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     libselinux-devel is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     libselinux-utils is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     openssl-devel is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     strace is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53     texinfo is needed by coreutils-8.30-7.el8.1.i586
2020-11-03 07:17:53 Building target platforms: i586
2020-11-03 07:17:53 Building for target i586
Log will be logged in ./log file. You will normally have dependencies like above to resolve for building packages. Recursively build and install them, or install from this site: 
i686# dnf --disablerepo=\* --enablerepo=ix86repo-\* install strace
Repeat recursive build, install and make until binary RPMs are built.

Packages modified for 32bit

Kernel

Although the not hardest to compile for 32bit, kernel package has the most patches.

Bootloader (grub2)

Patched to make grub2-pc-modules as .i586, not .noarch .

RPM toolset

These are patched to let .i686 packages install on i586 host. (Usually rpm refuses so)

anaconda

Patched for slower machine deployment.

D-Bus and systemd patched to allow longer timeout

Eliminate hardcoded cmov, sse2

Refer Source directory for other modified packages.

Note: QtWebEngine (Chromium) is about 20 time heavier than KHTML even after eliminating SSE2; Konqueror provided on this page is patched to use KHTML by default. (EPEL Konqueror defaults to QtWebEngine)

Opcode emulation kernel

The kernel above has CMOV, NOPL, FCOMI, FCMOVcc opcode emulation, and some SSE2 emulation enough to run compiler-generated (-mfpmath=sse) SSE2 instructions. This will let .i686 binaries run on i586 CPUs, which lacks cmov and sse2 capability.

Note: opcode emulation is very slow in nature. Recompile for .i586 whenever possible.

Counts of emulated opcodes will be available under /proc/emulated_ops: 

$ cat /proc/emulated_ops
cmov:   3750123
nopl:   0
fcomi:  0
fucomi: 0
fcmov:  0
sse:    32682918
sse2:   1489899

GNOME3 is heavy; use KDE5

On CentOS 8 (RHEL 8), for desktop environment, only GNOME 3 is provided, but on slow machines I strongly recommend KDE. GNOME3 had become too heavyweight for sub-GHz, single-thread processor. KDE 5 is provided via EPEL, but since it provides only x86_64 binaries, every KDE 5 components had to be recompiled for 32bit.

Packages needed to be copied from CentOS repository

Collect .i686 packages listed in centos-packages.txt from your favorite CentOS x86_64 repository or mirror. For downloading, dnf --downloadonly package will download it in the current directory, but timestamp is not preserved. Recommend downloading by wget or curl -R -O .

Note: console-setup.noarch package, is from CentOS 7. This ought to be obsoleted but needed to compile kbd and kbd-misc packages.
Some .i686 packages are not provided in the latest repository (i.e 8.3.2011/); try searching into older releases' directory.

Packages needed to be compiled

Compile packages listed in compile-packages.txt .

These packages needs to be compiled because either

Use Source RPMs in Source directory if available. If there wasn't, use Source RPMs from http://vault.centos.org/ .

Packages needed to be checked out from Git

Following packages may not have corresponding source RPM in vault.centos.org. You must directly check it out from Git.

datefudge libabigail pam_wrapper python-cryptography-vectors python-pretend rubygem-coderay rubygem-kramdown unicode-emoji

Example: 

user@i686$ git clone https://git.centos.org/rpms/datefudge.git datefudge-c8 ## checks out into ./datefudge-c8/ directory

Packages requiring nonstandard rpmbuild Makefile

bind
bind:
	## jsoncpp-devel (EPEL) conflicts with build
	-sudo rpm -ev jsoncpp-devel
	:
	## need PKCS11, else et al will be orphand
	./rpmbuild --target=i586 -v -bb \
	-D 'dist .el8' \
	--with PKCS11 --without SDB \
	--without UNITTEST --without GEOIP2 \
	SPECS/$@.spec 2>&1 | \
	(IFS=""; while read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done ) | \
	tee log

src:
	./rpmbuild --target=i586 -v -bs \
	-D 'dist .el8' \
	--with PKCS11 --without SDB \
	--without UNITTEST --without GEOIP2 \
	SPECS/bind.spec
kernel
all:
	echo 'Use "make kernel-i686" or "make kernel-i586" .'
	false

kernel-i686:
	#--without perf \
	# RHEL8.3 made %define with_perf 0 on %{with_baseonly};
	# needs omitting --with baseonly to build python3-perf package
	#
	./rpmbuild --target=i686 -v -bb \
	--without debug --without debuginfo \
	--without kabidupchk --without kabidwchk --without kabidw_base \
	--without bpftool \
	SPECS/kernel.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done | \
	tee log.i686

kernel-i586:
	# --without perf \
	# --with baseonly \
	#
	./rpmbuild --target=i586 -v -bb \
	--without debug --without debuginfo \
	--without kabidupchk --without kabidwchk --without kabidw_base \
	--without bpftool \
	SPECS/kernel.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done | \
	tee log.i586

src:
	./rpmbuild -v -bs \
	--without debug --without debuginfo \
	--without kabidupchk --without kabidwchk --without kabidw_base \
	--without bpftool \
	SPECS/kernel.spec
patchutils
patchutils:
	## do not let it use /bin/perl; use /usr/bin/perl
	## otherwise dnf cannot derive perl-interpreter package
	env PATH=/usr/bin:/usr/sbin:/bin:/sbin \
	./rpmbuild --target=i586 -v -bb SPECS/$@.spec
python-cffi
python-cffi:
	## demands defining %{__python}
	./rpmbuild --target=i586 -v -bb \
	-D '__python /usr/bin/python3' \
	SPECS/$@.spec
python3
python3:
	## Needs to move out existing /usr/lib/python3.6/site-packages ,
	## otherwise test.test_site.StartupImportTests FAILs
	-sudo mv /usr/lib/python3.6/site-packages /usr/lib/python3.6/site-packages81
	env LANG=C \
	./rpmbuild --target=i586 -v -bb \
	-D 'dist .el8' \
	SPECS/$@.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done| \
	tee log
	sudo mv /usr/lib/python3.6/site-packages81 /usr/lib/python3.6/site-packages
ruby
DIST=.module_el8.1.0+249+93480f15
ruby:
	## move out existing ones else TestOpenURI#test_200 freezes
	-sudo sh -c 'mv /usr/share/rubygems{,81}; mv /usr/share/ruby{,81}; mv /usr/lib/ruby{,81}'
	:
	# USER= needed to be reset to the running user, else euid tests fail
	# (i686env sets these to "root")
	# --without jit needed to circumvent "hardened" gcc 8.3
	# otherwise %check fails with
	# "error: one or more PCH files were found, but they were invalid";
	# needs patched SPECS/ruby.spec
	:
	USER=`whoami` USERNAME=`whoami` LOGNAME=`whoami` \
	./rpmbuild --target=i586 -v -bb \
	-D 'dist $(DIST)' \
	--without jit \
	SPECS/$@.spec </dev/null 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done| \
	tee log
	:
	sudo sh -c 'mv /usr/share/rubygems{81,}; mv /usr/share/ruby{81,}; mv /usr/lib/ruby{81,}'
rust
For rust.i586, bootstrap rust.i686 first, then use make rust-i586 to make rust.i586. You need this to build librsvg2.i586 package, which extensively uses cmov and endbr32 in .i686 library code. 
DIST=.module_el8.3.0+485+0058aead
rust:
	# --with bundled_llvm
	## from rust-1.41, 32bit /bin/ld.bfd barfs with
	## /bin/ld: failed to set dynamic section sizes: Memory exhausted
	## try using /bin/ld.gold
	sudo alternatives --set ld /usr/bin/ld.gold
	./rpmbuild --target=i686 -v -bb \
	-D 'channel stable' \
	-D 'dist $(DIST)' \
	SPECS/$@.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done | \
	tee log
	sudo alternatives --auto ld ## return to ld.bfd

rust-bootstrap:
	# --with bundled_llvm
	sudo alternatives --set ld /usr/bin/ld.gold
	./rpmbuild --target=i686 -v -bb \
	-D 'channel stable' \
	-D 'dist $(DIST)' \
	-D 'bootstrap_arches i686' \
	SPECS/rust.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done | \
	tee log.boot
	sudo alternatives --auto ld ## return to ld.bfd

rust-i586:
	## from rust-1.41, 32bit /bin/ld.bfd barfs with
	## /bin/ld: failed to set dynamic section sizes: Memory exhausted
	## try using /bin/ld.gold
	sudo alternatives --set ld /usr/bin/ld.gold
	./rpmbuild --target=i586 -v -bb \
	-D 'channel stable' \
	-D 'dist $(DIST)' \
	SPECS/rust.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done | \
	tee log.i586
	sudo alternatives --auto ld ## return to ld.bfd

src:
	./rpmbuild --target=noarch -v -bs \
	-D 'channel stable' \
	-D 'dist $(DIST)' \
	-D 'srcrpm 1' \
	SPECS/rust.spec
samba
samba:
	if rpm -q libbsd; then \
		echo '*** STRONGLY RECOMMENDED: uninstall EPEL libbsd package beforehand to prevent dependency'; \
	fi
	./rpmbuild --target=i586 -v -bb \
	-D 'dist .el8' \
	SPECS/$@.spec 2>&1 | \
	(IFS=""; while read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done)| \
	tee log
sssd
sssd:
	# With po4a enabled, it tries to rebuild translated manpages
	# and discards files below 80% translation rate.
	# Squelch such behavior by pretending po4a is not installed,
	# just like koji.
	env ac_cv_prog_PO4A="no" \
	./rpmbuild --target=i586 -v -bb \
	-D 'dist .el8' \
	SPECS/$@.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done| \
	tee log
subversion
PKG=subversion
binary:
	./rpmbuild --target=i586 -v -bb \
	--without kwallet \
	--without python2 --without pyswig --with python3 \
	-D 'dist .module_el8.0.0+45+75bba4f4' \
	SPECS/$(PKG).spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done | \
	tee log
systemd
DIST=.el8_3
systemd:
	# needed for "meson test test-cgroup-util":
	# parent: mount -o bind /sys/fs/cgroup/systemd /chroot/i686/sys/fs/cgroup/systemd
	# -D 'debug_package %{nil}'
	:
	LOGNAME=`whoami` USER=`whoami` USERNAME=`whoami` \
	./rpmbuild --target=i586 -v -bb \
	-D 'dist $(DIST)' \
	SPECS/$@.spec 2>&1 | \
	(IFS=""; while read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done)| \
	tee log
systemtap
systemtap:
	./rpmbuild --target=i586 -v -bb \
	-D 'dist .el8' \
	--without virthost -D 'with_virthost 0' \
	SPECS/$@.spec 2>&1 | \
	while IFS="" read line; do echo `date '+%Y-%m-%d %T'` "$$line"; done| \
	tee log
webkit2gtk3
Needs to fix /lib/openjpeg-2.3/OpenJPEGConfig.cmake: 
get_filename_component(OPENJPEG_INCLUDE_ROOT "${SELF_DIR}/../../include/openjpeg-2.3" ABSOLUTE)
	to
get_filename_component(OPENJPEG_INCLUDE_ROOT "${SELF_DIR}/../../usr/include/openjpeg-2.3" ABSOLUTE)

EPEL packages for KDE 5

For KDE 5 desktop, additional EPEL packages must be compiled.

If you encountered an error similar to the following: 

CMake Error: The source directory "/home/user/c8builds/kf5-kwindowsystem-epel-c8/BUILD/kwindowsystem-5.68.0/i586-redhat-linux-gnu" does not appear to contain CMakeLists.txt.
Add -D '__cmake_in_source_build 1' to the rpmbuild arguments.

Collecting Compiled Packages into a Repository

You need to collect the compiled RPMs and assemble them as a repository, which lorax(1) requires.

Preparing a Repository directory

Prepare another directory for a repository: 

user@i686$ cd
user@i686$ mkdir ix86.repo
user@i686$ cd ix86.repo
user@i686$ mkdir -p BaseOS/Packages AppStream/Packages PowerTools/Packages epel/Packages

Collecting compiled RPMs

Then, collect the compiled binary RPMs into the above directories. You don't have to strictly follow the location of the directory as published in CentOS/RHEL; if you are lazy, just cramming everything into BaseOS/Packages/ may work. 

user@i686$ find ../c8builds/coreutils-c8/RPMS/ -name '*.rpm' | egrep -v -- '-debuginfo-|-debugsource-' | xargs ln -t BaseOS/Packages
You would like to take advantage of "reponame" file made earlier for automatic sieving.

Run createrepo_c

First, browse in the CentOS x86_64 binary repository for comps.xml file. If your favorite mirror is located at http://ftp.iij.ad.jp/pub/linux/centos/8.3.2011/BaseOS/x86_64/os/ , the comps.xml file may be at http://ftp.iij.ad.jp/pub/linux/centos/8.3.2011/BaseOS/x86_64/os/repodata/0ecc0ca5e2097e89d9f66ab3393c8a6659b46243865a003eefdc7da34ed68946-comps-BaseOS.x86_64.xml . Copy it as ./comps-BaseOS.x86_64.xml (for later use).

Then, invoke createrepo_c against the RPM collection directory: 

user@i686$ mkdir -p ./BaseOS/repodata/
user@i686$ cp -p comps-BaseOS.x86_64.xml ./BaseOS/repodata/comps.xml
user@i686$ createrepo_c -v --groupfile repodata/comps.xml ./BaseOS
Do the same thing against ./AppStream/ , ./PowerTools/ and ./epel/ directory.

Add module information

Since the compiled RPMs are not "modularized", there are no module info in the repository and anaconda installation will fail. Install a https://github.com/sgallagher/repo2module program. 

user@i686$ cd
user@i686$ git clone https://github.com/sgallagher/repo2module
user@i686$ cd repo2module
## Follow the instructions in README.md
user@i686$ python3 setup.py install --user
## This will install ~/.local/bin/repo2module
Add module information to the repository: 
user@i686$ cd
user@i686$ cd ix86.repo
user@i686$ ~/.local/bin/repo2module --module-name=BaseOS --module-stream=stable ./BaseOS modules.yaml
## ./modules.yaml will be created. No need to edit for current purpose
user@i686$ modifyrepo_c --mdtype=modules modules.yaml BaseOS/repodata
Do the same thing against ./AppStream/ and ./PowerTools/ .

Check if the repo is queriable

Use dnf --repofrompath option to temporary make the new repository recognized, and list the contents. 

user@i686$ dnf --disablerepo=\* --repofrompath=testrepo-BaseOS,`pwd`/BaseOS clean all
user@i686$ dnf --disablerepo=\* --repofrompath=testrepo-BaseOS,`pwd`/BaseOS list
...
zlib.i586                            1.2.11-13.el8           testrepo-BaseOS
zlib-devel.i586                      1.2.11-13.el8           testrepo-BaseOS
zlib-static.i586                     1.2.11-13.el8           testrepo-BaseOS

Assembling anaconda installer

After you had made the repository, it's the time to use lorax(1) to assemble the anaconda installer.

On the work machine, install lorax from official repository or media. 

base$ sudo yum --disablerepo=\* --enablerepo=c8-media\* install lorax

Create boot image with lorax

Do this on x86_64 work host. Prepare a separate directory for boot image assembling. Begin with a pristine directory to store the boot.iso file: 

base$ cd
base$ mkdir c8lorax
base$ cd c8lorax

base$ rm -fr ./img8

Download add_template.tmpl file. You need this if the target machine is a slow machine.

Then, invoke the lorax with some options. Below assumes that assembled repository resides in ../c8builds/ix86.repo/ . 

base$ sudo lorax -p CentOS -v 8 -r 8 \
	-s `pwd`/../c8builds/ix86.repo/BaseOS \
	-s `pwd`/../c8builds/ix86.repo/AppStream \
	-s `pwd`/../c8builds/ix86.repo/PowerTools \
	--add-template `pwd`/add_template.tmpl \
	--buildarch=i686 --nomacboot \
	`pwd`/img8
This takes a long time! (about 40 minutes on 3GHz 2-thread machine) Logfiles are automatically created as ./lorax.log, ./pylorax.log, ./program.log .

As a result, ./img8/ will be populated with installer boot files, notably ./img8/images/boot.iso .

Scratch directories, owned by root, may be lying around in /var/tmp/lorax/ directory. You can safely delete them.

The recreated files are owned by root, so you would like to 

base$ sudo chown -R $LOGNAME ./img8
base$      chmod -R +w       ./img8

The generated ./img8/images/boot.iso of about 651MB should boot as a network installer. You would like to try it out on target machine to see if the anaconda installer would work.

Discussion

lorax does not use the native files of the work machine to build the installer; it unpacks files from the *.rpm in the -s path directory. Thus the work host doesn't have to be i686; working on x86_64 should be okay.

Collect packages from the repository to a DVD tree

Copy over your repository contents to ./img8/ . 

base$ sudo chown -R $LOGNAME ./img8
base$ rm -fr ./img8/Packages ./img8/SPackages ./img8/repodata
base$ rm -fr ./img8/{BaseOS,AppStream,PowerTools,epel}
base$ cp -rpl ../ix86.repo/{BaseOS,AppStream,PowerTools,epel} ./img8/

Optionally, if the target machine is slow, append the kernel boot line with inst.xtimeout=600 (wait for 600 seconds for Xorg to start; default 60 secs) 

base$ sed -i -e 's/ quiet$/ inst.xtimeout=600&/' ./img8/isolinux/isolinux.cfg

Regenerate .treeinfo

The ./img8/.treeinfo generated by lorax doesn't include BaseOS/, AppStream/ et al directives. You must recreate it. Download productmd.py file, and invoke as: 

base$ cp -p ./img8/.treeinfo ./img8/.treeinfo.lorax
base$ python3 ./productmd.py -t ./img8 AppStream BaseOS PowerTools epel > ./img8/treeinfo
base$ mv ./img8/treeinfo ./img8/.treeinfo
This program just regenerates the .treeinfo file in .ini format, so it could be done without the program if you want. Using python3 was only because it had an .ini file parse/output library.

Respin the media

Now you can respin the DVD media image. 

base$ mkisofs -o ./DVD1.iso \
	-b isolinux/isolinux.bin -c isolinux/boot.cat \
	-no-emul-boot -boot-load-size 4 -boot-info-table \
	-R -T -J \
	-v \
	-V "CentOS-8-i386" \
	-m upgrade.img -m boot.iso \
	./img8
base$ implantisomd5 ./DVD1.iso

The volume label (-V "CentOS-8-i386") should match with kernel option in ./img8/isolinux/isolinux.cfg, inst.stage2=hd:LABEL=CentOS-8-i386 .
Excluding boot.iso (~651MB) is just for lowering size of the final DVD.iso .

This will create a DVD1.iso image aound 4.3GB. Burn the DVD1.iso and try it out on your target machine.

Troubleshooting

Unknown error occured in anaconda installer

Symptom: During anaconda startup, anaconda bails out: 

08:46:07,998 CRT exception: Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/pyanaconda/ui/gui/__init__.py", line 988, in _on_continue_clicked
    nextAction = self._instantiateAction(self._actions[1])

  File "/usr/lib/python3.6/site-packages/pyanaconda/ui/gui/__init__.py", line 773, in _instantiateAction
    obj = actionClass(self.data, self.storage, self.payload, self.instclass)

  File "/usr/lib/python3.6/site-packages/pyanaconda/ui/gui/spokes/network.py", line 1692, in __init__
    self.network_control_box.current_hostname = self._network_module.proxy.GetCurrentHostname()

  File "/usr/lib/python3.6/site-packages/pydbus/proxy_method.py", line 102, in __call__
    raise error

pyanaconda.modules.common.errors.DBusError: g-dbus-error-quark: GDBus.Error:org.freedesktop.DBus.Error.TimedOut: Failed to activate service 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) (20)

Cause: DBus timeout is by default 25 seconds in bus/config-parser.c: 

parser->limits.activation_timeout = 25000; /* 25 seconds */
which could be too short on slow machine.

Workaround: The timeout could be adjusted by adding /etc/dbus-1/system-local.conf XML file, which is included from /usr/share/dbus-1/system.conf .

Add /etc/dbus-1/system-local.conf : 

<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<!-- dbus: enlengthen default hardcoded
   <limit name="service_start_timeout">25000</limit>
   (dbus package, bus/config-parser.c:parser->limits.activation_timeout = 25000; /* 25 seconds */)
   to longer value, using system drop-in included from
   /usr/share/dbus-1/system.conf
-->
<busconfig>
  <limit name="service_start_timeout">240045</limit>
</busconfig>
Doing this is already added in add_template.tmpl .

Unknown error occured in anaconda installer

Symptom: During install, anaconda bails out: 

09:04:05,946 INF threading: Thread Failed: AnaInstallThread (-1313998016)
09:04:05,946 DBG exception: running handleException
09:04:05,954 CRT exception: Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/pyanaconda/threading.py", line 286, in run
    threading.Thread.run(self)

  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)

  File "/usr/lib/python3.6/site-packages/pyanaconda/installation.py", line 388, in doInstall
    installation_queue.start()

  File "/usr/lib/python3.6/site-packages/pyanaconda/installation_tasks.py", line 304, in start
    item.start()

  File "/usr/lib/python3.6/site-packages/pyanaconda/installation_tasks.py", line 304, in start
    item.start()

  File "/usr/lib/python3.6/site-packages/pyanaconda/installation_tasks.py", line 472, in start
    self.run_task()

  File "/usr/lib/python3.6/site-packages/pyanaconda/installation_tasks.py", line 438, in run_task
    self._task(*self._task_args, **self._task_kwargs)

  File "/usr/lib/python3.6/site-packages/pyanaconda/payload/dnfpayload.py", line 1089, in install
    raise payload.PayloadError(msg)

pyanaconda.payload.PayloadError: Payload error - DNF installation has ended up abruptly: No available modular metadata for modular packageTraceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pyanaconda/payload/dnfpayload.py", line 283, in do_transaction
    base.do_transaction(display=display)
  File "/usr/lib/python3.6/site-packages/dnf/base.py", line 834, in do_transaction
    self.transaction._populate_rpm_ts(self._ts)
  File "/usr/lib/python3.6/site-packages/dnf/db/group.py", line 334, in _populate_rpm_ts
    raise dnf.exceptions.Error(_("No available modular metadata for modular package"))
dnf.exceptions.Error: No available modular metadata for modular package

Resolution: Create fake modular metadata on the repository.

tmux hogs CPU during anaconda installer

Symptom: tmux server seems to use all CPU power available (30% ~ 50%) during the anaconda install.

Cause: The following /usr/share/anaconda/tmux.conf line during anaconda setup, read by tmux, repeatedly invokes /bin/sh and eats up CPU. (included as data/tmux.conf in anaconda source code) 

set-option status-right '#[fg=blue]#(echo -n "Switch tab: Alt+Tab | Help: F1 ")'

Resolution: Since this is a static string, there's no need to invoke a shell via #(); just replacing it to 

set-option status-right '#[fg=blue]Switch tab: Alt+Tab | Help: F1 '
will make tmux CPU usage sane. The anaconda source and binary package provided by this site already has this patch applied.

rngd eats up the CPU

Symptom: On old machine without a TPM chip, rngd eats up CPU to nearly 100% for several minutes on boot.

Workaround: Use haveged package provided from EPEL. It is much lighter on CPU on slow machines. 

# dnf --disablerepo=\* --enablerepo=ix86repo\* install haveged
# systemctl disable rngd
# systemctl enable haveged

This still launches rngd in initrd, so boot could still be slow. To get rid of it (and you aren't using crypto filesystems), uninstall rng-tools package and re-dracut: 

# rpm -ev rng-tools
# dracut -vv -f

To replace rngd with haveged in anaconda installer, place the haveged.i586.rpm in PowerTools/Packages/ of the repository, then invoke lorax with --installpkgs=haveged . add_template.tmpl already have quirks to replace rngd with haveged for anaconda.

Many services fail on boot

Symptom: On slow machine, some systemd service fail to boot due to timeout.

Resolution: Enlenghten TimeoutStartSec settings using drop-in settings on such services. add_template.html includes such quirks.

/etc/dbus-1/system-local.conf
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN"
 "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<!-- dbus: enlengthen default hardcoded
   <limit name="service_start_timeout">25000</limit>
   (dbus package, bus/config-parser.c:parser->limits.activation_timeout = 25000; /* 25 seconds */)
   to longer value, using system drop-in included from
   /usr/share/dbus-1/system.conf
-->
<busconfig>
  <limit name="service_start_timeout">240045</limit>
</busconfig>
/etc/systemd/system/systemd-hwdb-update.service.d/timeout.conf
[Service]
TimeoutStartSec=5min
/etc/systemd/system/polkit.service.d/timeout.conf
[Service]
TimeoutStartSec=5min
/etc/systemd/system/NetworkManager-wait-online.service.d/timeout.conf
[Service]
TimeoutStartSec=5min
Additionally, edit /usr/lib/systemd/system/NetworkManager-wait-online.service to use --timeout=300 instead of --timeout=30 .
/etc/systemd/system/systemd-hostnamed.service.d/timeout.conf
[Service]
TimeoutStartSec=5min
/etc/systemd/system/udisks2.service.d/timeout.conf
[Service]
TimeoutStartSec=5min
/etc/systemd/system/systemd-logind.service.d/timeout.conf
[Service]
RestartSec=2s
TimeoutStartSec=5min

Wi-Fi interface not recognized after install

Symptom: The machine has a wifi network adaptor. It was usable in installer and rescue disc, but after HDD install, it doesn't work.
It is listed under ip link as 

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp2s1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:11:25:44:08:c3 brd ff:ff:ff:ff:ff:ff
3: wlp2s2: <NO-CARRIER,BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT group default qlen 1000
    link/ether 00:0e:9b:99:bb:19 brd ff:ff:ff:ff:ff:ff
but NetworkManager seems to ignore the interface; scanning for Access Points (nmcli dev wifi list) nor authenticated connection (nmcli --ask dev wifi connect) doesn't work.

When NetworkManager -d is invoked manually, it says 

<info>  [1604490696.4915] manager: (wlp2s2): 'wifi' plugin not available; creating generic device
<info>  [1604490696.4927] manager: (wlp2s2): new Generic device (/org/freedesktop/NetworkManager/Devices/3)

Cause: NetworkManager-wifi plugin is not installed. It may not be installed on upgrading minimum install to something else.

Resolution: Install NetworkManager-wifi package. After install, NetworkManager -d output should be 

...
<info>  [1604490850.9231] Loaded device plugin: NMWifiFactory (/usr/lib/NetworkManager/1.22.8-5.el8_2/libnm-device-plugin-wifi.so)
...
<info>  [1604490850.9926] device (wlp2s2): driver supports Access Point (AP) mode
<info>  [1604490850.9938] manager: (wlp2s2): new 802.11 Wi-Fi device (/org/freedesktop/NetworkManager/Devices/3)
<info>  [1604490850.9964] device (wlp2s2): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
<info>  [1604490851.1861] device (wlp2s2): set-hw-addr: set MAC address to D2:BF:13:B9:78:4F (scanning)
<info>  [1604490851.2284] supplicant: wpa_supplicant running
<info>  [1604490851.2286] device (wlp2s2): supplicant interface state: init -> starting

Starting KDE invokes the screenlocker and cannot reach desktop on slow machine

Symptom: Started KDE by startx on a slow machine. Since it's slow, it invokes the default screenlocker during startup, and the machine is slow enough to be screenlocker reinvoked after disarming the screenlocker. I can't access the desktop. systemsettings5 can't be invoked to change the setting.

Workaround: Manually disable the spash screen and screenlocker by creating following files: 

; ~/.config/ksplashrc
[KSplash]
Engine=none
Theme=None

; ~/.config/kscreenlockerrc
[$Version]
update_info=kscreenlocker.upd:0.1-autolock

[Daemon]
Autolock=false

[Greeter]
Theme=org.kde.breeze.desktop

cannot change input method

Symptom:Trying to switch ibus input method. Click-selecting the keyboard icon, or using [Super]-Space, does show the alternate input method (after terrible delay), but keyboard icon doesn't change and input method doesn't change.

Cause: Default timeout of 5 seconds may be too short for slow machines

Resolution: set a longer timeout on ibus-daemon. 

% pkill ibus-daemon
% IBUS_TIMEOUT=30000 /usr/bin/ibus-daemon -d -r --xim --timeout=30000

There are two configurable timeouts for ibus-daemon; ibus timeout (IBUS_TIMEOUT), which defaults to 6 seconds, and gdbus timeout (--timeout) which defaults to 5 seconds. These may be too short for slow machines. Above example sets them to 30 seconds. (--timeout seems to be enough)

Making change permanent

Edit /etc/X11/xinit/xinput.d/ibus.conf as: 

XIM=ibus
XIM_PROGRAM="/usr/bin/ibus-daemon"
ICON="ibus"
XIM_ARGS="-r --xim --timeout=30000"
PREFERENCE_PROGRAM=/usr/bin/ibus-setup
...

ZD1211 USB WiFi does not work

Symptom: ZyDAS zd1211 based USB wifi adapter (such as Buffalo WLI-UC-AG) is not recognized. On insertion, /var/log/messages or journalctl -xe says: 

Nov 06 21:10:17 scorpio kernel: usb 1-4: Direct firmware load for zd1211/zd1211b_ub failed with error -2
Nov 06 21:10:17 scorpio kernel: usb 1-4: Could not load firmware file zd1211/zd1211b_ub. Error number -2
Nov 06 21:10:17 scorpio kernel: zd1211rw 1-4:1.0: couldn't load firmware. Error number -2
Nov 06 21:10:17 scorpio NetworkManager[675]: <info>  [1604664617.4144] device (wlp0s29f7u4): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Nov 06 21:10:17 scorpio NetworkManager[675]: <warn>  [1604664617.4266] device (wlp0s29f7u4): firmware may be missing.
Nov 06 21:10:17 scorpio NetworkManager[675]: <info>  [1604664617.4274] device (wlp0s29f7u4): set-hw-addr: set MAC address to 0E:E1:3E:98:C5:2F (scanning)

Resolution: Install zd1211-firmware package. The one supplied with CentOS 6 will work nicely.

Note: you cannot use 802.11a band; the driver lacks implementation.

To make the driver work in installation/rescue disc, you must add zd1211-firmware-1.4-4.el6.noarch.rpm in BaseOS/Packages/, createrepo_c, and pass --installpkgs=zd1211-firmware to lorax.

BCM43xx based WiFi does not work

Symptom: On insertion of Broadcom BCM43xx based (Cardbus) WiFi adapter, journalctl or /var/log/messages says 

Nov 07 10:48:47 scorpio kernel: pcmcia_socket pcmcia_socket0: pccard: CardBus card inserted into slot 0
Nov 07 10:48:47 scorpio kernel: pci 0000:03:00.0: [14e4:4320] type 00 class 0x028000
Nov 07 10:48:47 scorpio kernel: pci 0000:03:00.0: reg 0x10: [mem 0x00000000-0x00001fff]
Nov 07 10:48:47 scorpio kernel: pci 0000:03:00.0: BAR 0: assigned [mem 0xd4000000-0xd4001fff]
Nov 07 10:48:47 scorpio kernel: pci 0000:03:00.0: cache line size of 32 is not supported
Nov 07 10:48:48 scorpio kernel: b43-pci-bridge 0000:03:00.0: enabling device (0000 -> 0002)
Nov 07 10:48:48 scorpio kernel: ssb: Found chip with id 0x4306, rev 0x03 and package 0x00
Nov 07 10:48:48 scorpio kernel: ssb: Core 0 found: ChipCommon (cc 0x800, rev 0x04, vendor 0x4243)
Nov 07 10:48:48 scorpio kernel: ssb: Core 1 found: IEEE 802.11 (cc 0x812, rev 0x05, vendor 0x4243)
Nov 07 10:48:48 scorpio kernel: ssb: Core 2 found: PCMCIA (cc 0x80D, rev 0x02, vendor 0x4243)
Nov 07 10:48:48 scorpio kernel: ssb: Core 3 found: V90 (cc 0x807, rev 0x02, vendor 0x4243)
Nov 07 10:48:48 scorpio kernel: ssb: Core 4 found: PCI (cc 0x804, rev 0x09, vendor 0x4243)
Nov 07 10:48:48 scorpio kernel: ssb: Sonics Silicon Backplane found on PCI device 0000:03:00.0
Nov 07 10:48:48 scorpio kernel: b43-phy1: Broadcom 4306 WLAN found (core revision 5)
Nov 07 10:48:48 scorpio kernel: b43-phy1: Found PHY: Analog 2, Type 2 (G), Revision 2
Nov 07 10:48:48 scorpio kernel: b43-phy1: Found Radio: Manuf 0x17F, ID 0x2050, Revision 2, Version 0
Nov 07 10:48:48 scorpio kernel: Broadcom 43xx driver loaded [ Features: PNLS ]
Nov 07 10:48:48 scorpio kernel: b43 ssb0:0: Direct firmware load for b43/ucode5.fw failed with error -2
Nov 07 10:48:48 scorpio kernel: b43 ssb0:0: Direct firmware load for b43/ucode5.fw failed with error -2
Nov 07 10:48:48 scorpio kernel: b43 ssb0:0: Direct firmware load for b43-open/ucode5.fw failed with error -2
Nov 07 10:48:48 scorpio kernel: b43 ssb0:0: Direct firmware load for b43-open/ucode5.fw failed with error -2
Nov 07 10:48:48 scorpio kernel: b43-phy1 ERROR: Firmware file "b43/ucode5.fw" not found
Nov 07 10:48:48 scorpio kernel: b43-phy1 ERROR: Firmware file "b43-open/ucode5.fw" not found
Nov 07 10:48:48 scorpio kernel: b43-phy1 ERROR: You must go to http://wireless.kernel.org/en/users/Drivers/b43#devicefirmware and download the correct firmware for this driver version. Please carefully read all instructions on this website.

Resolution: Install a firmware package. b43-openfwwf-5.2-10.el6.noarch.rpm supplied with CentOS 6 works. If it doesn't, you need to use b43-fwcutter by yourself to extract a firmware.

The URL mentioned in the kernel message is old. Currently it is https://wireless.wiki.kernel.org/en/users/drivers/b43 , although it doesn't have much information now.

To make the driver work in installation/rescue disc, you must add b43-openfwwf-5.2-10.el6.noarch.rpm in BaseOS/Packages/, createrepo_c, and pass --installpkgs=b43-openfwwf to lorax.

3Com 3c509 does not work

Symptom: Using 3com EtherlinkIII 3c509 network adaptor. When manually modprobe 3c509 is done, the card is detected, but does not link up, even if I said ethtool -s eth0 port tp. Kernel messages says: 

eth0: 3c5x9 found at 0x310, BNC port, address 00:a0:24:28:c2:80, IRQ 3.
3c509.c:1.20 04Feb2008 becker@scyld.com
------------[ cut here ]------------
WARNING: CPU: 0 PID: 737 at net/sched/sch_generic.c:356 dev_watchdog+0x1f3/0x200
NETDEV WATCHDOG: eth0 (3c509): transmit queue 0 timed out
Modules linked in: ip_set nfnetlink ebtable_nat ...
CPU: 0 PID: 737 Comm: nmcli Not tainted 3.10.0-862.el7.v37.i586 #1
Hardware name: System Manufacturer System Name/ALADDIN5, BIOS 0626 07/15/95
Call Trace:
 <SOFTIRQ>  [<c09c5ba0>] dump_stack+0x16/0x18
 [<c0452daa>] __warn+0xea/0x110
 [<c08e8f53>] ? dev_watchdog+0x1f3/0x200
 [<c0452e16>] warn_slowpath_fmt+0x46/0x60
 [<c08e8f53>] dev_watchdog+0x1f3/0x200
 [<c04612f2>] call_timer_fn+0x32/0x100
 [<c07bc312>] ? scsi_decide_disposition+0x1d2/0x210
 [<c047cc69>] ? put_cred_rcu+0x89/0xb0
 [<c08e8d60>] ? dev_deactivate_queue.constprop.30+0x50/0x50
 [<c0463429>] run_timer_softirq+0x199/0x270
 [<c08e8d60>] ? dev_deactivate_queue.constprop.30+0x50/0x50
 [<c045b933>] __do_softirq+0xc3/0x1f0
 [<c045b870>] ? tasklet_action+0x110/0x110
 [<c041a565>] call_on_stack+0x45/0x50
 <EOI>  [<c045bbb5>] ? irq_exit+0x95/0xa0
 [<c0419e95>] ? do_IRQ+0x45/0xd0
 [<c09d0810>] ? __do_page_fault+0x490/0x490
 [<c09d4ef3>] ? common_interrupt+0x33/0x40
---[ end trace b49bc66903ceb9e0 ]---
eth0: transmit timed out, Tx_status 00 status 2000 Tx FIFO room 1332.
eth0: transmit timed out, Tx_status 00 status 2000 Tx FIFO room 1484.
eth0: transmit timed out, Tx_status 00 status 2000 Tx FIFO room 1528.
eth0: transmit timed out, Tx_status 00 status 2000 Tx FIFO room 1408.
eth0: transmit timed out, Tx_status 00 status 2000 Tx FIFO room 1328.
eth0: transmit timed out, Tx_status 00 status 2000 Tx FIFO room 1260.

Cause: Media autodetection is disabled on the card. When the card is set to use particular link port, ethtool -s eth0 port tp doesn't have any effect.

Resolution: Use 3C5X9CFG.EXE, included in 3Com EtherDisk, to set the card to use TP (RJ-45) medium.

Cannot login on console under heavy load

Symptom: Under heavy load, login on vt console is rejected by 

Login timed out after 60 seconds

Resolution: Add login(1) LOGIN_TIMEOUT in /etc/login.defs: 

## default 60secs is short under heavy load
LOGIN_TIMEOUT	300

You may have to boot into single user mode to make this change, since multi-user mode keeps CPU load > 5 for several minutes.

Cannot login as root on single-user mode

Symptom: Passed single as kernel command line, to drop into single-user mode. Root password is denied as: 

Give root password for maintenance
(or type Control-D to continue): ********
sulogin: crypt failed: Invalid argument
Login incorrect

Give root password for maintenance
(or type Control-D to continue): _

Cause: SELinux is in enforcing mode, and /etc/shadow is not labelled correctly as 

----------. root root system_u:object_r:shadow_t:s0    /etc/shadow

Resolution: Boot with enforcing=0 single kernel option. You should be able to login as root. At shell prompt, do restorecon -v /etc/shadow to reset the SELinux context.

Text screen resolution reset when using Matrox video card

Symptom: Using a Matrox video card (such as G200). Text console resolution is reset to 80x30 regardess of vga=... nomodeset kernel option.

Cause: matroxfb_base module is read in by udev, which does not honor kernel options such as video=, vga=, nomodeset.

Resolution: Prepare /etc/modprobe.d/matrox.conf, or append to /etc/modprobe.d/blacklist.conf the following to prevent matroxfb from loading:

## /etc/modprobe.d/matrox.conf
# disable loading matroxfb by udev.
# udev scans /sys/**/modalias and loads driver by modprobe -b;
# "video=matroxfb:disabled" isn't honored by udev

blacklist matroxfb_base
If you want to explicitly set the resolution, you may say
#blacklist matroxfb_base
# 1024x768 
options matroxfb_base vesa=0x118 left=168
# 800x600
#options matroxfb_base vesa=0x115 upper=22 left=112 right=48 
# 1360x768 widescreen
#options matroxfb_base xres=1360 yres=768 fv=60 hslen=112 vslen=7 upper=18 lower=3 left=256 right=64
See Documentation/fb/matroxfb.txt (provided by kernel-doc package) for options.

Note: VESA graphics console (vga=...) is very slow on older CPU, occasionaly causing "tsc unstable" log. Using matroxfb is recommended.

Widescreen not enabled on Matrox MGA G200 in X

Symptom: Using Matrox G200 graphics card and widescreen monitor. Full 1360x768 resolution was available in CentOS 6 X11 desktop, but only 1024x768 is provided in CentOS 8, which seems to use VESA driver.

Cause: Xorg server of RHEL8 is patched to not find legacy xorg-x11-drv-* drivers which isn't shipped with RHEL8. MGA (xorg-x11-drv-mga) is among such driver.

Resolution: Replace the xorg-x11-server-Xorg with the one provided on this page, and additionally install the driver below. This will enable full resolution available for desktop Matrox graphics cards.

To enable full resolution in anaconda installer, add --installpkgs=xorg-x11-drv-mga argument to lorax.

Fails to load SoundBlaster AWE32 driver

Symptom: Using SoundBlaster AWE32 soundcard. Since this card isn't detected automatically, I tried to load the driver by insmod, but fails as following: 

# modprobe snd_sbawe
sbawe 01:01.00: activated
sbawe 01:01.02: activated
sbawe: fatal error - EMU-8000 synthesizer not detected at 0x620
sbawe 01:01.00: disabled
sbawe 01:01.02: disabled

Workaround: Add snd_sbawe seq_ports=0 option. This skips detection of EMU-8000 wavetable sequencer, so PCM part of driver loads. 

# modprobe snd_sbawe seq_ports=0
sbawe 01:01.00: activated
sbawe 01:01.02: activated
Note: snd_sbawe.ko module is not compiled in for stock CentOS kernel.

NE2000 (RTL8019AS) does not work with SoundBlaster AWE32

Symptom: RTL8019AS and SoundBlaster AWE32 both in PnP mode. Manually modprobe ne; modprobe snd-sbawe will recognize the driver, but ne driver does not work. It seems to stop working after modprobe snd-sbawe.

Resolution: Try modprobe in opposite order, i.e modprobe snd-sbawe; modprobe ne. snd-sbawe driver seems to be less forgiving in PnP resource allocation, so let it allocate resources first, then load ne which is more flexible in resource allocation.

Unable to mount root fs on unknown-block(0,0)

Symptom: On boot, kernel panics at early stage: 

...
[    7.030357] md: ... autorun DONE.
[    7.035143] List of all partitions:
[    7.038643] No filesystem could mount root, tried:
[    7.038645]
[    7.045754] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    7.054114] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.18.0-193.6.3.el8_2.centos.plus.v3.i586 #1
[    7.054114] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008  12/07/2018
[    7.054114] Call Trace:
[    7.054114]  dump_stack+0x58/0x7e
[    7.054114]  panic+0x94/0x230
[    7.054114]  mount_block_root+0x242/0x25b
[    7.054114]  mount_root+0x76/0x7b
...

Cause: The grub menu is lacking initrd ($root)/initramfs-4.18.0-193.6.3.el8_2.centos.plus.v3.i586.img line. You are very likely have interrupted the kernel installation by Ctrl+C, which leaves /boot/grub2/grub.cfg in transient state, missing initrd line.

Resolution: The easiest way is to reinstall the kernel-core package, after booting in previous kernel: 

# dnf reinstall ./kernel-core-4.18.0-193.6.3.el8_2.centos.plus.v3.i586.rpm

pulseaudio hogs CPU in desktop environment

Symptom: Used startx to start a desktop environment. Watching the top(1) output. pulseaudio is hogging the cpu and desktop initialization crawls to nearly halt.

Workaround: Disable autospawn of pulseaudio. Edit /etc/pulse/client.conf, and add 

autospawn = no
This will prevent pulseaudio from automatically starting. After the desktop had initialized, manually invoke pulseaudio by pulseaudio --start .

Since pulseaudio is designed for platform where CPU power >> soundcard processing power, you are not likely to have satisfactory performance when using ISA soundcards. PCI soundcard is recommended.

I want a progress meter for grub2 loader

Symptom: On a slow machine, grub just sits there during vmlinuz and initrd load. Isn't there a progress meter like grub-legacy had?

Resolution: Add below somewhere in /boot/grub2/grub.cfg . It shows percentage text of things loading in 1.5 seconds interval. 

insmod progress

How do I eject a PCMCIA/CardBus card safely

Resolution: Install pcmciautils, and issue pccardctl eject 0 .

kabe.sra-tohoku.co.jp