boycott systemd

(link collection)

SystemD – it keeps getting worse

systemd is just wrong

Stop the madness before it is too late!

If you are a desktop linux user, especially if you like Gnome, there’s a big change (risk) that you are using systemd. Fine. No problem. Good for you.

If you however want to be in control of your server, enjoy flexibility and simple-to-grasp concepts, pretty much the essence of unix for the last 40+ years, you have probably come across an init or rc script that you can read and understand, as well as figure out how to replace it with something else. Perhaps when switching from sendmail to postfix.

Computers still get faster and faster, especially in parallel processing power, and operating systems, especially with a graphic desktop environment, get more and more complex. Thus it makes sense to utilise the increased processing power by doing more things  at the same time, thereby reducing start times.

Another aspect of the increased complexity in todays computer systems are that, partly thanks to the open source software movement, these complex systems are built much according to the unix philosophy: modularly. You take generic pieces of software that does something (and doing it well), and stick them together. This way you get much functionality, built on stable components, which is much better than in the closed-source version, where everybody reinvents the wheel, only with less quality.

How can I say that? Mostly because of very simple mathematical proof:

  • software takes time to write
  • time exists only 24h each day
  • the human brain can only focus on 1 thing at a time
  • bugs can be found in software
  • software is written by humans with brains

Because there are bugs in software, the less software (smaller codebase) there is, the fewer number of bugs it can contain. Smaller, less complex software pieces are more stable than big monoliths. Making changes in a big monolith is more difficult than changing the internals of a small tool/module with a clearly defined interface. The small module can be replaced/rewritten and exchanged with something that implements the same interface, without affecting the functionality of the complex system using it.

What is systemd?

A very bad idea in many ways, that looks appealing in some other ways.

Why is it bad?

Because it replaces /sbin/init, (PID EINS! as the author titles the pages on his nullpointer blog). PID 1, /sbin/init is the most important program in a unix system. It is responsible only for reaping zombies, being the parent process of daemons, and initially starting the system (hence the name “init”). This is the only special process in a unix system, and it is in fact so special that if this process dies, the entire system (the kernel) dies.

What do we know about bugs and code size/complexity? How would we want our most important process? Small and bug-free? Big and bloated? I’d like to say it’s your pick, choose what you like, but with systemd it’s not that simple, because it infects the major linux distributions, gaining momentum and requiring everybody that writes system-oriented open system software to adapt to systemd. After a while it will be too much trouble to maintain compatibility with traditional/portable solutions that have functioned very well for 40+ years now. Things that work on Solaris, BSD, Linux, OS-X that change to systemd, will be Linux-only, because systemd is linux-only, and will never be ported to other kernels.

Remember when Gnome was a desktop environment that you could run on Solaris and BSD? Well, no more. Gnome will have dependencies on systemd, meaning that because systemd always will be linux-only, so will Gnome.I’m late to the party, screaming about this now, many years too late, but it is IMPORTANT. Someone is wrong on the internet. Many before me have been upset about systemd and the many ways in which systemd is bad. There are lists detailing the top 5 systemd troubles, other good summaries on why systemd is bad for you. Some funnier than others, but I very much recommend reading all the linked pages from this post. Most of them are much more insightful and debating than what I can show in this short blog post.


X11 connection rejected because of wrong authentication – X11 forwarding suddenly fails

After ages of flawless X11 forwarding over SSH, today I started getting authentication errors and couldn’t even get a remote xterm to display locally over my ssh tunnel.


I tried ssh -Y, ssh -X and changes in sshd_conf on the remote server and ssh_conf locally, even though I knew that nothing had changed except a few patches to unrelated software on the local machine. Of course that didn’t help.

I ran xauth on the remote server, no indication of any errors.

It turned out that the remote /home filesystem was out of space, and this prevented the ssh X11 forwarding from working properly. I write this as a note-to-self, as it could happen again…

gentoo gnunet build fails with MHD_post_process linker error

gnunet ebuild (zugaina layman overlay) fails with linker errors about MHD_destroy_post_processor and MHD_post_process ?

Add to /etc/portage/package.use:

net-libs/libmicrohttpd  messages

emerge libmicrohttpd again, and then emerge gnunet.


(at least for me)

RHEL6 apache httpd virtual host the proper way

My recipie for name based virtual hosts in separate directories on RHEL:

We place all the virtual hosts under a new directory tree /var/www/vhosts:

# yum install httpd
# mkdir /var/www/vhosts
# semanage fcontext -a -t httpd_sys_content_t /var/www/vhosts
# restorecon -Rv /var/www/vhosts
# mkdir -p /var/www/vhosts/{site1,site2,site3}/{logs,htdocs}
# chown -R apache:apache /var/www/vhosts

I recommend using the FQDN of each site instead of the words “site1”, “site2”, in these examples.

Create the file /etc/httpd/conf.d/vhosts.conf with appropriate content such as:

NameVirtualHost *:80

<VirtualHost *:80>
  ServerName site1
  DocumentRoot /var/www/vhosts/site1/htdocs
  CustomLog "/var/www/vhosts/site1/logs/access.log" common
  ErrorLog "/var/www/vhosts/site1/logs/error.log"

  <Directory "/var/www/vhosts/site1/htdocs">
     Options None
     AllowOverride All
     Order Deny,Allow
     Allow from

<VirtualHost *:80>
  ServerName site2
  DocumentRoot /var/www/vhosts/site2/htdocs
  CustomLog "/var/www/vhosts/site2/logs/access.log" common
  ErrorLog "/var/www/vhosts/site2/logs/error.log"

  <Directory "/var/www/vhosts/site2/htdocs">
     Options None
     AllowOverride All
     Order Deny,Allow
     Allow from

and so on

(Dont forget to set the Directory permissions properly. Above is just an example!)

Then activate the goodness:

# apachectl restart

Why is this method good?

1. Creating the vhosts.conf in conf.d doesn’t modify any vendor-supplied files, which means that we won’t lose them if we reinstall the package.

2. Keeping each virtual host and its logs under its own directory tree makes maintenance a breeze and permissions can be separated to give developers access to specific vhosts only.

officially best way to get up to date LAMP on RHEL6

Q: How do I update php, mysql, and apache on RHEL6 without breaking stuff?

A: Use the great packages from IUS!

1. set up the IUS repo

$ wget
$ wget
$ sudo rpm -Uvh ius-release*.rpm epel-release*.rpm

2. make sure you have an up to date ca-certificates bundle installed.

3. See what php packages are available: yum list | grep -w ius | grep ^php

4. The “downside” (a minor inconvenience) of the greatness of the IUS is that the packages they build provides the same things as the original outdated redhat packages, but don’t obsolete them. This is intentional, and what makes me think it is the best way to obtain a current LAMP on RHEL or CentOS. What this boils down to is that the IUS packages have different names but cannot be installed at the same time as the RedHat/CentOS packages.
This means that we must uninstall the original packages (if they are installed) before we can install the more recent IUS packages.

IUS provides a neat yum plugin called “replace”, that can be used to do this en masse for a whole bunch of packages based on a certain name. If you have the stock packages “php”, “php-devel”, “php-common” and “php-cli” installed, you can “upgrade” them to the IUS php54 equivalents with a pretty oneliner like “yum replace php –replace-with php54“! (If you want to use the plugin, first install it with: “sudo yum install yum-plugin-replace“).

5. install the IUS packages the usual way if not using the replace plugin.

In case of RHEL6, postfix (terribly outdated 2.6.6) requires mysql-libs, so you cannot install mysql55 straight away. What I did was two-steps:

# yum erase postfix
# yum install postfix php54 mysql55-server

This means that I uninstalled postfix which was dependent on mysql-libs, and then reinstalled it at the same time as php54 and mysql55. Then it uses mysql55-libs instead.

 Package          Arch      Version               Repository               Size
 mysql55          x86_64    5.5.31-1.ius.el6      ius                     9.1 M
 mysql55-server   x86_64    5.5.31-1.ius.el6      ius                     9.6 M
 php54            x86_64    5.4.16-1.ius.el6      ius                     2.7 M
 postfix          x86_64    2:2.6.6-2.2.el6_1     rhel-x86_64-server-6    2.0 M
Installing for dependencies:
 apr              x86_64    1.3.9-5.el6_2         rhel-x86_64-server-6    123 k
 apr-util         x86_64    1.3.9-3.el6_0.1       rhel-x86_64-server-6     87 k
 apr-util-ldap    x86_64    1.3.9-3.el6_0.1       rhel-x86_64-server-6     15 k
 httpd            x86_64    2.2.15-28.el6_4       rhel-x86_64-server-6    821 k
 httpd-tools      x86_64    2.2.15-28.el6_4       rhel-x86_64-server-6     73 k
 mailcap          noarch    2.1.31-2.el6          rhel-x86_64-server-6     27 k
 mysql55-libs     x86_64    5.5.31-1.ius.el6      ius                     783 k
 mysqlclient16    x86_64    5.1.61-1.ius.el6      ius                     3.8 M
 perl-DBD-MySQL   x86_64    4.013-3.el6           rhel-x86_64-server-6    134 k
 perl-DBI         x86_64    1.609-4.el6           rhel-x86_64-server-6    707 k
 php54-cli        x86_64    5.4.16-1.ius.el6      ius                     2.6 M
 php54-common     x86_64    5.4.16-1.ius.el6      ius                     894 k

Transaction Summary
Install      15 Package(s)

That’s all, folks!

Error: Cannot retrieve repository metadata (repomd.xml) for repository: epel. Please verify its path and try again

Error: Cannot retrieve repository metadata (repomd.xml) for repository: epel. Please verify its path and try again

I tried installing EPEL on a fresh install of RHEL6, and after adding the repo, yum fails with the above error. I have RHEL6.1 (Santiago) and get the above error.

This happens because the RHEL/CentOS installation doesn’t trust the HTTPS certificate used by, that is issued by “GeoTrust SSL CA“.

In my case the package ca-certificates was not installed and the /etc/pki/tls/certs/ folder didn’t contain any ca-bundle.crt or !

Solution: yum install ca-certificates

(I had to temporarily rpm –erase epel-release first, to get yum working again)

I once got the same error message eventhout ca-certificates was installed and up to date. Then it was a firewall blocking https (port 443) traffic.

I worked around that by changing from https to http in /etc/yum.repos.d/epel.repo

RHEL6 package name for libdb is db4

Close to impossible to understand, but I just spent quite some time to figure out the package name for the Berkeley DB, libdb on RedHat (RHEL6).

Silly me. I should have known that the package is called “db4” and nothing else. After figuring that out, tacking on a “-devel” to get the headers package was piece of cake.

authorized_keys SELinux pubkey authentication on RHEL / CentOS

So, you have correct permissions on your home directory and all the way up to /, with no other-writable directories in the path, as well as correct permissions on the .ssh directory in $HOME, and it still doesn’t work? You probably have SELinux, and need to put the newly created files in the correct security context. Do it with restorecon like this:

chmod 700 ~/.ssh
cd ~/.ssh
chmod 600 ~/.ssh/*
chmod 644 ~/.ssh/authorized_keys
chmod 644 ~/.ssh/known_hosts
chmod 644 ~/.ssh/config
restorecon -R -v ~/.ssh


uuencode package name

Sometimes you have a tiny file you wish to include in a block of plain text, perhaps an email. When I was young(er), -in the era of UUCP and modems, before the world wide web and HTML were invented, when RFC-821 was still new, -there were no MIME attachments to email.

If you wanted to send a file by mail, you had to encode it in a way that could be included in plain text without breaking. That meant 7-bit ASCII only, max 72 chars on each line, and a lot of other limitations.

Bandwidth and storage were limited, so uuencode was invented to “efficiently” encode 3 bytes of binary data into 4 printable characters. Pretty clever.

I recently had a need for uuencode, and it was not installed on my CentOS/RedHat system by default. The package containing uuencode is called “sharutils”. The name comes from the “shar” utility to encode binaries into a shell script, shell archive (shar file).

yum install sharutils” – and voila, I have uuencode and uudecode available.