The problem

A web application which requires a Samba-Share (cifs) via openvpn (IPv6) runs on a Debian server (jessie, systemd 215). With the naive configuration, manual starting and stopping works just fine, but rebooting doesn’t work properly.

Naive configuration:

• Debian packages: apache2 cifs-utils openvpn
• cifs mount in /etc/fstab configured
• openvpn configuration in /etc/openvpn/client.conf

When you start everything individually, it works with no problem:

• systemctl start openvpn@client.service
• mount /var/www/data
• systemctl start apache2.service

Stopping also works fine manually:

• systemctl stop apache2.service
• umount /var/www/data
• systemctl stop openvpn@client.service

But with a reboot, you can’t start up…

• When starting, the mount is attempted before the VPN is available and, of course, it fails.
• Apache does run, but the web application has a problem, as soon as it tries to access files in the cifs mount.

… or shut down:

• The VPN was interrupted before the umount was completed.
• The system freezes for 2 minutes until the umount times out. Very annoying.

Approach #1

The solution shouldn’t be too difficult with systemd units.

Since no dependencies can be explicitly formulated in /etc/fstab, a unit file is created for the mount. We start with the automatically generated unit file:

systemctl cat var-www-data.mount > /etc/systemd/system/var-www-data.mount

And we delete the entries SourcePath and Documentation, so that we’re left with just the minimum:

#/etc/systemd/system/var-www-data.mount
[Unit]
Before=remote-fs.target

[Mount]
What=//v6.smb.example.com/WwwDataShare
Where=/var/www/data
Type=cifs
Options=ro,guest,iocharset=utf8

Then the entry from /etc/fstab is deleted, and the unit is activated with:

systemctl enable var-www-data.mount; systemctl daemon-reload

It should now be possible to properly mount and unmount /var/www/data.

Now we can formulate the dependency on openvpn:

#/etc/systemd/system/var-www-data.mount
[Unit]
Before=remote-fs.target
After=openvpn@client.service
Requires=openvpn@client.service

systemctl daemon-reload

Still doesn’t work, unfortunately. After various analyses with journalctl -f, it became apparent that

• when it starts, the openvpn unit thinks that it is “finished”, even though the IPv6 routes haven’t been correctly set, yet.
• when it stops, the umount of /var/www/data thinks it is finished, even though it still has pending network communication.

Solution

First, it needs a script to check the preconditions for starting (ping) and stopping (umount):

/etc/openvpn/checks_updown

#!/bin/bash
LOGGER_RED="systemd-cat -t $0 -p err"
LOGGER_BOLD="systemd-cat -t $0 -p notice"
LOGGER_NORM="systemd-cat -t $0 -p info"

case "$1" in
  ping) 
    while true
    do
      if ping6 -q -c 3 v6.smb.example.com > /dev/null 2>&1 
      then
        echo ping6 ok | $LOGGER_BOLD
        exit 0
      else
        echo "." | $LOGGER_NORM
        sleep 4
      fi
    done
    ;;
  umount) 
    while true
    do
      if mount | grep "//v6.smb.example.com/" >/dev/null 2>&1
      then
        echo ";" | $LOGGER_NORM
        sleep 4
      else
        echo umount ok | $LOGGER_BOLD
        exit 0
      fi
    done
    ;;
  *) 
    echo "invalid call: $1" | tee | $LOGGER_RED
    exit 1
    ;;
esac

For this script, we are building our own unit:

# /etc/systemd/system/vpnbarrier.service
[Unit]
Requires=openvpn@client.service
After=openvpn@client.service

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/etc/openvpn/checks_updown ping
ExecStop=/etc/openvpn/checks_updown umount

[Install]
RequiredBy=var-www-data.mount

The RemainAfterExit=yes causes (along with oneshot) ExecStop to only be executed when this unit is stopped. Without this setting, ExecStop would be executed immediately after ExecStart, while start-up is still in progress.

Expand dependencies in the mount unit:

# /etc/systemd/system/var-www-data.mount
[Unit]
Before=apache2.service
Requires=vpnbarrier.service
After=vpnbarrier.service

[Mount]
...

[Install]
RequiredBy=apache2.service

systemctl enable vpnbarrier.service; systemctl daemon-reload

Only now does everything work as it should:

• systemctl stop openvpn@client.service: First apache2 was stopped, then /var/www/data was suspended, and only after that, the VPN was stopped.
• systemctl start apache2.service: First the VPN was started, then the cifs-mount and finally apache2.
• A reboot takes just about 20 seconds.

Comments

• systemd documentation is really good, for example with digitalocean. However, the emphasis is usually on starting up, and it’s hard to find anything on shutting down.
• A somewhat different discussion has shown that there are also others that think something is missing.
• The most important insight: Always use both Requires and After. That’s the only way to get shut-down under control, too.
• Root cause: Is the problem a generic systemd problem or a bug in the openvpn package? And with cifs umount?
• Is there a somewhat smarter solution to the problem, e.g. , forgoing a separate vpnbarrier.service unit and properly integrating the “ping”- and “umount” check with entries in *.conf.d directories?
• Occasionally, it wasn’t enough to do a systemd daemon-reload, and a reboot had to be done before everything worked as expected.