2

How to configure and secure a VNC server on Linux with TigerVNC? (on a screen less server or a classic machine)

Since the VNC Server (TigerVNC) configuration is the same on most Linux distributions and only the installation method differ, this question is targeting: OpenSUSE, Fedora, CentOS, RHEL, Debian, Mageia, Void Linux, Arch Linux, Manjaro and FreeBSD (in order to be useful for more people)

intika
  • 14,406

1 Answers1

10

Installing a VNC Server

On Linux (on a classic machine or a screen less server) there are multiple (opensource) possibility for a VNC server such as TightVNC, TigerVNC and TurboVNC (this is a non exhaustive list, this guide will be using the native version of TigerVNC):

  • TigerVNC Server: using native or Java code and actively maintained.
  • TurboVNC Server: only using Java, actively maintained.
  • TighVNC Server: as of 2020 current Linux's version is v1.3.10 from 2009.

We first need a desktop (E.g. XFCE or KDE):

# OpenSUSE (XFCE)
zypper in -t pattern xfce

OpenSUSE (KDE)


zypper install -t pattern kde kde_plasma


Fedora/OpenSUSE (XFCE)


dnf groupinstall -y "Xfce Desktop"


Fedora/OpenSUSE (KDE)


dnf -y group install "KDE Plasma Workspaces"


CentOS/RHEL (Gnome)


dnf -y group install "Server with GUI"


CentOS/RHEL (XFCE)


dnf --enablerepo=epel group -y install "Xfce" "base-x"


CentOS/RHEL (KDE)


dnf --enablerepo=epel group -y install "KDE Plasma Workspaces"


CentOS v8 (KDE)


dnf --enablerepo=epel,PowerTools 
dnf -y group install "KDE Plasma Workspaces" "base-x"


Debian (XFCE)


apt install task-xfce-desktop


Debian (KDE)


apt install task-kde-desktop


Mageia (XFCE)


dnf install task-xfce


Mageia (KDE)


dnf install task-plasma5


FreeBSD (XFCE)


pkg install xfce


FreeBSD (KDE)


pkg install x11/kde5


Void Linux (XFCE)


xbps-install -S xfce4


Void Linux (KDE)


xbps-install -S kde5


and optionally, kde5-baseapps


Arch Linux (XFCE)


pacman -S xfce4 xfce4-goodies


Arch Linux (KDE)


pacman -S plasma-desktop


or plasma for the full desktop


pacman -S plasma


Manjaro (XFCE)


pacman -S xfce4-gtk3 xfce4-goodies xfce4-terminal 

network-manager-applet xfce4-notifyd-gtk3 

xfce4-whiskermenu-plugin-gtk3 tumbler engrampa


Manjaro (KDE)


pacman -S plasma kio-extras


optional kde-applications

Install the TigerVNC X server:

# The package name may change depending on the used distro

CentOS


yum install tigervnc-server


Mageia/Fedora/CentOS/RHEL


dnf install tigervnc-server


ALT Linux


apt install tigervnc-server


openSUSE DNF


dnf install xorg-x11-Xvnc


openSUSE


zypper install xorg-x11-Xvnc


Debian


apt install tigervnc-standalone-server tigervnc-common


FreeBSD


pkg install tigervnc-server


Void Linux


xbps-install -S tigervnc


Arch Linux


pacman -S tigervnc


Manjaro


pacman -S tigervnc

Setup and configuration

Setup the password (the hashed version will be saved at ~/.vnc/passwd):

vncpasswd

Edit the config file (startup script, executed when the server start) ~/.vnc/xstartup as follow:

#!/bin/sh

unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS


exec startxfce4


XFCE: startxfce4 or xfce4-session


#exec startxfce4
#exec xfce4-session


KDE: startkde or startplasma-x11


#exec startkde
#exec startplasma-x11


Gnome: startx


#exec startx

Prepare and/or locate the VNC server's config file at:

~/.vnc/config or /etc/vnc/config

Adapt VNC server's settings with the config file... to get the full list of available options we can use Xvnc -help or man Xvnc, here is a config example (also note that on some system like Suse VNC will/may not work if the option geometry is not set):

## Supported server options to pass to vncserver upon invocation can be listed
## in this file. See the following manpages for more: vncserver(1) Xvnc(1).
## Several common ones are shown below. Uncomment and modify to your liking.
##
##
# -------------
#  Xvnc --help
# -------------
##
# Start server : vncserver
# Stop server : vncserver -kill :1
##
###############################################################################

Only allow connection from local hosts


#localhost


VNC tcp port


rfbport=5900


TCP port to listen for HTTP (default=0)


httpport=0


Directory containing files to serve via HTTP (default=)


httpd=


Protocols...


#nolisten=UDP
#listen=TCP


IP settings


useipv4
#useipv6


Interface, listen on the specified network address (default=all)


#interface=127.0.0.1


Use protocol version 3.3 for backwards compatibility


protocol3.3=0


Unix socket access mode (default=384)


#rfbunixmode=384


Unix socket to listen for RFB protocol (default=)


rfbunixpath=


Name of VNC desktop


desktop=MyVNC


Geometry original peppy


geometry=1366x768


Colors


depth=24


Sharing with multiple clients


#alwaysshared
nevershared


Disconnect existing clients if an incoming connection is non-shared.


If combined with NeverShared then new connections will be refused while


there is a client active


disconnectclients


Security, specify which security scheme to use (None, VncAuth, Plain,


TLSNone, TLSVnc, TLSPlain, X509None, X509Vnc, X509Plain) (default=TLSVnc,VncAuth)


securitytypes=TLSVnc,VncAuth


Path to the key of the X509 certificate in PEM format (default=)


#X509Key=


Path to the X509 certificate in PEM format (default=)


#X509Cert=


Set maximum number of clients (power of two)


#maxclients=64


Terminate after s seconds of user inactivity (default=0)


#maxidletime=0


Terminate when a client has been connected for s seconds (default0)


#maxconnectiontime=0


Terminate when no client has been connected for s seconds (default=0)


#maxdisconnectiontime=0


The number of seconds after which an idle VNC connection will be dropped


(zero means no timeout) (default=0)


#idletimeout=0


Zlib compression level (default=-1)


#zlibLevel=-1


The maximum number of updates per second sent to each client (default=60)


#framerate=60


GnuTLS priority string that controls the TLS session’s handshake algorithms.


See the GnuTLS manual for possible values. Default is NORMAL.


#GnuTLSPriority=

VNC server can be started/stopped with the following commands; After starting it you can connect to your server with any VNC client to server-ip:used-port (note that you may probably need to open the used port on the firewall)

# Start the server
vncserver

Stop the server :1


vncserver -kill :1


Forcing multiple server to stop


killall Xvnc

Security

Basic VNC setup does not use encryption for the exchanged stream, here are 4 commons way of securing a VNC connection:

  • Using an X509 certificate: the certificate's location need to be added on the config file, and you need to have the certification on the client as well (setting it's location on the client app). Generating an X509 certificate is explained further bellow.

  • Tunneling VNC through an SSH session with the local sock proxy provided by an SSH session: On the server edit /etc/ssh/sshd_config and enable/add AllowTcpForwarding yes then restart your sshd service systemctl restart sshd.service you can then use vncviewer with the option -via or establish the tunnel connection manually, then use any client to connect:
    ssh serverIP -p 22 -i /home/my/private/key -L 5900:127.0.0.1:5900 -C -N
    Then connect to 127.0.0.1:5900 with any VNC client.

  • Run vncviewer on the server and display it's x window on the client over an SSH X forwarded session: On the server edit /etc/ssh/sshd_config and enable/add X11Forwarding yes then restart your sshd service systemctl restart sshd.service.
    On an SSH session (server's SSH shell) running vncviewer :1 will display the vncviewer window on the client.

  • Tunneling VNC through a VPN connection: this is not covered here.

VNC as a systemd service

VNC server can be used as a Systemd's service with the following config file /etc/systemd/system/vncserver.service, enabling the service with systemctl enable vncserver.service will make it auto-start when the system boot (this does not apply to Void Linux or Systemd less Linux's distributions).

# /etc/systemd/system/vncserver.service

[Unit]
Description=TigerVNC Server
After=syslog.target network.target


[Service]
Type=simple
#Type=forking
User=MY-USER
Group=MY-USER-GROUP


#If ran with root
#WorkingDirectory=/root
#PIDFile=/root/.vnc/%H%i.pid


#If ran with any other user
WorkingDirectory=/home/MY-USER
PIDFile=/home/%u/.vnc/%H%i.pid


#Environment is required when using a custom GnuTLS version
#Environment=LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/usr/lib


ExecStartPre=-/usr/bin/vncserver -kill :%i > /dev/null 2>&1
#ExecStart for forking type version
#ExecStart=/usr/bin/vncserver
ExecStart=/usr/bin/vncserver -fg
ExecStop=/usr/bin/vncserver -kill :%i


[Install]
WantedBy=multi-user.target

Generating an X509 certificate

For a simple secure setup this official wiki can be followed, otherwise this is not a light topic for a further understanding of such encryption see the documentation/links for this section. In short the documentation/links indicate that most secure keys as of 2020 are RSA with a high key size (at least 4096 bits) and EdDSA's Ed25519 implementation which is the EdDSA signature scheme using SHA-512 (SHA-2) and Curve25519 (an elliptic curve offering 128 bits of security). Ed25519 resistance is equivalent to an RSA with a 3072-bits key. (alternatively there is EdDSA's ED448 which is the EdDSA signature scheme using SHAKE256 (SHA-3) and Curve448; it is equivalent to an RSA with a ~12448-bits key, an updated OpenSSL install is required)

Generate an RSA-4096-Bits/Ed25519/ED448 private key for the CA:
For Ed25519 and ED448 an updated version of OpenSSL and GnuTLS are required (check the next section).

mkdir ~/.vnc/ssl

Generate an RSA 4096-bits key


openssl genrsa -out ~/.vnc/ssl/ca.private.rsa.4096.key.pem 4096




genrsa is superseded by genpkey (PKCS#1 vs PKCS#8 format), the following command is similar to the previous one.


openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:4096 -out ~/.vnc/ssl/ca.private.rsa.4096.key.pem


Or an ED25519 key (equivalent to an RSA with a 3072-bits key, updated openssl required)


openssl genpkey -algorithm ED25519 -out ~/.vnc/ssl/ca.private.eddsa.ed25519.key.pem


Or an ED448 key (equivalent to an RSA with a ~12448-bits key, updated openssl required)


openssl genpkey -algorithm ED448 -out ~/.vnc/ssl/ca.private.eddsa.ed448.key.pem

The previous step can be done in a more secure way by protecting the key with a password but TigerVNC does not support that.

Check/View the newly generated key (not required):

openssl pkey -in ~/.vnc/ssl/ca.private.rsa.4096.key.pem -text
# or
openssl pkey -in ~/.vnc/ssl/ca.private.eddsa.ed25519.key.pem -text
# or
openssl pkey -in ~/.vnc/ssl/ca.private.eddsa.ed448.key.pem -text

Generate a signing CA, make it valid for 2 years and adding the server IP (this is required, change 88.44.88.33 with your IP):

# RSA 4096-bits 
openssl req -new -x509 -days 730 -key ~/.vnc/ssl/ca.private.rsa.4096.key.pem -out ~/.vnc/ssl/ca.sign.rsa.4096.key.pem -subj '/CN=88.44.88.33' -addext "subjectAltName=IP:88.44.88.33"

#Or ED25519
openssl req -new -x509 -days 730 -key ~/.vnc/ssl/ca.private.eddsa.ed448.key.pem -out ~/.vnc/ssl/ca.sign.eddsa.ed448.key.pem -subj '/CN=88.44.88.33' -addext "subjectAltName=IP:88.44.88.33"


Or ED448


openssl req -new -x509 -days 730 -key ~/.vnc/ssl/ca.private.eddsa.ed448.key.pem -out ~/.vnc/ssl/ca.sign.eddsa.ed448.key.pem -subj '/CN=88.44.88.33' -addext "subjectAltName=IP:88.44.88.33"

Update the server configuration to use X509:

# Security, specify which security scheme to use (None, VncAuth, Plain,
# TLSNone, TLSVnc, TLSPlain, X509None, X509Vnc, X509Plain) (default=TLSVnc,VncAuth)
securitytypes=X509Vnc

Path to the key of the X509 certificate in PEM format (default=)


X509Key=/home/USER/.vnc/ssl/ca.private.eddsa.ed448.key.pem


Path to the X509 certificate in PEM format (default=)


X509Cert=/home/USER/.vnc/ssl/ca.sign.eddsa.ed448.key.pem

Connecting with TigerVNC Viewer:

  • Copy ca.sign.eddsa.ed448.key.pem to the client
  • On TigerVNC Viewer under Options > Security, on the encryption section leave only TLS with X509 certificates selected and put the path to ca.sign.eddsa.ed448.key.pem on Path to X509 CA certificate (leave CRL section empty)
  • On the authentication section select only Standard VNC

Checking the GnuTLS session's handshake algorithms

TigerVNC is using GnuTLS for the encryption, on the server/client the setting GnuTLSPriority sets the priority string that controls the TLS session's handshake algorithms (TLS1.0/TLS1.1/TLS1.2/TLS1.3/etc.); supported algorithms can be listed with gnutls-cli --list

We can for instance test the TLS v1.2 support with the following:

vncviewer GnuTLSPriority=NORMAL:-VERS-ALL:+VERS-TLS1.2 -log='*:stdout:100'

Here is how we can enforce TLS v1.2/v1.3 on the server config file:

# GnuTLS priority string that controls the TLS session’s handshake algorithms.  
# See the GnuTLS manual for possible values. Default is NORMAL.

Only TLS v1.2


#GnuTLSPriority=NORMAL:-VERS-ALL:+VERS-TLS1.2


Only TLS v1.3


GnuTLSPriority=NORMAL:-VERS-ALL:+VERS-TLS1.3


Verifying if only TLS v1.2/v1.3 policy is working with the following


vncviewer GnuTLSPriority=NORMAL:-VERS-TLS1.2 -log='*:stdout:100' # v1.2


vncviewer GnuTLSPriority=NORMAL:-VERS-TLS1.3 -log='*:stdout:100' # v1.3


This mean use all but v1.2/v1.3 to test if the setting is enforced correctly,


and thus the connection will be refused for handshake algorithm mismatch.


Other example of gnutlspriority values (warning, this is just for the syntax)


#GnuTLSPriority=NORMAL:-VERS-SSL3.0:-VERS-TLS1.0:-VERS-TLS1.1:-VERS-TLS1.3
#NORMAL:+SECURE128:-SHA384:-SHA256:-VERS-SSL3.0:-VERS-TLS1.0:-VERS-TLS1.1
#NORMAL:+VERS-TLS1.2:+VERS-TLS1.3:+AES-128-CBC:+RSA:+SHA1:+COMP-NULL

Updating OpenSSL and GnuTLS

OpenSSL and GnuTLS are two main application/library of most Linux system, updating them manually may introduce security issues as they will not be updated automatically anymore, we can limit that side negative effect by using a custom version just for our purpose (TigerVNC).

A newer OpenSSL version is needed to generate a Ed25519/ED448 key, while GnuTLS is needed for vncviewer(client's viewer) / Xvnc (TigerVNC server) to add support for Ed25519/ED448 algorithms

Check what algorithm is currently supported:

#GnuTLS
gnutls-cli --list | grep EdDSA

#OpenSSL
man -P cat genpkey | grep "Valid built-in algorithm"
openssl list -public-key-algorithms | grep ED


#If EdDSA targeted algorithm is supported there is no need to install from sources

Build, install and use OpenSSL:

wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
tar -xvf openssl-1.1.1g.tar.gz
cd openssl-1.1.1g/
./config no-nextprotoneg no-weak-ssl-ciphers no-ssl3 no-shared -DOPENSSL_NO_HEARTBEATS -fstack-protector-strong enable-tls1_3
make install -j2

After install, OpenSSL can be used


for instance as follow to generate the needed key


/usr/local/bin/openssl genpkey -algorithm ED448 -out ~/.vnc/ssl/ca.private.eddsa.ed448.key.pem

Build, install and use GnuTLS:

wget https://www.gnupg.org/ftp/gcrypt/gnutls/v3.6/gnutls-3.6.14.tar.xz
tar -xvf gnutls-3.6.14.tar.xz
cd gnutls-3.6.14/
./configure --without-tpm --disable-tests --disable-full-test-suite --disable-non-suiteb-curves --disable-ssl2-support
make install -j2

This is required for Xvnc and vncviewer, after install can be used


by setting LD_LIBRARY_PATH before running vncviewer or Xvnc,


like the following example


TigerVNC Server


export LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/usr/lib
vncserver


TigerVNC Viewer


env LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/usr/lib vncviewer


To check the used version we can use ldd for instance


ldd /usr/bin/vncviewer

Experiments notes

While using KDE with TigerVNC server; when stopping/starting the server several times, some X applications remain running, probably related to this systemd issue, to fix that the following script can be used to stop the server (can be used on the systemd service config file as well), and ps aux | sort | grep USER-NAME | grep -v '\[' command can be used to check if something is still running after stopping the server.

#cat /usr/bin/vncserver-stop (this is meant for KDE)

#!/bin/sh


vncserver -kill :0
vncserver -kill :1
vncserver -kill :2
vncserver -kill :3


killall Xvnc -9
killall kwin_x11 -9
killall startplasma-x11 -9
killall plasma_session -9


killall Xvnc -9
killall kwin_x11 -9
killall startplasma-x11 -9
killall plasma_session -9

After several tests with the viewer, the best speed performances were achieved with vncviewer -QualityLevel=4 -CompressLevel=2 -PreferredEncoding=Raw

If the server needs to be exposed to the internet, note that VNC Server can be easily discovered with an nmap scan (nmap -sV -sC TARGET-IP), VNC protocol need to announce itself and thus without creating a custom version of the server and client there is not much than can be done to obfuscate a VNC server in the case of a global internet exposure.

Documentations

intika
  • 14,406