Wayland vs X11: How to Choose and Configure Each

Wayland has been the default display protocol on most major distributions for several years now, yet X11 remains available and sometimes necessary. Knowing when to use each, how to verify what you are running, and how to switch cleanly saves real time when something breaks or a workflow demands a specific stack.

How Wayland and X11 Differ

X11 (the X Window System, protocol version 11) has been the Linux display foundation since 1987. It runs through an X server — traditionally Xorg — that acts as a central broker between applications and hardware. Every window, input event, and pixel crosses that server, which enables powerful features like network transparency but also introduces latency and a large attack surface.

Wayland is a display server protocol, not a single program. Each desktop environment ships its own Wayland compositor: GNOME uses Mutter, KDE Plasma uses KWin, wlroots-based compositors (Sway, Hyprland, river) are common on the tiling side. The compositor talks directly to kernel graphics interfaces (DRM/KMS) and input devices via libinput, eliminating the X server entirely. The result is lower latency, better isolation between applications, and first-class HiDPI and mixed-DPI support.

When Wayland Works Well

• GNOME 45+ and KDE Plasma 6+ — both are effectively production-ready on Wayland. Most regressions from earlier years are resolved.
• HiDPI and multi-monitor with different scale factors — Wayland handles per-output fractional scaling natively; X11 requires hacks.
• Screen security — applications cannot snoop on other windows' contents without explicit portal permission (xdg-desktop-portal).
• Variable refresh rate (VRR/FreeSync/G-Sync) — supported by KWin and Mutter on Wayland; X11 support is compositor-dependent and fragile.
• Touch and stylus input — Wayland's input model maps cleanly to touchscreens and drawing tablets via libinput.

When X11 Is Still the Right Choice

• NVIDIA GPUs with older drivers (pre-560 series) — GBM support landed in the 495 driver, but EGLStreams caused pain for years. Driver version 560+ with explicit sync is reliable; anything below should be tested carefully before committing to Wayland.
• Legacy or proprietary applications — some CAD tools, older Electron apps, and VirtualBox's seamless mode still misbehave under XWayland.
• Remote desktop workflows using VNC or classic X forwarding (ssh -X) — these are X11 concepts. Wayland remoting requires PipeWire screen capture + RDP (GNOME's Remote Desktop) or dedicated tools like waypipe.
• Screen recording tools that rely on XSHM — OBS works on Wayland via PipeWire, but requires setup; some simpler tools do not support it at all.
• Accessibility tools — AT-SPI2 and some screen readers have Wayland gaps that are still being closed as of 2024.

Checking Your Current Session Type

Before changing anything, confirm what protocol your current session uses.

echo $XDG_SESSION_TYPE

Output will be wayland, x11, or occasionally mir. You can also check:

loginctl show-session $(loginctl | awk 'NR==2{print $1}') -p Type

That returns something like Type=wayland and is reliable even inside scripts.

Switching Session Types at Login

Session type is selected at the display manager login screen, not inside the running session. The steps differ slightly by display manager.

GDM (GNOME — Ubuntu, Fedora, Debian with GNOME)

Click your username, then click the gear icon in the bottom-right corner before entering your password. Choose between GNOME (Wayland) and GNOME on Xorg. GDM remembers your last choice.

To force GDM to offer only X11 for all users — for example on a shared workstation with a legacy application — edit the GDM custom configuration:

sudo nano /etc/gdm3/custom.conf       # Debian/Ubuntu
sudo nano /etc/gdm/custom.conf        # Fedora/RHEL

Uncomment or add under [daemon]:

WaylandEnable=false

Then restart GDM (this will drop your current session):

sudo systemctl restart gdm

SDDM (KDE Plasma — Fedora KDE, Arch, openSUSE)

SDDM shows a Session drop-down at login. Select Plasma (Wayland) or Plasma (X11). SDDM also remembers the last-used session per user.

To set the default session system-wide, create or edit /etc/sddm.conf.d/10-session.conf:

sudo mkdir -p /etc/sddm.conf.d
sudo tee /etc/sddm.conf.d/10-session.conf <<'EOF'
[Desktop]
Session=plasmawayland
EOF

Replace plasmawayland with plasma to force X11. Valid session names match filenames in /usr/share/wayland-sessions/ and /usr/share/xsessions/.

LightDM (common on Ubuntu flavors, Mint, Xfce spins)

LightDM uses a session selector in its greeter. The gtk-greeter gear icon works similarly to GDM. For scripted control, set the autologin session in /etc/lightdm/lightdm.conf:

sudo nano /etc/lightdm/lightdm.conf

Under [Seat:*] set:

user-session=xfce          # for X11 Xfce
# or
user-session=xfce-wayland  # if an xfce-wayland session file exists

Per-Application X11 Fallback Under Wayland (XWayland)

Most Wayland compositors ship XWayland, a compatibility layer that runs X11 applications inside a nested X server. It starts automatically on demand — you rarely need to configure it. To check whether a specific application is running through XWayland:

xlsclients -display :0 2>/dev/null

Applications listed here are using XWayland. To force a single application back to native Wayland if it is incorrectly detecting X11 — common with Electron apps — set the appropriate flag:

# Electron-based apps (VS Code, Slack, etc.)
code --enable-features=UseOzonePlatform --ozone-platform=wayland

# Or set persistently in the app's flags file, e.g.:
echo '--enable-features=UseOzonePlatform --ozone-platform=wayland' >> ~/.config/code-flags.conf

Conversely, to force an app to use X11/XWayland even in a Wayland session:

WAYLAND_DISPLAY= GDK_BACKEND=x11 your-application

NVIDIA-Specific Wayland Configuration

On systems with NVIDIA GPUs and driver 560+, enable the required kernel module options and DRM modesetting. Confirm your driver version first:

nvidia-smi --query-gpu=driver_version --format=csv,noheader

Enable DRM kernel modesetting (required for GBM/Wayland):

# Debian/Ubuntu — edit GRUB
sudo nano /etc/default/grub
# Add nvidia-drm.modeset=1 to GRUB_CMDLINE_LINUX_DEFAULT, then:
sudo update-grub

# Fedora/RHEL — use grubby
sudo grubby --update-kernel=ALL --args="nvidia-drm.modeset=1"

Reboot after this change. On Wayland with NVIDIA you may also need to set the GBM backend environment variable in /etc/environment:

echo 'GBM_BACKEND=nvidia-drm' | sudo tee -a /etc/environment
echo '__GLX_VENDOR_LIBRARY_NAME=nvidia' | sudo tee -a /etc/environment

These are only needed on some configurations; test without them first on driver 560+.

Verification

After switching and logging back in, verify the session:

echo $XDG_SESSION_TYPE
loginctl show-session $(loginctl | awk 'NR==2{print $1}') -p Type

For Wayland, also confirm the compositor is active:

ps aux | grep -E 'mutter|kwin_wayland|sway|hyprland' | grep -v grep

Troubleshooting

Screen is black or compositor crashes on Wayland

Check the journal immediately after a crash:

journalctl -b -p err -g 'mutter|kwin|wl'

Common culprits: missing firmware, NVIDIA DRM modesetting not enabled, or a Mesa version too old for your GPU. Fall back to X11 temporarily at the login screen while you investigate.

XWayland not starting, X11 apps fail with "cannot open display"

Confirm XWayland is installed:

# Debian/Ubuntu
dpkg -l xwayland

# Fedora/RHEL
rpm -q xorg-x11-server-Xwayland

# Arch
pacman -Q xorg-xwayland

If missing, install it. On Arch: sudo pacman -S xorg-xwayland. On Fedora: sudo dnf install xorg-x11-server-Xwayland.

Wayland session reverts to X11 on Fedora/RHEL with NVIDIA

GNOME's udev rules blacklist Wayland for NVIDIA by default on older Fedora releases. Override it:

sudo ln -sf /dev/null /etc/udev/rules.d/61-gdm.rules
sudo systemctl restart gdm

Only do this with driver 495+ and DRM modesetting confirmed active.