Add VNC support (Milestone 6, issue #3)

Implements RFB (RFC 6143) directly against QTcpSocket. No permissively
licensed VNC client library exists to vendor the way FreeRDP was for RDP:
LibVNCClient is GPLv2, gtk-vnc is LGPL but GTK-tied, and vendoring either
would force a licensing decision on the whole (MIT) project. This is
from-scratch protocol code instead, threaded like SshSessionBackend (a
QObject on its own QThread driven by Qt's own async socket signals)
rather than RdpSessionBackend's manual worker-thread/blocking-loop
pattern, since QTcpSocket is already async.

Scope, matching SessionTab's existing SSH/RDP dispatch pattern
(session_backend_factory.cpp, session_tab.cpp's widget construction and
signal wiring) and VncDisplayWidget mirroring RdpDisplayWidget's
scale-to-fit rendering:
- Protocol handshake: RFB 3.3/3.7/3.8 negotiated explicitly (the
  SecurityResult message only exists in 3.8; pre-3.8 servers signal auth
  failure by closing the socket, which the disconnect handler accounts
  for)
- VNC Authentication (DES challenge-response, via OpenSSL's classic DES
  API) and no-auth security types
- Raw + CopyRect framebuffer decoding into a persistent QImage, requesting
  a fixed 32bpp format whose byte layout matches QImage::Format_RGB32
  directly (same zero-conversion trick RdpSessionBackend uses for
  FreeRDP's GDI buffer)
- Keyboard (Qt key -> X11 keysym, including the Unicode-beyond-Latin-1
  keysym convention) and mouse/wheel input forwarding

Explicit non-goals for this pass (see docs/PROGRESS.md for the full
list): Apple's Screen Sharing auth (so this can't yet reach macOS's
built-in VNC server), compression encodings beyond Raw/CopyRect, dynamic
resize, remote cursor shape sync, clipboard sync.

19 unit tests (tests/test_vnc_session_backend.cpp): pure-function
coverage (DES key prep verified against an independently documented test
vector for password "COW", X11 keysym mapping, socket-error mapping) plus
state-machine coverage against a scripted in-process fake RFB server
covering all three protocol-version handshake shapes, auth success/
failure, unsupported security types, and pixel-accurate Raw decoding.
That harness caught a real re-entrancy bug: QAbstractSocket::abort()
synchronously re-emits disconnected() before returning, so
failConnection() calling it was silently letting a second, generic
disconnected-socket handler overwrite an already-correct, specific error
message.

Also verified live against a real, independently implemented VNC server
(TightVNC on Windows): connect with VNC Authentication, correct
framebuffer dimensions and pixel data, clean disconnect, reconnect.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-15 18:02:49 -06:00
co-authored by Claude Sonnet 5
parent dbcc20d155
commit 6da9dc6ca5
12 changed files with 2097 additions and 21 deletions
+42 -8
View File
@@ -102,16 +102,50 @@ Delivered:
Git:
- Tag: `v0-m5-done`
## Milestone 6 - VNC Fully Working
## Milestone 6 - VNC Working (initial scope)
Status: Deferred (temporarily postponed)
Status: Completed (initial scope; see gaps below)
Planned Scope:
- Replace current unsupported VNC path with complete VNC implementation
- Deliver usable in-app VNC session behavior aligned to SSH/RDP UX
- Implement VNC connect/disconnect/reconnect lifecycle handling
- Extend profile/session connect options needed by VNC
- Standardize event log and error mapping behavior with SSH/RDP
Delivered:
- `VncSessionBackend`: an original RFB (RFC 6143) client implementation
against `QTcpSocket` -- no permissively licensed VNC client library
exists to vendor the way FreeRDP was for RDP (LibVNCClient is GPLv2,
gtk-vnc is LGPL but GTK-tied), so this is from-scratch protocol code,
threaded like `SshSessionBackend` (a `QObject` on its own `QThread`
driven by Qt's own async socket signals) rather than `RdpSessionBackend`'s
manual worker-thread/blocking-loop pattern
- Full connect/disconnect/reconnect lifecycle, RFB protocol-version
negotiation (3.3/3.7/3.8 handshake differences handled explicitly),
VNC Authentication (DES challenge-response, using OpenSSL's classic DES
API) and no-auth security types, Raw + CopyRect framebuffer decoding,
keyboard (Qt key -> X11 keysym mapping) and mouse/wheel input forwarding
- `VncDisplayWidget` mirroring `RdpDisplayWidget`'s scale-to-fit rendering
and input-forwarding shape
- 19 unit tests (`tests/test_vnc_session_backend.cpp`): pure-function
coverage (DES key prep verified against an independently documented test
vector, keysym mapping, socket-error mapping) plus state-machine
coverage against a scripted in-process fake RFB server (all three
protocol-version handshake shapes, auth success/failure, unsupported
security types, pixel-accurate Raw decoding) -- caught and fixed a real
re-entrancy bug (`abort()` synchronously re-firing `disconnected()`
mid-`failConnection()`, silently overwriting a specific error with a
generic one)
- Verified live against a real, independently implemented VNC server
(TightVNC on Windows): connect with VNC Authentication, correct
framebuffer dimensions and pixel data, clean disconnect, reconnect
Known gaps (explicit scope decisions, not oversights -- see issue #3 for
follow-up tracking):
- Apple's Screen Sharing authentication (Diffie-Hellman + AES, security
type 30) isn't implemented, so this can't yet reach macOS's built-in VNC
server -- only standard VNC Authentication (type 2) and no-auth (type 1)
- Raw + CopyRect encodings only -- no Hextile/ZRLE/Tight compression, so
bandwidth usage is higher over slow links than a full VNC client
- No dynamic resize (connects at the server's native resolution, scaled to
fit locally -- the same way `RdpDisplayWidget` already renders
regardless of server resolution, so not a UX regression vs. RDP)
- No remote cursor shape sync (local default cursor only)
- No clipboard sync
## Milestone 7 - Cross-Platform Protocol Hardening