Add VNC support (Milestone 6, issue #3)

Implements RFB (RFC 6143) directly against QTcpSocket. No permissively
licensed VNC client library exists to vendor the way FreeRDP was for RDP:
LibVNCClient is GPLv2, gtk-vnc is LGPL but GTK-tied, and vendoring either
would force a licensing decision on the whole (MIT) project. This is
from-scratch protocol code instead, threaded like SshSessionBackend (a
QObject on its own QThread driven by Qt's own async socket signals)
rather than RdpSessionBackend's manual worker-thread/blocking-loop
pattern, since QTcpSocket is already async.

Scope, matching SessionTab's existing SSH/RDP dispatch pattern
(session_backend_factory.cpp, session_tab.cpp's widget construction and
signal wiring) and VncDisplayWidget mirroring RdpDisplayWidget's
scale-to-fit rendering:
- Protocol handshake: RFB 3.3/3.7/3.8 negotiated explicitly (the
  SecurityResult message only exists in 3.8; pre-3.8 servers signal auth
  failure by closing the socket, which the disconnect handler accounts
  for)
- VNC Authentication (DES challenge-response, via OpenSSL's classic DES
  API) and no-auth security types
- Raw + CopyRect framebuffer decoding into a persistent QImage, requesting
  a fixed 32bpp format whose byte layout matches QImage::Format_RGB32
  directly (same zero-conversion trick RdpSessionBackend uses for
  FreeRDP's GDI buffer)
- Keyboard (Qt key -> X11 keysym, including the Unicode-beyond-Latin-1
  keysym convention) and mouse/wheel input forwarding

Explicit non-goals for this pass (see docs/PROGRESS.md for the full
list): Apple's Screen Sharing auth (so this can't yet reach macOS's
built-in VNC server), compression encodings beyond Raw/CopyRect, dynamic
resize, remote cursor shape sync, clipboard sync.

19 unit tests (tests/test_vnc_session_backend.cpp): pure-function
coverage (DES key prep verified against an independently documented test
vector for password "COW", X11 keysym mapping, socket-error mapping) plus
state-machine coverage against a scripted in-process fake RFB server
covering all three protocol-version handshake shapes, auth success/
failure, unsupported security types, and pixel-accurate Raw decoding.
That harness caught a real re-entrancy bug: QAbstractSocket::abort()
synchronously re-emits disconnected() before returning, so
failConnection() calling it was silently letting a second, generic
disconnected-socket handler overwrite an already-correct, specific error
message.

Also verified live against a real, independently implemented VNC server
(TightVNC on Windows): connect with VNC Authentication, correct
framebuffer dimensions and pixel data, clean disconnect, reconnect.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-15 18:02:49 -06:00
co-authored by Claude Sonnet 5
parent dbcc20d155
commit 6da9dc6ca5
12 changed files with 2097 additions and 21 deletions
+55 -7
View File
@@ -3,6 +3,7 @@
#include "rdp_display_widget.h"
#include "session_backend_factory.h"
#include "terminal_view.h"
#include "vnc_display_widget.h"
#include <KodoTerm/KodoTerm.hpp>
@@ -79,6 +80,7 @@ SessionTab::SessionTab(const Profile& profile,
== 0),
m_sshTerminal(nullptr),
m_rdpDisplay(nullptr),
m_vncDisplay(nullptr),
m_terminalOutput(nullptr),
m_eventLog(nullptr),
m_toggleEventsButton(nullptr),
@@ -238,6 +240,8 @@ SessionTab::SessionTab(const Profile& profile,
[this](const QImage& frame) {
if (m_rdpDisplay != nullptr) {
m_rdpDisplay->setFrame(frame);
} else if (m_vncDisplay != nullptr) {
m_vncDisplay->setFrame(frame);
}
},
Qt::QueuedConnection);
@@ -247,6 +251,8 @@ SessionTab::SessionTab(const Profile& profile,
[this](int width, int height) {
if (m_rdpDisplay != nullptr) {
m_rdpDisplay->setRemoteDesktopSize(width, height);
} else if (m_vncDisplay != nullptr) {
m_vncDisplay->setRemoteDesktopSize(width, height);
}
},
Qt::QueuedConnection);
@@ -405,6 +411,12 @@ void SessionTab::clearTerminal()
if (m_rdpDisplay != nullptr) {
m_rdpDisplay->clearFrame();
m_rdpDisplay->setFocus();
return;
}
if (m_vncDisplay != nullptr) {
m_vncDisplay->clearFrame();
m_vncDisplay->setFocus();
}
}
@@ -717,6 +729,9 @@ void SessionTab::setupUi()
} else if (m_profile.protocol.compare(QStringLiteral("RDP"), Qt::CaseInsensitive) == 0) {
m_rdpDisplay = new RdpDisplayWidget(this);
rootLayout->addWidget(m_rdpDisplay, 1);
} else if (m_profile.protocol.compare(QStringLiteral("VNC"), Qt::CaseInsensitive) == 0) {
m_vncDisplay = new VncDisplayWidget(this);
rootLayout->addWidget(m_vncDisplay, 1);
} else {
m_terminalOutput = new TerminalView(this);
QFont fallbackFont = defaultTerminalFont();
@@ -726,13 +741,7 @@ void SessionTab::setupUi()
m_terminalOutput->setFont(fallbackFont);
m_terminalOutput->setMinimumHeight(260);
m_terminalOutput->setReadOnly(true);
if (m_profile.protocol.compare(QStringLiteral("VNC"), Qt::CaseInsensitive) == 0) {
m_terminalOutput->setPlaceholderText(
QStringLiteral("Embedded VNC session output appears here when the backend is available."));
} else {
m_terminalOutput->setPlaceholderText(
QStringLiteral("Session output appears here."));
}
m_terminalOutput->setPlaceholderText(QStringLiteral("Session output appears here."));
rootLayout->addWidget(m_terminalOutput, 1);
}
@@ -891,6 +900,37 @@ void SessionTab::setupUi()
[this](int x, int y, int deltaX, int deltaY) {
emit requestMouseWheelEvent(x, y, deltaX, deltaY);
});
} else if (m_vncDisplay != nullptr) {
connect(m_vncDisplay,
&VncDisplayWidget::viewportSizeChanged,
this,
[this](int width, int height) { emit requestTerminalSize(width, height); });
connect(m_vncDisplay,
&VncDisplayWidget::displayScaleChanged,
this,
[this](qreal ratio) { emit requestDisplayScale(ratio); });
connect(m_vncDisplay,
&VncDisplayWidget::keyInput,
this,
[this](int key, quint32 nativeScanCode, const QString& text, bool pressed, int modifiers) {
emit requestKeyEvent(key, nativeScanCode, text, pressed, modifiers);
});
connect(m_vncDisplay,
&VncDisplayWidget::mouseMoveInput,
this,
[this](int x, int y) { emit requestMouseMoveEvent(x, y); });
connect(m_vncDisplay,
&VncDisplayWidget::mouseButtonInput,
this,
[this](int x, int y, int button, bool pressed) {
emit requestMouseButtonEvent(x, y, button, pressed);
});
connect(m_vncDisplay,
&VncDisplayWidget::mouseWheelInput,
this,
[this](int x, int y, int deltaX, int deltaY) {
emit requestMouseWheelEvent(x, y, deltaX, deltaY);
});
}
}
@@ -1110,6 +1150,14 @@ void SessionTab::refreshActionButtons()
if (isConnected) {
m_rdpDisplay->setFocus();
}
return;
}
if (m_vncDisplay != nullptr) {
m_vncDisplay->setEnabled(isConnected);
if (isConnected) {
m_vncDisplay->setFocus();
}
}
}