Add VNC ZRLE decoding

Implements RFC 6143 SS7.7.6: a ZRLE rectangle is a 4-byte compressed
length followed by that many zlib-compressed bytes, decompressing to
64x64 tiles each using one of five subencodings (Raw, Solid, packed
palette, Plain RLE, Palette RLE). The zlib stream persists for the
whole connection rather than being reset per-rectangle or per-update,
so VncSessionBackend now owns a lazily-initialized, persistent
z_stream torn down only in resetProtocolState() on a fresh
connect/reconnect.

Since the entire rectangle's compressed data decompresses into memory
in one shot, tile parsing is a plain synchronous loop rather than
needing its own RfbState values -- only the compressed-length and
compressed-data reads are actual protocol states. Tile decoding (the
five subencodings, including the continuation-byte run-length
encoding shared by two of them) lives in vnc_pixel_codecs.h/.cpp
alongside the Hextile decoder, unit-tested with 6 new tests covering
each subencoding plus a persistence test that splits one continuous
deflate stream across two separate FramebufferUpdate messages -- it
only decodes correctly if the connection's inflate stream is retained
between them.

Adds a top-level find_package(ZLIB REQUIRED) + ZLIB::ZLIB link
(previously only pulled in transitively via vendored FreeRDP's own
smartcard-emulation feature, which happened to have it enabled but
shouldn't be relied on for that).

Live-verified against the TightVNC test server that nothing regressed
(connect, cursor, clipboard); that server consistently sends Raw for
actual framebuffer content regardless of announced encodings, so
Hextile/ZRLE's live decode path isn't independently confirmed against
a real server -- the unit tests are the primary evidence here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-15 21:02:26 -06:00
co-authored by Claude Sonnet 5
parent e49fa0cf26
commit bb022edcf2
8 changed files with 772 additions and 19 deletions
+32 -10
View File
@@ -121,18 +121,25 @@ Delivered:
keyboard (Qt key -> X11 keysym mapping) and mouse/wheel input forwarding
- `VncDisplayWidget` mirroring `RdpDisplayWidget`'s scale-to-fit rendering
and input-forwarding shape
- 19 unit tests (`tests/test_vnc_session_backend.cpp`): pure-function
- 36 unit tests (`tests/test_vnc_session_backend.cpp`): pure-function
coverage (DES key prep verified against an independently documented test
vector, keysym mapping, socket-error mapping) plus state-machine
coverage against a scripted in-process fake RFB server (all three
protocol-version handshake shapes, auth success/failure, unsupported
security types, pixel-accurate Raw decoding) -- caught and fixed a real
re-entrancy bug (`abort()` synchronously re-firing `disconnected()`
mid-`failConnection()`, silently overwriting a specific error with a
generic one)
security types, pixel-accurate Raw/Hextile/ZRLE decoding including a
ZRLE zlib-stream-persistence test across two separate
`FramebufferUpdate` messages, cursor/clipboard round trips) -- caught
and fixed a real re-entrancy bug (`abort()` synchronously re-firing
`disconnected()` mid-`failConnection()`, silently overwriting a specific
error with a generic one)
- Verified live against a real, independently implemented VNC server
(TightVNC on Windows): connect with VNC Authentication, correct
framebuffer dimensions and pixel data, clean disconnect, reconnect
framebuffer dimensions and pixel data, clean disconnect, reconnect,
live cursor-shape and clipboard-send checks. That particular server
consistently chose Raw for actual framebuffer content regardless of
which compression encodings were announced, so Hextile/ZRLE's
real-world decode path isn't independently confirmed live -- the unit
tests are the primary correctness evidence for those two
- Per-tab VNC-only display mode toggle (tab-bar right-click ->
`Display Mode`): `Scale to Fit` (default, matches RDP's behavior) or
`Actual Size (Scrollbars)` -- renders the remote framebuffer at its
@@ -142,20 +149,35 @@ Delivered:
it degenerates to an exact 1:1 mapping once the widget's own bounds are
fixed to the remote's size, so no separate rendering path was needed.
Persisted across sessions like the terminal theme preference.
- Robustness fix: an unrecognized `FramebufferUpdate` rectangle encoding
used to abort the connection generically; `kAnnouncedEncodings` is now
the single source of truth for what `SetEncodings` announces and what
the rectangle-dispatch `switch` can decode, with a regression test
pinning that every announced encoding has a working case
- Clipboard sync (`ServerCutText`/`ClientCutText`, Latin-1 only -- no
Unicode extension) in both directions
- Remote cursor shape sync via RFB's Cursor pseudo-encoding, mirroring
`RdpDisplayWidget`'s cursor handling in `VncDisplayWidget`
- Hextile and ZRLE compression encodings, in addition to Raw + CopyRect --
meaningfully reduces bandwidth over slower links versus Raw alone. Pure
tile/pixel decode logic lives in `src/vnc_pixel_codecs.h/.cpp`, kept
separate from the wire-sequencing state machine so it's unit-testable
without a socket. ZRLE decoding links `ZLIB::ZLIB` (found via a fresh
top-level `find_package(ZLIB REQUIRED)`, independent of whether
vendored FreeRDP's own internal zlib usage stays enabled)
Known gaps (explicit scope decisions, not oversights -- see issue #3 for
follow-up tracking):
- Apple's Screen Sharing authentication (Diffie-Hellman + AES, security
type 30) isn't implemented, so this can't yet reach macOS's built-in VNC
server -- only standard VNC Authentication (type 2) and no-auth (type 1)
- Raw + CopyRect encodings only -- no Hextile/ZRLE/Tight compression, so
bandwidth usage is higher over slow links than a full VNC client
- Tight compression isn't implemented yet (Raw + CopyRect + Hextile + ZRLE
are); Tight typically compresses best of all of these, so bandwidth over
very slow links is still not as good as a full VNC client would achieve
- No dynamic resize (connects at the server's native resolution; the
`Scale to Fit`/`Actual Size` toggle changes how that fixed resolution is
displayed locally, not what resolution is requested from the guest --
VNC has no equivalent of RDP's MS-RDPEDISP for that)
- No remote cursor shape sync (local default cursor only)
- No clipboard sync
## Milestone 7 - Cross-Platform Protocol Hardening