Files
orbithub/src/vnc_pixel_codecs.h
T
ksmithandClaude Sonnet 5 e49fa0cf26 Add VNC Hextile decoding
Implements RFC 6143 SS7.7.4: rectangles announced as Hextile (type 5)
tile the update into 16x16 blocks, each either raw pixels or a
background fill plus an optional list of foreground/individually-
colored subrects, with background/foreground persisting across tiles
within one rectangle when not re-specified.

The pure byte-decode logic (tile metadata, subrect list) lives in new
src/vnc_pixel_codecs.h/.cpp, kept separate from
VncSessionBackend's wire-sequencing state machine so it's unit-testable
without a socket -- the pattern the plan calls for continuing into the
ZRLE/Tight work still ahead. Adds 5 fake-server tests covering a raw
tile, a background-only solid fill, uncoloured and individually-colored
subrects, and a 4-tile rectangle proving background persistence and
correct tile-cursor wraparound.

Verified against the live TightVNC test server (connect, frame,
cursor, clipboard all still work); that particular server always
chose Raw for the actual framebuffer content during this session, so
Hextile's real-world path isn't independently confirmed live -- the
unit tests are the primary correctness evidence for this phase.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-15 20:49:08 -06:00

66 lines
2.7 KiB
C++

#ifndef ORBITHUB_VNC_PIXEL_CODECS_H
#define ORBITHUB_VNC_PIXEL_CODECS_H
#include <QByteArray>
#include <QRect>
#include <QRgb>
#include <QVector>
// Pure, state-free decode helpers for VNC/RFB pixel encodings, kept out of
// VncSessionBackend so the wire-sequencing (when to read what off the
// socket) and wire-decoding (how to interpret already-buffered bytes) stay
// separate and the latter is unit-testable without a live connection.
namespace VncPixelCodecs {
// Converts one pixel's worth of raw bytes as sent under
// VncSessionBackend's negotiated SetPixelFormat (32bpp, little-endian,
// byte order B,G,R,pad) into a QRgb.
QRgb rgbFromPixelBytes(const uchar* bytes);
// A single filled rectangle, in tile-local coordinates (0,0 = the tile's
// own top-left corner, not the enclosing FramebufferUpdate rectangle's).
struct HextileSubrect {
QRect rect;
QRgb color = 0;
};
// RFC 6143 SS7.7.4 Hextile tile subencoding byte flags.
namespace HextileFlags {
constexpr quint8 kRaw = 0x01;
constexpr quint8 kBackgroundSpecified = 0x02;
constexpr quint8 kForegroundSpecified = 0x04;
constexpr quint8 kAnySubrects = 0x08;
constexpr quint8 kSubrectsColoured = 0x10;
}
// Byte length of a Hextile tile's "fixed" metadata -- the optional
// background/foreground color updates plus the optional subrect count --
// derivable from the subencoding byte alone, before any of those bytes are
// available. Only meaningful when HextileFlags::kRaw is *not* set (a Raw
// tile has no metadata at all, just tileWidth*tileHeight raw pixels).
int hextileFixedMetaByteCount(quint8 subencoding);
// Parses the `hextileFixedMetaByteCount(subencoding)` bytes described
// above. Updates *background/*foreground in place only when the
// corresponding flag is set in `subencoding` -- callers should persist
// their previous values across tiles in the same rectangle and pass them
// in here unchanged when a color isn't re-specified, since RFC 6143 has
// each tile inherit the last-specified colors. Returns the subrect count
// (0 if HextileFlags::kAnySubrects isn't set).
int decodeHextileFixedMeta(quint8 subencoding, const QByteArray& data, QRgb* background,
QRgb* foreground);
// Byte length of `subrectCount` subrects' worth of data, given whether
// they're individually colored (HextileFlags::kSubrectsColoured).
int hextileSubrectByteCount(bool coloured, int subrectCount);
// Parses `subrectCount` subrects (xy + wh bytes, plus a per-subrect color
// when `coloured`) out of `data`, substituting `foreground` in for any
// that aren't individually colored.
QVector<HextileSubrect> decodeHextileSubrects(bool coloured, int subrectCount,
const QByteArray& data, QRgb foreground);
}
#endif