Add Apple Screen Sharing authentication for VNC (security type 30)

Apple's macOS Screen Sharing server doesn't speak standard VNC
Authentication -- it uses a Diffie-Hellman key exchange followed by
AES-128-ECB-encrypted credentials, security type 30. Apple never
published this scheme (it's not part of RFC 6143); this implements
the well-established reverse-engineered wire format: the server sends
a generator, prime, and its own DH public key; the client generates
an ephemeral keypair, derives the shared secret, MD5-hashes it into
an AES key, and sends back its public key plus a 128-byte encrypted
username+password buffer.

The DH/AES math lives in new src/vnc_apple_dh_auth.h/.cpp as a pure,
socket-free helper (mirroring vncAuthResponse()'s shape for standard
VNC Auth), built entirely on modern EVP_PKEY-based OpenSSL 3.0 APIs --
no deprecated low-level DH_* calls, unlike VNC Authentication's
necessary use of classic DES. Reuses Profile::username (already a
shared field) since Apple's scheme needs an actual macOS account name,
unlike password-only VNC Authentication.

Security-type preference when multiple are offered is now None >
AppleDH > VNCAuth, since DH+AES is strictly stronger than static-
challenge DES. Adds a DH round-trip test (generates a real 512-bit
group at test time, computes the response, then independently
re-derives the shared secret as the server would and decrypts the
credentials back out -- proving self-consistency without needing a
hand-computed expected value), a fake-server integration test for the
full RFB 3.8 handshake sequencing, and a preference-order test.

Not yet live-verified against a real macOS Screen Sharing server --
that's next.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

src/vnc_session_backend.h

@@ -24,11 +24,13 @@ struct z_stream_s;
 // C API to wrangle here.
 //
 // Scope (see plan / issue #3 for the full rationale): standard VNC
-// Authentication (security type 2) and no-auth (type 1) only -- not
-// Apple's Screen Sharing scheme (type 30). Raw + CopyRect + Hextile + ZRLE
-// + Tight encodings. No dynamic resize. Clipboard sync (Latin-1 only, per
-// RFB's ServerCutText/ClientCutText) and remote cursor shape sync (the
-// Cursor pseudo-encoding) are supported.
+// Authentication (security type 2), no-auth (type 1), and Apple's Screen
+// Sharing scheme (type 30, Diffie-Hellman + AES -- see
+// vnc_apple_dh_auth.h; not part of RFC 6143, implemented against the
+// well-established reverse-engineered wire format). Raw + CopyRect +
+// Hextile + ZRLE + Tight encodings. No dynamic resize. Clipboard sync
+// (Latin-1 only, per RFB's ServerCutText/ClientCutText) and remote cursor
+// shape sync (the Cursor pseudo-encoding) are supported.
 //
 // Tight decoding gap: the real protocol allows the server to skip zlib
 // compression entirely for very small Basic-mode payloads; this decoder
@@ -109,6 +111,10 @@ private:
         WaitingTightFilterId,
         WaitingTightLengthByte,
         WaitingTightPayload,
+        WaitingAppleAuthGeneratorLength,
+        WaitingAppleAuthGeneratorBytes,
+        WaitingAppleAuthPrimeLength,
+        WaitingAppleAuthPrimeAndServerKey,
         WaitingSetColourMapHeader,
         WaitingSetColourMapData,
         WaitingServerCutTextHeader,
@@ -178,6 +184,16 @@ private:
     quint8 m_tightFilterId;
     int m_tightLengthByteIndex;
 
+    // Apple Screen Sharing authentication state (security type 30, not
+    // part of RFC 6143 -- see vnc_apple_dh_auth.h). m_appleAuthGenerator
+    // must persist across the generator-length/generator-bytes state
+    // hop; m_appleAuthPrimeLength must persist from when it's first read
+    // until the combined prime+server-public-key buffer (2x that length)
+    // has fully arrived, since m_pendingLength gets reused to track that
+    // combined byte count in the meantime.
+    QByteArray m_appleAuthGenerator;
+    quint32 m_appleAuthPrimeLength;
+
     void setState(SessionState state, const QString& message);
     void resetProtocolState();
     void processReceiveBuffer();