Internal
Public Access
Add Apple Screen Sharing authentication for VNC (security type 30)
Apple's macOS Screen Sharing server doesn't speak standard VNC Authentication -- it uses a Diffie-Hellman key exchange followed by AES-128-ECB-encrypted credentials, security type 30. Apple never published this scheme (it's not part of RFC 6143); this implements the well-established reverse-engineered wire format: the server sends a generator, prime, and its own DH public key; the client generates an ephemeral keypair, derives the shared secret, MD5-hashes it into an AES key, and sends back its public key plus a 128-byte encrypted username+password buffer. The DH/AES math lives in new src/vnc_apple_dh_auth.h/.cpp as a pure, socket-free helper (mirroring vncAuthResponse()'s shape for standard VNC Auth), built entirely on modern EVP_PKEY-based OpenSSL 3.0 APIs -- no deprecated low-level DH_* calls, unlike VNC Authentication's necessary use of classic DES. Reuses Profile::username (already a shared field) since Apple's scheme needs an actual macOS account name, unlike password-only VNC Authentication. Security-type preference when multiple are offered is now None > AppleDH > VNCAuth, since DH+AES is strictly stronger than static- challenge DES. Adds a DH round-trip test (generates a real 512-bit group at test time, computes the response, then independently re-derives the shared secret as the server would and decrypts the credentials back out -- proving self-consistency without needing a hand-computed expected value), a fake-server integration test for the full RFB 3.8 handshake sequencing, and a preference-order test. Not yet live-verified against a real macOS Screen Sharing server -- that's next. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
2fe2022182
commit
1f026dde70
7 files changed
+579
-16
No files matched your search
@@ -0,0 +1,44 @@
|
||||
#ifndef ORBITHUB_VNC_APPLE_DH_AUTH_H
|
||||
#define ORBITHUB_VNC_APPLE_DH_AUTH_H
|
||||
|
||||
#include <QByteArray>
|
||||
#include <QString>
|
||||
|
||||
// Apple's Screen Sharing authentication scheme (RFB security type 30):
|
||||
// Diffie-Hellman key exchange followed by AES-128-ECB-encrypted
|
||||
// credentials. Apple never published this officially -- it's not part of
|
||||
// RFC 6143 -- so this implements the well-established reverse-engineered
|
||||
// wire format used by several independent VNC clients, not a primary
|
||||
// spec. Kept as a pure, state-free helper (no socket access) so it's
|
||||
// unit-testable without a live connection, mirroring
|
||||
// VncSessionBackend::vncAuthResponse()'s shape for standard VNC
|
||||
// Authentication.
|
||||
namespace VncAppleDhAuth {
|
||||
|
||||
struct Response {
|
||||
// Same byte length as the server's prime, big-endian, zero-padded.
|
||||
// Empty on any failure (malformed input, an OpenSSL operation
|
||||
// failing) -- callers should treat an empty clientPublicKey as "could
|
||||
// not compute a response" rather than send a degenerate one.
|
||||
QByteArray clientPublicKey;
|
||||
// Always exactly 128 bytes on success (16 AES blocks): a 64-byte
|
||||
// NUL-padded/truncated username followed by a 64-byte NUL-padded/
|
||||
// truncated password, AES-128-ECB encrypted (no padding, since the
|
||||
// plaintext is already an exact multiple of the block size) with a
|
||||
// key derived as MD5(sharedSecret).
|
||||
QByteArray encryptedCredentials;
|
||||
};
|
||||
|
||||
// Computes the DH keypair, the shared secret, the derived AES key, and
|
||||
// the encrypted credential blob, using modern EVP-based OpenSSL 3.0 APIs
|
||||
// throughout (no deprecated low-level DH_*/legacy calls -- unlike VNC
|
||||
// Authentication's classic DES usage, none of MD5/AES-ECB/the EVP_PKEY DH
|
||||
// APIs are deprecated, so no compatibility pragma is needed here).
|
||||
// Returns a Response with an empty clientPublicKey on any failure.
|
||||
Response computeResponse(const QByteArray& generator, const QByteArray& prime,
|
||||
const QByteArray& serverPublicKey, const QString& username,
|
||||
const QString& password);
|
||||
|
||||
}
|
||||
|
||||
#endif
|
||||
Reference in new issue
Block a user