Adding support for ARD AuthType 33/35/36 by Chapoly1305 · Pull Request #698 · LibVNC/libvncserver

Chapoly1305 · 2026-03-20T21:21:31Z

This pull request introduces support for Apple Remote Desktop (ARD) authentication schemes and adds new cryptographic functionality to the codebase (to support this PR). The changes include updates to the build system for OpenSSL support on macOS, new ARD auth configuration options in the client, and implementations for key cryptographic primitives such as SHA-512, PBKDF2-HMAC-SHA512, and RSA encryption. The client authentication selection logic has been refactored to support multiple ARD schemes and prioritize user preferences.

ARD Authentication Support

Added new ARD authentication schemes (rfbARDAuthDH, rfbARDAuthRSASRP, rfbARDAuthKerberosGSSAPI, rfbARDAuthDirectSRP) and logic to select and handle them in the client, including ardauth.h and integration in rfbclient.c. [1] [2] [3] [4] [5] [6] [7] [8] [9]
Extended the rfbClient struct with optional ARD Kerberos authentication configuration fields and added setter functions for these fields. [1] [2]

Cryptography Enhancements

Added SHA-512 hash, PBKDF2-HMAC-SHA512 key derivation, and RSA PKCS#1 encryption primitives to the cryptographic API, with implementations for OpenSSL and Libgcrypt. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Build System and Platform Support

Updated CMakeLists.txt to build crypto with OpenSSL on macOS, link ARD authentication sources, and add macOS-specific frameworks for the VNC client. [1] [2] [3]
Added new example and test targets for ARD authentication probing in the build system. [1] [2]

These changes collectively enable ARD authentication in the VNC client, provide the necessary cryptographic primitives, and ensure proper build and linking, especially for macOS platforms.

Copilot

Pull request overview

Adds Apple Remote Desktop (ARD) authentication support to libvncclient (auth types 30/33/35/36) and introduces new crypto primitives/build plumbing needed to implement those schemes (notably SHA-512, PBKDF2-HMAC-SHA512, and RSA PKCS#1 encryption).

Changes:

Implement ARD authentication handlers (DH, RSA-SRP, Kerberos GSS-API, Direct SRP) and refactor client auth-type selection to support them.
Extend the public client API with optional ARD Kerberos configuration fields + setters.
Add crypto primitives across OpenSSL/Libgcrypt backends and update CMake to prefer OpenSSL crypto on macOS and link required Apple frameworks; add an ARD probe example.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/ardauthprobe.c	Adds a headless probe client for forcing/testing a specific ARD auth type.
src/libvncclient/vncviewer.c	Initializes/frees new ARD Kerberos configuration fields in rfbClient lifecycle.
src/libvncclient/rfbclient.c	Adds ARD auth support hooks, auth-type selection updates, and new public setters.
src/libvncclient/ardauth.h	Declares the ARD auth handler entry point.
src/libvncclient/ardauth.c	Implements ARD DH / RSA-SRP / Kerberos GSS-API / Direct SRP flows.
src/common/crypto_openssl.c	Adds SHA-512, PBKDF2-HMAC-SHA512, and RSA SPKI DER encryption via OpenSSL.
src/common/crypto_libgcrypt.c	Adds SHA-512 + PBKDF2-HMAC-SHA512 via Libgcrypt (RSA encrypt currently stubbed).
src/common/crypto_included.c	Adds stubs for the new crypto APIs when crypto is not available.
src/common/crypto.h	Exposes new crypto APIs/constants (SHA512, PBKDF2, RSA encrypt).
include/rfb/rfbproto.h	Introduces new ARD auth type constants (replacing the old rfbARD macro).
include/rfb/rfbclient.h	Extends rfbClient struct + declares new ARD Kerberos setter APIs.
CMakeLists.txt	Prefers OpenSSL crypto on macOS, links ardauth.c, adds Apple frameworks, and builds the probe example.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

test/ardauthprobe.c

src/libvncclient/ardauth.c

include/rfb/rfbproto.h

bk138

Thanks for the PR!

General thought: keep it as simple as possible, if we cannot support a functionality for the other platforms, leave it out :-)

bk138 · 2026-03-22T19:46:32Z

include/rfb/rfbclient.h


-        /* flag to indicate wheter updateRect is managed by lib or user */
-        rfbBool isUpdateRectManagedByLib;
+	        /* flag to indicate wheter updateRect is managed by lib or user */


please don't add unrelated whitespace changes

bk138 · 2026-03-22T19:47:44Z

include/rfb/rfbclient.h

 extern rfbBool ConnectToRFBServer(rfbClient* client,const char *hostname, int port);
 extern rfbBool ConnectToRFBRepeater(rfbClient* client,const char *repeaterHost, int repeaterPort, const char *destHost, int destPort);
 extern void SetClientAuthSchemes(rfbClient* client,const uint32_t *authSchemes, int size);
+extern rfbBool rfbClientSetARDAuthRealm(rfbClient *client, const char *realm);


please doc-comment these

bk138 · 2026-03-22T19:49:07Z

include/rfb/rfbproto.h

 #define rfbTLS 18
 #define rfbVeNCrypt 19
 #define rfbSASL 20
-#define rfbARD 30


don't change the API here.

I have thought about this. The issue is that ARD is the name for the full protocol, but authType 30 is just one of several authentication methods. Since this PR aims to add support for more authentication types, I think maybe this adjustment is worth considering.

bk138 · 2026-03-22T19:49:25Z

include/rfb/rfbproto.h

 #define rfbSASL 20
-#define rfbARD 30
+#define rfbARDAuthDH 30
+#define rfbARDAuthRSASRP 33


please add comments here what these are

bk138 · 2026-03-22T19:59:28Z

test/ardauthprobe.c

+ *   -auth 35   ARD Kerberos GSS-API
+ *   -auth 36   ARD Direct SRP
+ *
+ * Kerberos sample:


Kind of important question: are all these auth methods readily configured out of the box on a MacOS host? If not; I see two options:

omit them from this PR for the sake of simplicity

add setup instructions for the server here

"are all these auth methods readily configured out of the box on a MacOS host?"

Yes, I tested on macOS 15.7, and Type 30, 33, 35, and 36 are supported out of the box. No additional configuration is required for the server side. The client can directly authenticate with the server. Type 30, 33, and 36 only require a username and password, while Type 35 requires additional information from the keychain. I’m not sure where this information (for 35) comes from, but it’s there.

bk138 · 2026-03-22T20:00:26Z

test/ardauthprobe.c

@@ -0,0 +1,152 @@
+/**
+ * @example ardauthprobe.c


maybe this should go to examples/client/ rather as it's not really a test that can be automated, what do you think?

bk138 · 2026-03-22T20:04:15Z

src/libvncclient/rfbclient.c

+	switch (authScheme) {
+	case rfbARDAuthDH:
+		return TRUE;
+#if defined(__APPLE__)


It is a must that functionalitly we add works on every supported platform. If it does not, simply omit it :-)

Chapoly1305 · 2026-03-22T20:29:22Z

Thank you so much for taking the time to review the PR. I’ll carefully review the comments and make the necessary changes within the next week(s).

Chapoly1305 added 9 commits March 20, 2026 14:49

include/rfb: split ARD auth scheme constants

ab58000

common: add SHA-512 hash helper

6ebd9d4

common: add PBKDF2-HMAC-SHA512 helper

57b87bb

common: add RSA SPKI encryption helper

687954c

libvncclient: prepare security type selection for ARD auth variants

643bb4c

libvncclient: add ARD principal and realm setters

7f98dc0

libvncclient: add ARD auth handlers

b4a3b61

CMake: add Apple build support for ARD auth

c80fc70

test: add ARD auth probe client

f0ca173

Copilot AI review requested due to automatic review settings March 20, 2026 21:21

Copilot started reviewing on behalf of Chapoly1305 March 20, 2026 21:22 View session

Chapoly1305 mentioned this pull request Mar 20, 2026

[Feature] Apple Screen Sharing High Performance #696

Open

6 tasks

Copilot AI reviewed Mar 20, 2026

View reviewed changes

test/ardauthprobe.c Outdated Show resolved Hide resolved

src/libvncclient/ardauth.c Outdated Show resolved Hide resolved

src/libvncclient/ardauth.c Show resolved Hide resolved

include/rfb/rfbproto.h Show resolved Hide resolved

libvncclient: fix ARD auth CI regressions

ce1308c

Chapoly1305 marked this pull request as draft March 20, 2026 22:18

libvncclient: guard ARD SASL state check

9b16bd1

Chapoly1305 marked this pull request as ready for review March 20, 2026 22:35

libvncclient: avoid ssize_t in ARD peek path

b16271e

Chapoly1305 force-pushed the ard-auth-pr branch from 17f42c5 to b16271e Compare March 22, 2026 16:07

bk138 requested changes Mar 22, 2026

View reviewed changes

Uh oh!

Conversation

Chapoly1305 commented Mar 20, 2026

ARD Authentication Support

Cryptography Enhancements

Build System and Platform Support

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bk138 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Chapoly1305 commented Mar 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants