Add SSL_CTX_add_client_custom_ext (OpenSSL-compat client custom extensions)#10625
Draft
julek-wolfssl wants to merge 6 commits into
Draft
Add SSL_CTX_add_client_custom_ext (OpenSSL-compat client custom extensions)#10625julek-wolfssl wants to merge 6 commits into
julek-wolfssl wants to merge 6 commits into
Conversation
…sions)
Implements the OpenSSL-compatible legacy client custom extension API for
TLS 1.2 and below. Application-defined extensions carry arbitrary IANA
types, which cannot live in the TLSX list (it keys every extension on a
fixed 72-bit semaphore index that an arbitrary type would overrun), so
they are kept in a separate list on the WOLFSSL_CTX and processed
alongside the unknown-extension handling, mirroring OpenSSL's custext.
- wolfSSL_CTX_add_client_custom_ext() registers a {add,free,parse} method
set with validation (ext_type <= 0xffff, no free_cb without add_cb, not
an internally handled type, no duplicates).
- ClientHello: TLSX_GetRequestSize runs each add_cb, serializes the wire
bytes into a per-connection cache and calls free_cb; TLSX_WriteRequest
copies them out. Honors add_cb returns 1/0/-1 (+alert).
- ServerHello: the unknown-type case in TLSX_Parse dispatches to parse_cb
(<=0 aborts with the requested alert). Gated to non-TLS-1.3 so TLS 1.3
keeps RFC 8446 unsupported_extension behavior.
- Lists/buffers freed in SSL_CtxResourceFree and SSL_ResourceFree.
- openssl/ssl.h maps SSL_CTX_add_client_custom_ext and the custom_ext_*_cb
typedefs.
Adds unit tests covering registration validation, a TLS 1.2 memio
handshake (add/free callbacks balance) and ServerHello parse dispatch.
Builds with and without OPENSSL_EXTRA; full API suite passes.
Address three OpenSSL-compatibility gaps in the client custom extension handling, matching ssl/statem/extensions_cust.c and extensions.c: 1. Unsolicited responses: track the custom extension types actually emitted in the ClientHello (ssl->customExtSent, built alongside customExtData). On ServerHello, a custom extension whose type was not sent is rejected with unsupported_extension, mirroring OpenSSL's SSL_EXT_FLAG_SENT check. add_cb declining to send a type for a given handshake now correctly makes a server echo unsolicited. 2. Resumption: OpenSSL registers the legacy API with SSL_EXT_IGNORE_ON_RESUMPTION and skips parsing on a resumed handshake. Suppress custom-extension parsing when ssl->options.resuming is set so a server echo is silently ignored. (Evaluated at ServerHello-parse time, i.e. the optimistic resumption-attempt state, since wolfSSL finalizes resumption later in CompleteServerHello. The send path is left intact, matching OpenSSL where s->hit is 0 at ClientHello.) 3. Duplicates: the semaphore-based duplicate detection cannot cover arbitrary custom types. Scan the already-parsed portion of the message for an earlier extension of the same registered custom type and abort with DUPLICATE_TLS_EXT_E, matching OpenSSL giving each custom extension its own slot. Adds tests for unsolicited rejection, duplicate rejection, and resumption ignore; updates the parse test to emit first. Builds with and without OPENSSL_EXTRA; full API suite passes.
…hods Follow-up addressing two issues from review of the previous fix: 1. Resumption (was: parse skipped on optimistic resuming flag, then on a cached-ticket heuristic). At ServerHello-parse time ssl->options.resuming only means the client *attempted* resumption, and a cached session ticket does not mean the server agreed to resume. Gate the ignore on the actual server-confirmed signal: resumption was attempted AND the server echoed our session ID back (RFC 5246, and RFC 5077 for tickets with a non-empty session ID). A server that falls back to a full handshake -- whether plain or after a ticket attempt -- does not echo our session ID, so its extensions are still parsed/validated and an unsolicited one is rejected. 2. Flexible client methods (was: API disabled when max version is TLS 1.3). The send path gated on ssl->version < TLS 1.3, but before the handshake that field holds the method's max version, so wolfSSLv23_client_method() / wolfTLS_client_method() (initialized to TLS 1.3) never offered the extension even when negotiating TLS 1.2. The ClientHello now always offers the extension regardless of max version, matching OpenSSL (is_tls13 is false while building the ClientHello). The negotiated-version restriction remains enforced on the parse side, where ssl->version is the negotiated version (TLS 1.3 ServerHellos are routed to DoTls13ServerHello before this code). Adds tests: flexible-method (v23 client -> TLS 1.2) and TLS 1.3 handshakes both offer the extension without breaking; resumption-fallback and ticket-fallback reject an unsolicited extension; resumption-ignored sets up a matching session ID. Builds with and without OPENSSL_EXTRA; full API suite passes.
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Implements OpenSSL-compatible legacy client custom extensions (SSL_CTX_add_client_custom_ext) for TLS 1.2 and below, including registration, ClientHello serialization, ServerHello parse dispatch, and related validation behaviors.
Changes:
- Added public API + OpenSSL-compat typedef/macro mappings for client custom extensions.
- Introduced internal storage and processing for app-defined custom extensions alongside unknown-extension handling.
- Added unit tests for registration validation, handshake behavior across TLS 1.2/1.3, parse dispatch, and unsolicited/duplicate/resumption handling.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| wolfssl/ssl.h | Adds public callback typedefs and wolfSSL_CTX_add_client_custom_ext API declaration. |
| wolfssl/openssl/ssl.h | Maps OpenSSL names (custom_ext_*_cb, SSL_CTX_add_client_custom_ext) onto wolfSSL API. |
| wolfssl/internal.h | Introduces WOLFSSL_CustomExt and adds per-CTX/per-SSL fields for custom ext wire bytes and sent-type tracking. |
| src/tls.c | Implements custom ext registration, ClientHello build/write integration, parse dispatch, and duplicate detection. |
| src/internal.c | Frees custom ext lists/buffers during ctx/ssl teardown. |
| tests/api/test_tls_ext.h | Exposes new custom-ext test declarations. |
| tests/api/test_tls_ext.c | Adds comprehensive unit tests for custom-ext behaviors. |
| tests/api.c | Registers new test cases in the test runner. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The unit tests in tests/api/test_tls_ext.c call TLSX_CustomExt_BuildRequest directly, but it was declared WOLFSSL_LOCAL, so it is given hidden visibility and is not exported from libwolfssl.so. Linking tests/unit.test against the shared library therefore failed with 'undefined reference to TLSX_CustomExt_BuildRequest' in every opensslextra-enabled smoke build. Declare it WOLFSSL_TEST_VIS (matching TLSX_Find, TLSX_Parse, etc.) so the symbol is exported when test visibility is enabled and remains WOLFSSL_LOCAL in production builds. Also add the WOLFSSL_API_PREFIX_MAP rename to match the surrounding convention.
- Gate the public custom-ext typedefs/declaration (wolfssl/ssl.h) and the OpenSSL-compat typedefs/macro (wolfssl/openssl/ssl.h) on HAVE_TLS_EXTENSIONS in addition to OPENSSL_EXTRA, matching the implementation guard so an OPENSSL_EXTRA-only build can no longer expose an unbuildable API. - Replace the hard-coded 62 semaphore boundary in TLSX_ToSemaphore() and TLSX_Parse() with a documented SEMAPHORE_MAX_DIRECT_TYPE constant so the duplicate-detection boundary lives in one place. - Mark the TLSX_CustomExt_BuildRequest definition WOLFSSL_TEST_VIS to match its internal.h declaration (the codebase repeats WOLFSSL_TEST_VIS on definitions but not WOLFSSL_LOCAL, which is inherited from the header). - Use the addresses of static objects for the custom-ext test add/parse args instead of casting integer constants to void*. - Fix the test_tls_ext.h include-guard #endif comment.
Address the second round of review on wolfSSL#10625: - TLSX_CustomExt_BuildRequest: if an add_cb returns 1 with outlen > 0 but leaves *out == NULL, reject with BAD_FUNC_ARG (running free_cb for cleanup) instead of dereferencing NULL in the XMEMCPY. - TLSX_GetRequestSize: the extensions accumulator is a word16, so a large custom extension could wrap "length += customSz" before the block-length check and under-report the buffer size, risking an out-of-bounds write in TLSX_WriteRequest. Reject an oversized total using a wider type before the addition. Adds test_wolfSSL_custom_ext_add_null covering the NULL-pointer path (clean failure plus free_cb invocation).
julek-wolfssl
force-pushed
the
client-custom-ext
branch
from
2fc5757 to
1de6045
Compare
Comment on lines
+16809
to
+16816
| WOLFSSL_TEST_VIS int TLSX_CustomExt_BuildRequest(WOLFSSL* ssl, word16* pSz) | ||
| { | ||
| WOLFSSL_CustomExt* meth; | ||
| byte* data = NULL; | ||
| word32 dataSz = 0; /* word32 to detect a word16 wire-field overflow */ | ||
| int ret = 0; | ||
|
|
||
| *pSz = 0; |
Comment on lines
+16984
to
+16990
| if (ssl->options.resuming && ssl->options.haveSessionId && | ||
| ssl->arrays != NULL && ssl->session != NULL && | ||
| ssl->arrays->sessionIDSz == ID_LEN && | ||
| ssl->session->sessionIDSz == ID_LEN && | ||
| XMEMCMP(ssl->arrays->sessionID, ssl->session->sessionID, | ||
| ID_LEN) == 0) { | ||
| return 0; |
Comment on lines
+18036
to
+18047
| else if (type > SEMAPHORE_MAX_DIRECT_TYPE && | ||
| TLSX_CustomExt_IsRegistered(ssl, type)) { | ||
| word16 scan = 0; | ||
| word16 upto = (word16)(offset - HELLO_EXT_TYPE_SZ - OPAQUE16_LEN); | ||
| while (scan + HELLO_EXT_TYPE_SZ + OPAQUE16_LEN <= upto) { | ||
| word16 sT, sS; | ||
| ato16(input + scan, &sT); | ||
| ato16(input + scan + HELLO_EXT_TYPE_SZ, &sS); | ||
| if (sT == type) | ||
| return DUPLICATE_TLS_EXT_E; | ||
| scan = (word16)(scan + HELLO_EXT_TYPE_SZ + OPAQUE16_LEN + sS); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the OpenSSL-compatible legacy client custom extension API
(
SSL_CTX_add_client_custom_ext) for TLS 1.2 and below.Application-defined extensions carry arbitrary IANA types, which cannot live
in the TLSX list (it keys every extension on a fixed 72-bit semaphore index
that an arbitrary type would overrun), so they are kept in a separate list on
the
WOLFSSL_CTXand processed alongside the unknown-extension handling,mirroring OpenSSL's custext.
Commits
Add
SSL_CTX_add_client_custom_ext— registers a{add,free,parse}method set with validation. ClientHello runs each
add_cband serializesthe wire bytes; ServerHello dispatches the unknown-type case to
parse_cb.Gated to non-TLS-1.3 so TLS 1.3 keeps RFC 8446
unsupported_extensionbehavior. Lists/buffers freed in
SSL_CtxResourceFree/SSL_ResourceFree;openssl/ssl.hmaps the API andcustom_ext_*_cbtypedefs.Reject unsolicited/duplicate exts and ignore on resumption — tracks the
custom types emitted in the ClientHello and rejects a server echo of an
unsent type (
unsupported_extension), suppresses parsing on a resumedhandshake (
SSL_EXT_IGNORE_ON_RESUMPTION), and scans for duplicate customtypes (
DUPLICATE_TLS_EXT_E), matchingextensions_cust.c/extensions.c.Confirmed-resumption gate and flexible client methods — gates the
resumption ignore on the server-confirmed signal (resumption attempted AND
the server echoed our session ID), and always offers the extension in the
ClientHello regardless of the method's max version, so
wolfSSLv23_client_method()/wolfTLS_client_method()offer it whennegotiating TLS 1.2. The negotiated-version restriction remains enforced on
the parse side.
Adds unit tests covering registration validation, TLS 1.2/1.3 and
flexible-method handshakes (add/free callbacks balance), ServerHello parse
dispatch, and unsolicited/duplicate/resumption handling. Builds with and
without
OPENSSL_EXTRA; full API suite passes.