Skip to content

database: replace uWebSockets and libuv with libcurl WebSockets - #1959

Open
AustinBenoit wants to merge 1 commit into
mainfrom
libcurl-websockets
Open

AustinBenoit wants to merge 1 commit into
mainfrom
libcurl-websockets

Conversation

@AustinBenoit

@AustinBenoit AustinBenoit commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Replaces Realtime Database's desktop WebSocket transport (uWebSockets v0.14 and libuv v1.40) with libcurl's native WebSocket implementation (CURLOPT_CONNECT_ONLY = 2L, curl_ws_recv(), curl_ws_send()), eliminating two legacy, unmaintained external dependencies from the Firebase C++ SDK.

Architecture & Key Changes

1. External Dependency Deletion (-2 Third-Party Libraries)

  • Removes uWebSockets (2017 v0.14 snapshot + patch) and libuv (2020 v1.40 snapshot) from CMakeLists.txt, cmake/external/CMakeLists.txt, cmake/external_rules.cmake, README.md, scripts/gha/print_matrix_configuration.py, and release_build_files/NOTICES.
  • Removes the Windows psapi and userenv system library link requirements for firebase_database (documented in release_build_files/readme.md).
  • Enforces libcurl >= 8.14.0 with ws / wss protocol support (CURLWS_NOAUTOPONG) at compile time (static_assert in app/rest/curl/curl_handles.h).

2. WebSocketClientImpl (database/src/desktop/connection/)

  • Preserves the existing threading model: each WebSocketClientImpl owns one background event loop thread (EventLoopRoutine / RunEventLoop) and a private CURLM* handle (curl_multi_poll() + curl_multi_wakeup()), marshalling all WebSocketClientEventHandler callbacks (OnOpen, OnMessage, OnClose, OnError) back onto the Database scheduler_ thread via safe_this_.
  • Non-blocking Handshake & Watchdog: Performs the HTTP 101 Switching Protocols upgrade non-blockingly on multi_ with an explicit handshake_deadline_ watchdog enforced on the I/O thread so stalled connects or destructor teardown abort immediately.
  • RFC 6455 Framing & Backpressure:
    • Uses curl_ws_recv() and curl_ws_send() in non-blocking mode (CURLWS_TEXT, CURLWS_BINARY, CURLWS_CONT, CURLWS_PING, CURLWS_PONG, CURLWS_CLOSE).
    • Tracks partial frame progress (frame_in_progress_, current_send_offset_) and waits for socket writability (CURL_WAIT_POLLOUT) on CURLE_AGAIN without busy-spinning.
    • Enforces RFC 6455 §5.5 125-byte control-frame payload limits and caps multi-frame message reassembly at 256 MiB (kMaxMessagePayloadBytes).
  • Mid-Session Error Reporting Fix (Connection::OnError): Removes the redundant second scheduler_->Schedule(...) hop inside Connection::OnError() so mid-session WebSocket errors report kDisconnectReasonWebsocketError before OnClose() transitions the connection to kStateDisconnected.

3. Shared libcurl Primitives (app/rest/curl/)

  • tls_policy.{h,cc}: Centralizes desktop TLS configuration shared by REST (transport_curl.cc) and WebSockets (web_socket_client_impl.cc):
    • Enforces TLS 1.2+ (CURL_SSLVERSION_TLSv1_2), CURLOPT_SSL_VERIFYPEER = 1L, and CURLOPT_SSL_VERIFYHOST = 2L.
    • Uses native OS trust stores on Windows (Schannel) and macOS (SecureTransport / Keychain), and resolves the runtime Linux CA bundle (CURL_CA_BUNDLE, SSL_CERT_FILE, and standard Debian/Ubuntu, RHEL/Fedora, openSUSE, and Alpine paths) and CA directory (SSL_CERT_DIR, /etc/ssl/certs) on Linux.
  • curl_handles.{h,cc}: Provides RAII std::unique_ptr deleters (UniqueEasy, UniqueMulti, UniqueSList), MSVC #pragma comment(lib, ...) directives for Windows SDK system libraries, and a process-wide, mutex-protected reference count (AcquireCurlGlobal() / ReleaseCurlGlobal()) around curl_global_init() / curl_global_cleanup() used across transport_curl.cc, util.cc, and web_socket_client_impl.cc.

4. Supporting Fixes

  • Initializes DatabaseInternal::persistence_enabled_ = false on desktop (database_desktop.cc).
  • Guards Logger::LogMessageImplV against a null parent_logger_ (app/src/logger.cc + unit test in app/tests/logger_test.cc).
  • Fixes build_scripts/desktop/package.sh temporary linker version script path generation (hide_symbols_*.lds).

File Breakdown

Area Files Description
Dependency Removal CMakeLists.txt, cmake/external/CMakeLists.txt, cmake/external_rules.cmake, cmake/external/libuv.cmake (deleted), cmake/external/uWebSockets.cmake (deleted), scripts/git/patches/uWebSockets/... (deleted), release_build_files/NOTICES, release_build_files/readme.md, README.md, scripts/gha/print_matrix_configuration.py Removes uWebSockets & libuv build rules, patches, notices, and Windows psapi/userenv link flags
Shared libcurl Core app/rest/CMakeLists.txt, app/rest/curl/curl_handles.{h,cc}, app/rest/curl/tls_policy.{h,cc}, app/rest/transport_curl.cc, app/rest/util.{h,cc} Shared curl_global refcount, RAII handle wrappers, and unified TLS 1.2+ / OS trust store policy
RTDB WebSocket Client database/CMakeLists.txt, database/src/desktop/connection/web_socket_client_impl.{h,cc}, database/src/desktop/connection/connection.cc, database/src/desktop/connection/util_connection.cc, database/src/desktop/database_desktop.{h,cc} Native libcurl WebSocket client with private CURLM* poll/wakeup loop
Tests & Packaging database/tests/CMakeLists.txt, database/tests/desktop/connection/web_socket_client_impl_test.cc, database/integration_test/CMakeLists.txt, app/src/logger.cc, app/tests/logger_test.cc, build_scripts/desktop/package.sh, cmake/CTestCustom.cmake Local loopback RFC 6455 WebSocket unit tests (8/8), logger unit test, and packaging script fix

Verification

  • Unit Tests:
    • firebase_rtdb_desktop_connection_web_socket_client_impl_test: 8 / 8 passed (happy-path connect/echo/close, immediate teardown before connect and mid-handshake, multi-fragment CURLWS_CONT reassembly with interleaved PING/PONG and 100 KB multi-chunk payload + User-Agent/X-Firebase-AppCheck headers, server-initiated CLOSE, connection refused OnError, stalled handshake watchdog timeout, and ws/wss URI scheme enforcement).
    • firebase_rtdb_desktop_connection_connection_test: 9 / 9 passed against live wss:// Realtime Database backend.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces the legacy uWebSockets and libuv WebSocket transport in the desktop Realtime Database SDK with a native libcurl-based implementation. It removes the libuv and uWebSockets dependencies entirely, centralizes the TLS policy across REST and WebSocket connections, and introduces a robust set of unit and integration tests. The reviewer identified a potential memory leak in web_socket_client_impl.cc where request_headers_.release() is passed directly to curl_slist_append; if the append fails, the original list is leaked. A safer helper lambda was suggested to handle appending safely.

Comment thread database/tests/CMakeLists.txt Resolved
Comment thread app/rest/curl/tls_policy.cc Resolved
Comment thread app/rest/curl/tls_policy.cc Resolved
Comment thread app/rest/util.cc Resolved
- Drop unmaintained external dependencies uWebSockets (v0.14) and libuv
  (v1.40) from CMake, external rules, patches, and release NOTICES.
- Migrate Realtime Database WebSocket client (WebSocketClientImpl) to
  libcurl's native WebSocket support (curl_ws_send, curl_ws_recv,
  CURLOPT_CONNECT_ONLY = 2L).
- Give each WebSocket connection its own lightweight event thread and private
  CURLM* handle using curl_multi_poll() and curl_multi_wakeup().
- Add stalled handshake deadline watchdog (5000 ms default).
- Centralize SDK transport security in app/rest/curl/tls_policy.{h,cc}
  (TLS 1.2+ minimum, peer/hostname verification, and native OS certificate
  trust store via Schannel on Windows, Keychain on macOS, and runtime CA bundle
  detection on Linux).
- Add thread-safe reference-counted AcquireCurlGlobal() / ReleaseCurlGlobal()
  and RAII handle deleters in app/rest/curl/curl_handles.{h,cc}.
- Initialize DatabaseInternal::persistence_enabled_ to false to prevent
  reading uninitialized member variables on desktop platforms.
- Guard Logger::LogMessageImplV when parent_logger_ is null.
- Fix package.sh symbol hiding script generation for libfirebase_database.a.
- Add local loopback RFC 6455 WebSocket unit tests (8/8 passing).
@AustinBenoit AustinBenoit added the tests-requested: quick Trigger a quick set of integration tests. label Nov 4, 2026
@github-actions github-actions Bot added tests: in-progress This PR's integration tests are in progress. and removed tests-requested: quick Trigger a quick set of integration tests. labels Nov 4, 2026
@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

❌  Integration test FAILED

Requested by @AustinBenoit on commit 1c8d0ae
Last updated: Wed Nov 4 10:21 PST 2026
View integration test log & download artifacts

Failures Configs
app_check [TEST] [FAILURE] [iOS] [macos] [1/2 ios_device: simulator_target]
(1 failed tests)  FirebaseAppCheckTest.TestGetTokenForcingRefresh

Add flaky tests to go/fpl-cpp-flake-tracker

@AustinBenoit

AustinBenoit commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

Triggered Firebase Unity SDK build & integration tests against firebase_cpp_sdk_version=libcurl-websockets (32dac605):

@github-actions github-actions Bot added the tests: failed This PR's integration tests failed. label Nov 4, 2026
@firebase-workflow-trigger firebase-workflow-trigger Bot removed the tests: in-progress This PR's integration tests are in progress. label Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: core api: database cla: yes dependencies Pull requests that update a dependency file tests: failed This PR's integration tests failed. type: cleanup

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant