Added support to connect and perform CRUD operations with couchbase by giriraj-singh-couchbase · Pull Request #3138 · apache/brpc

giriraj-singh-couchbase · 2025-11-05T06:55:07Z

This pull request introduces Couchbase Binary Protocol support to bRPC, enabling communication with Couchbase Server and Capella deployments. It adds a new protocol handler, request/response builders, comprehensive example clients, performance benchmarking tools, and detailed documentation. The changes are grouped below by theme:

Couchbase Binary Protocol Integration

Protocol Layer (src/brpc/policy/):

Added couchbase_protocol.h and couchbase_protocol.cpp implementing the complete binary protocol framing, parsing, with support for all standard Couchbase commands (GET, SET, ADD, REPLACE, DELETE, etc.)

High-Level API (src/brpc/):

Implemented CouchbaseOperations class in src/brpc/couchbase.h and src/brpc/couchbase.cpp (~3,500 lines), providing simplified APIs for:
- Authentication: authenticate() and authenticateSSL() for both non-SSL and SSL connections (required for Couchbase Capella)
- Bucket management: selectBucket() for bucket selection
- CRUD operations: get(), upsert(), add(), replace(), append(), prepend(), delete_() with Result struct for clean error handling
- Pipeline batching: beginPipeline(), pipelineRequest(), executePipeline(), clearPipeline() for batching multiple operations in a single network call.
Implemented CouchbaseRequest and CouchbaseResponse classes (inherit from NonreflectableMessage) providing low-level request construction and response parsing
Implemented CouchbaseManifestManager singleton for thread-safe collection manifest management with automatic caching and refresh

Example Clients

Basic Client (example/couchbase_c++/couchbase_client.cpp - 462 lines):

Demonstrates authentication (both SSL and non-SSL), bucket selection, basic CRUD operations, collection-scoped operations, and error handling patterns

Multithreaded Client (example/couchbase_c++/multithreaded_couchbase_client.cpp - 375 lines):

Demonstrates two threading patterns:
- Multiple threads sharing a single CouchbaseOperations instance (same bucket, shared connection)
- Multiple threads with dedicated CouchbaseOperations instances (different buckets, isolated connections)

Documentation

Comprehensive Guide (docs/en/couchbase_example.md):

Key Features

SSL/TLS Support: for Couchbase Capella and secure on-premises deployments
Collection Management: Native Couchbase 7.0+ collections with automatic ID resolution, manifest caching.
Pipeline Batching: Batch multiple operations.
Error Handling: Clean Result struct with success flag, error messages, status codes, and values

What problem does this PR solve?

Added native support to interact with Couchbase servers using the binary protocol, enabling efficient CRUD operations, SSL/TLS encrypted connections and collection management.

Side effects:

Performance effects: None
Breaking backward compatibility: NO

Check List:

Please make sure your changes are compilable.
When providing us with a new feature, it is best to add related tests.
Please follow Contributor Covenant Code of Conduct.

Merging from source

…ouchbaselabs/cb_brpc into couchbase_binary_protocol_brpc t :wq

…functionality to handle server side manifest updates.

Unnecessary file

updated the documentation on thread safe operations and fixed small small discrepancies.

…ownload certificate

Added support to connect and perform CRUD operations with couchbase

Copilot

Pull Request Overview

This pull request adds comprehensive support for Couchbase Binary Protocol to bRPC, enabling communication with Couchbase Server clusters using the native binary protocol. The implementation provides both high-level and low-level APIs for key-value operations with support for collections, pipelining, SSL/TLS, and multithreading.

Key Changes:

Complete Couchbase Binary Protocol implementation with request/response parsing and serialization
High-level CouchbaseOperations API with simple CRUD operations and pipeline support
SSL/TLS support for secure connections to Couchbase Capella (cloud)
Collection-scoped operations with automatic manifest caching and refresh
Multithreading support with both shared and separate instance patterns

Reviewed Changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
src/brpc/proto_base.proto	Adds base message types for Couchbase request/response
src/brpc/policy/couchbase_protocol.h	Defines protocol structures, enums, and function declarations for Couchbase Binary Protocol
src/brpc/policy/couchbase_protocol.cpp	Implements message parsing, serialization, and protocol handling
src/brpc/couchbase.h	Defines high-level `CouchbaseOperations` API with request/response classes and manifest management
src/brpc/couchbase.cpp	Implements CRUD operations, authentication, collection management, and pipeline functionality
src/brpc/options.proto	Adds PROTOCOL_COUCHBASE enum value for protocol registration
src/brpc/global.cpp	Registers Couchbase protocol with bRPC framework
example/couchbase_c++/couchbase_client.cpp	Single-threaded example demonstrating all features
example/couchbase_c++/multithreaded_couchbase_client.cpp	Multithreaded example with shared and separate instance patterns
example/couchbase_c++/Makefile	Build configuration for examples
example/couchbase_c++/CMakeLists.txt	CMake build configuration for examples
docs/en/couchbase_example.md	Comprehensive documentation with usage patterns and examples
CMakeLists.txt	Updates C++ standard to C++17 for modern features

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

wwbmmm · 2025-11-07T03:08:49Z

@giriraj-singh-couchbase Thank you for your contribution! This is quite a complicated job. Your code looks very good! I have add some comments.

wwbmmm · 2025-11-07T03:11:41Z

A small advice: you can add an unit test for the couchbase protocol, just base on the code in example/couchbase_c++/couchbase_client.cpp

Traditional bRPC coding approach doesn't uses high level functions but provides more control to the user fixed formatting issues. fixed the bug in couchbase.cpp where logic to check the cache is empty was inverted

giriraj-singh-couchbase · 2025-11-11T07:38:58Z

Hi @wwbmmm, I have added the unit test cases, traditional approach example and updated the documentation regarding it.
Can you take a look at it? Thank you.

wwbmmm · 2025-11-12T04:19:03Z

Hi @wwbmmm, I have added the unit test cases, traditional approach example and updated the documentation regarding it. Can you take a look at it? Thank you.

Most look good, can you remove the using namespace std; in the couchbase.h file? This header file may be included by user, so the using namespace std; may cause some conflicts in user code.

Updated base branch

giriraj-singh-couchbase · 2025-11-12T09:28:29Z

Hi @wwbmmm, I have added the unit test cases, traditional approach example and updated the documentation regarding it. Can you take a look at it? Thank you.

Most look good, can you remove the using namespace std; in the couchbase.h file? This header file may be included by user, so the using namespace std; may cause some conflicts in user code.

Removed.

wwbmmm · 2025-11-12T13:05:48Z

LGTM
I think the unittest failure is not related to this PR, we can ignore it.

giriraj-singh-couchbase · 2025-11-14T17:12:11Z

LGTM I think the unittest failure is not related to this PR, we can ignore it.

Can you re-run the checks?

wwbmmm · 2025-11-15T06:46:51Z

LGTM I think the unittest failure is not related to this PR, we can ignore it.

Can you re-run the checks?

I have fixed the unittest in this PR: #3153 . You can merge this PR and then re-run the checks.

giriraj-singh-couchbase · 2025-11-18T23:02:15Z

Hi @wwbmmm & @chenBright, can I get further feedback on this PR?

…pache#3138) * Implemented Couchbase binary protocol support * added support for single connection type for couchbase * removed unnecessary cout statements * added protocol code for helo packet * fixed vbucketID code for identification, fixed add and get functions * Added test cases for threaded get and add functions * Added Error Handling code and made upsert and delete examples * added makefile for example/couchbase_c++ * fixed bugs in couchbase header files * Added License and formatted to google c++ norms * fixed bugs, added support for collections and added couchbase_client.md * fixed license issue * added custom logic for caching collectionIDs * added caching of collection manifests * Added example code for multithreaded demonstration * updated CMake * Abstracted CRUD operations * Added pipeline/batching support * commented unused variables * Updated support for C++17 * fixed some issue. * Using Mutex instead of shared lock to support c++11 * Formatted code to google c++ format * Introduced local cache per-instance of CouchbaseOperations and added functionality to handle server side manifest updates. * Delete MODULE.bazel.lock Unnecessary file * Fixed bugs in local collection cache and collection refresh logic * remove recurring statements * Fixed bugs/repetitive calls to refreshing manifest on server * Formatted function/variable naming scheme and formatted code in c++ google format * removed unnecessary code * updated comments * updated comments * updated documentation * updated documentation * updated documentation * updated documentation * Updated documentation * Updated documentation * Update documentation * Added features and fixed bugs in multithreaded environment Using connection_groups to differentiate between connections across CouchbaseOperations instances to different buckets. Renamed CollectionManifestTracker class to CollectionManifestManager and all the related functionality inside it as before refreshing method was outside this class Added two different authenticate method authenticate(not secure) and authenticateSSL(secure) * Updated multithreaded and single threaded code. Added an example where a single instance is being shared across the threads when operating on single bucket. * updated documentation updated the documentation on thread safe operations and fixed small small discrepancies. * removed commented code and updated readme to have links for cluster download certificate * removed unused code. * Added traditional bRPC coding approach Traditional bRPC coding approach doesn't uses high level functions but provides more control to the user fixed formatting issues. fixed the bug in couchbase.cpp where logic to check the cache is empty was inverted * updated couchbase_example.md * added unit test cases * removed using namespace std from couchbase.h * restored original CMakeLists.txt

giriraj-singh-couchbase added 30 commits July 30, 2025 07:02

Implemented Couchbase binary protocol support

0c758fa

added support for single connection type for couchbase

aa034c7

removed unnecessary cout statements

9d81669

added protocol code for helo packet

7946cb4

fixed vbucketID code for identification, fixed add and get functions

d2ca0d4

Added test cases for threaded get and add functions

d1497c1

Added Error Handling code and made upsert and delete examples

aae6a3d

Merge branch 'apache:master' into couchbase_binary_protocol_brpc

95ba8c0

added makefile for example/couchbase_c++

2ff17ff

fixed bugs in couchbase header files

10964aa

Added License and formatted to google c++ norms

8bc4c59

fixed bugs, added support for collections and added couchbase_client.md

b38bb2d

fixed license issue

07d1b0c

added custom logic for caching collectionIDs

f8defb1

added caching of collection manifests

6a8200d

Added example code for multithreaded demonstration

db1afc1

updated CMake

753e662

Abstracted CRUD operations

4bdb6a4

Added pipeline/batching support

80713be

commented unused variables

e77e3cd

Merge pull request #5 from apache/master

792a004

Merging from source

Updated support for C++17

1ce8f28

Merge branch 'couchbase_binary_protocol_brpc' of https://github.com/c…

d229fc2

…ouchbaselabs/cb_brpc into couchbase_binary_protocol_brpc t :wq

fixed some issue.

f03e0f6

Using Mutex instead of shared lock to support c++11

68b4a51

Formatted code to google c++ format

464cf69

Introduced local cache per-instance of CouchbaseOperations and added …

a66208e

…functionality to handle server side manifest updates.

Delete MODULE.bazel.lock

8384ad8

Unnecessary file

Fixed bugs in local collection cache and collection refresh logic

2c21f49

remove recurring statements

510f5d8

giriraj-singh-couchbase added 4 commits October 17, 2025 12:21

updated documentation

1639f5d

updated the documentation on thread safe operations and fixed small small discrepancies.

removed commented code and updated readme to have links for cluster d…

e4242b9

…ownload certificate

removed unused code.

90d5522

Merge pull request #4 from couchbaselabs/couchbase_binary_protocol_brpc

cf7ce31

Added support to connect and perform CRUD operations with couchbase

wwbmmm reviewed Nov 7, 2025

View reviewed changes

Comment thread src/brpc/couchbase.h

Comment thread src/brpc/couchbase.h

Comment thread src/brpc/couchbase.h

Comment thread src/brpc/couchbase.cpp

wwbmmm requested a review from Copilot November 7, 2025 02:42

Copilot AI reviewed Nov 7, 2025

View reviewed changes

giriraj-singh-couchbase added 3 commits November 10, 2025 16:19

Added traditional bRPC coding approach

c1c76f1

Traditional bRPC coding approach doesn't uses high level functions but provides more control to the user fixed formatting issues. fixed the bug in couchbase.cpp where logic to check the cache is empty was inverted

updated couchbase_example.md

d45e5f3

added unit test cases

3e225de

giriraj-singh-couchbase added 3 commits November 12, 2025 14:17

Merge branch 'apache:master' into master

da1790b

removed using namespace std from couchbase.h

7e4560d

Merge branch 'master' of https://github.com/couchbaselabs/cb_brpc

a34b66d

Updated base branch

wwbmmm requested a review from chenBright November 12, 2025 13:07

restored original CMakeLists.txt

9d7f9be

Merge branch 'apache:master' into master

4a4e62f

giriraj-singh-couchbase requested a review from wwbmmm November 25, 2025 08:20

wwbmmm merged commit d2ea819 into apache:master Nov 25, 2025
17 checks passed

This was referenced Dec 8, 2025

When will couchbase 7.6 and above be supported? #3027

Closed

支持Couchbase v7.6以上版本 #3025

Closed

Uh oh!

Conversation

giriraj-singh-couchbase commented Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Couchbase Binary Protocol Integration

Example Clients

Documentation

Key Features

What problem does this PR solve?

Check List:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wwbmmm commented Nov 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wwbmmm commented Nov 7, 2025

Uh oh!

giriraj-singh-couchbase commented Nov 11, 2025

Uh oh!

wwbmmm commented Nov 12, 2025

Uh oh!

giriraj-singh-couchbase commented Nov 12, 2025

Uh oh!

wwbmmm commented Nov 12, 2025

Uh oh!

giriraj-singh-couchbase commented Nov 14, 2025

Uh oh!

wwbmmm commented Nov 15, 2025

Uh oh!

giriraj-singh-couchbase commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

giriraj-singh-couchbase commented Nov 5, 2025 •

edited

Loading

wwbmmm commented Nov 7, 2025 •

edited

Loading

giriraj-singh-couchbase commented Nov 18, 2025 •

edited

Loading