Skip to content

RCBC-304: Add Cluster#wait_until_ready and Bucket#wait_until_ready - #277

Merged
avsej merged 3 commits into
couchbase:mainfrom
avsej:rcbc-304-wait-until-ready
Oct 20, 2026
Merged

avsej merged 3 commits into
couchbase:mainfrom
avsej:rcbc-304-wait-until-ready

Conversation

@avsej

@avsej avsej commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Fixes RCBC-304.

Problem

The SDK has no way to wait for a cluster or bucket to become fully usable. Cluster.connect and Cluster#bucket return after bootstrap. Sessions to the other nodes and services may still be opening.

Change

The first commit makes the backend raise Error::ClusterClosed as a C++ exception, so a closed cluster no longer skips the calling binding's destructors.

The second commit adds Cluster#wait_until_ready and Bucket#wait_until_ready, through the C++ core's wait_until_ready.

cluster = Couchbase::Cluster.connect("couchbase://127.0.0.1", "Administrator", "password")
options = Couchbase::Options::WaitUntilReady.new(service_types: [:kv])
cluster.bucket("travel-sample").wait_until_ready(10_000, options)
  • desired_state: :online waits for every endpoint the ping reports for each requested service. :degraded waits for at least one.
  • For a bucket, :online also waits until every vbucket has its active copy and each placeable replica assigned to a node.
  • With no service_types, every service in the ping report counts. A requested service that no node runs is never ready.
  • The wait raises:
    • Error::UnambiguousTimeout when the timeout elapses;
    • Error::ClusterClosed once the cluster is disconnected;
    • Error::InvalidArgument for an unknown state or service type, or for a timeout that is negative or longer than 365 days.

The third commit makes the test helper wait for the test bucket under CAVES. On CI, a test's first operation sometimes fails with UnambiguousTimeout and retry_attempts: 0.

Known gaps

  • Over couchbase2:// both methods raise Error::FeatureNotAvailable. That client is implemented in Ruby, not over the core.
  • Thread#raise and Timeout.timeout do not interrupt the wait, because the binding blocks without an unblocking function. Cluster#disconnect from another thread ends it with Error::ClusterClosed.
  • The rate of the CAVES timeouts on CI before and after the third commit has not been measured.

This comment was marked as resolved.

@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 9655dd0 to 55ea1e1 Compare October 19, 2026 18:09
@avsej
avsej requested a balanced review from Copilot October 19, 2026 18:09

This comment was marked as resolved.

@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 55ea1e1 to 468e938 Compare October 19, 2026 19:02
@avsej
avsej requested a balanced review from Copilot October 19, 2026 19:02

This comment was marked as resolved.

Motivation
----------
cb_backend_to_public_api_cluster() called rb_raise() when the backend
was closed. rb_raise() leaves the method body without running its C++
destructors. A binding that had already built an object holding heap
memory leaked it.
Every binding reaches this function, directly or through
cb_backend_to_core_api_cluster().

Modifications
-------------
The function throws ruby_exception with Error::ClusterClosed, and the
caller's frames unwind before cb_method raises it in Ruby.
Backend#cluster_labels, the one caller not yet registered through
cb_method, now is.

Results
-------
Calling a backend method after the cluster is closed raises the same
Error::ClusterClosed and message as before, without leaking the
caller's objects. The leak itself is not observable from Ruby, so no
test covers it.
@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 468e938 to 51b5c18 Compare October 19, 2026 19:19
@avsej
avsej requested a balanced review from Copilot October 19, 2026 19:19

This comment was marked as resolved.

@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 51b5c18 to 4e29aa7 Compare October 19, 2026 19:42
@avsej
avsej requested a balanced review from Copilot October 19, 2026 19:42

This comment was marked as resolved.

@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 4e29aa7 to 4dc865b Compare October 19, 2026 19:55
@avsej
avsej requested a balanced review from Copilot October 19, 2026 19:55

This comment was marked as resolved.

Comment thread ext/rcb_diagnostics.cxx Outdated
@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 4dc865b to c2974f8 Compare October 20, 2026 16:38
@avsej
avsej requested a balanced review from Copilot October 20, 2026 16:38

This comment was marked as resolved.

@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from c2974f8 to 3846f25 Compare October 20, 2026 17:19
@avsej
avsej requested a balanced review from Copilot October 20, 2026 17:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Native exception handling and concurrent-disconnect behavior require human integration validation.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 3846f25 to ee2c8e9 Compare October 20, 2026 17:26
@avsej
avsej requested a balanced review from Copilot October 20, 2026 17:26

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Upper-bound timeout safety remains unresolved, and native deadline and concurrent-disconnect behavior require validation against the unavailable C++ core.

1 open issue

Review effort: Balanced

Comment thread ext/rcb_utils.cxx Outdated
@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from ee2c8e9 to 2f13c3e Compare October 20, 2026 17:52
@avsej
avsej requested a balanced review from Copilot October 20, 2026 17:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Native exception unwinding, timeout boundaries, and concurrent disconnect behavior need human integration validation against the C++ core.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

avsej added 2 commits October 20, 2026 11:45
Motivation
----------
Cluster.connect and Cluster#bucket return once bootstrap completes.
Sessions to the remaining nodes and services may still be opening,
and a bucket's replicas may not be placed yet. An operation routed to
such a node waits inside its own timeout, and can time out before it
is sent. The SDK offers no way to wait for those first.

Modifications
-------------
Cluster#wait_until_ready(timeout, options) and
Bucket#wait_until_ready(timeout, options) wait through the C++ core's
wait_until_ready, which pings the services until they respond or the
timeout elapses. Options::WaitUntilReady selects the desired state and
the service types:

- :online waits for every endpoint the ping reports for each requested
  service. :degraded waits for at least one.
- For a bucket, :online also waits until every vbucket has its active
  copy and each placeable replica assigned to a node.
- With no service types, every service in the ping report counts. A
  requested service that no node runs is never ready.

The wait raises Error::UnambiguousTimeout when the timeout elapses,
Error::ClusterClosed once the cluster is disconnected, and
Error::InvalidArgument for an unknown state or service type, or for a
timeout that is negative or too long for the deadline to represent. It
works over couchbase:// and couchbases://; couchbase2:// raises
Error::FeatureNotAvailable.

Results
-------
A caller can block until the cluster or bucket serves the services it
needs. Thread#raise, Thread#kill and Timeout.timeout do not interrupt
the wait. It ends when the state is reached, the timeout elapses, or
Cluster#disconnect runs on another thread.
Motivation
----------
Each test that uses CAVES starts its own CAVES cluster. On CI, the
first KV operation of such a test sometimes fails with
UnambiguousTimeout after the default 2.5 s KV timeout, with
retry_attempts 0.

Modifications
-------------
Under CAVES, TestUtilities#connect calls Bucket#wait_until_ready for
the test bucket, with service_types [:kv] and up to 10 s. It returns
once every KV endpoint answers a ping and the vbucket map is placed.
Runs against a real cluster do not wait.

Results
-------
A test's first KV operation starts after CAVES answers KV pings. A
CAVES cluster not ready within 10 s now fails TestUtilities#connect
with Error::UnambiguousTimeout. The rate of the first-operation
timeouts on CI before and after this change has not been measured.
@avsej
avsej force-pushed the rcbc-304-wait-until-ready branch from 2f13c3e to 3be3849 Compare October 20, 2026 18:45
@avsej
avsej requested a balanced review from Copilot October 20, 2026 18:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Native exception handling and concurrent disconnect behavior need human review supported by integration-test results.

No open issues 🎉

Review effort: Balanced

@avsej
avsej merged commit a3289f8 into couchbase:main Oct 20, 2026
131 of 133 checks passed
@avsej
avsej deleted the rcbc-304-wait-until-ready branch October 20, 2026 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sdk review needed Core logic or public API changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants