Repository navigation
Test redacted Durable Object failure reports and flags through the router - #37
Draft
RhysSullivan wants to merge 6 commits into
Draft
RhysSullivan wants to merge 6 commits into
RhysSullivan wants to merge 6 commits into
Conversation
RhysSullivan
added this pull request to stack #38
October 8, 2026 04:35
RhysSullivan
force-pushed
the
d040/do-failure-reports-tests
branch
2 times, most recently
from
October 8, 2026 05:46
3d0bb86 to
0315e92
Compare
RhysSullivan
force-pushed
the
d040/do-failure-reports-tests
branch
from
October 8, 2026 07:32
0315e92 to
d19ac97
Compare
Replaces the retry tests: a retryable reset and a WorkOS authorize are answered once. Synthetic tokens, codes, addresses and the instance name in an error, path or header must not reach the response or the log. A one-shot storage failure inside the router, the control plane's catches and the object's load keeps its flags all the way to the Worker's 503.
A Miniflare probe runs the real Worker and Durable Object with injected storage, addressing and stub failures, and checks every tail event, workerd's output and each response: no exception events and no secret. Unit tests cover secret-bearing error names and methods, body and addressing failures, and plain seed and credential failures.
RhysSullivan
force-pushed
the
d040/do-failure-reports-tests
branch
from
October 8, 2026 09:44
d19ac97 to
16d21da
Compare
RhysSullivan
force-pushed
the
d040/do-failure-reports-tests
branch
from
October 8, 2026 10:15
d4e5cff to
5347c6b
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tests for #36.
workerd runtime probe (
runtime.test.ts). This test bundles the real Worker and Durable Object and runs them in Miniflare/workerd with a tail consumer. It records every tail event (the source of Workers Logs and Issues, uncaught exception events included), workerd's raw stdout and stderr, each response, and the Analytics Engine data points the code writes. The probe'sFAILURESdataset turns each point into a tagged log line. Seven faults are injected, each with a synthetic secret in the message, the errornameand the instance URL:/_emulate/seedand/_emulate/credentials;idFromNamethrowing;The test asserts:
ok;event.request.url, and thex-emulator-instance/x-emulator-base-urlheaders our Worker sets on its request to the object. Cloudflare can store an invocation's request with telemetry, which is why observability is off. Any new field carrying it fails the test.It adds
miniflareandesbuildas dev dependencies of@emulators/cloudflare. They are pinned to versions already in the store, because ofminimumReleaseAge.Telemetry settings.
wrangler.jsoncsets observability, logs, traces and issues each tofalse, has notail_consumersorlogpush, and bindsFAILUREStoemulate_hosts_failures.Worker. Every failure test also asserts that the console stays silent and that the stored data points carry no secret.
retryablereset is answered once, with the full report, and recorded as exactly one data point without the instance hash. No replay.retryableWorkOSGET /oauth2/authorizeis not replayed.authorizationheader do not appear in the response or the data point. The data point does not hold the instance hash either.cf-rayare reported asunknown,unmatchedandnull.overloadedfailure gets a 503; aremotefailure gets a 500.idFromNamethat throws, is reported without reaching the object.worker_error.SYNTHETICtoken482913is reported asother, aTypeErrorasTypeError, and a methodSYNTHETICTOKENasOTHER.Durable Object. These tests use a real object over in-memory storage; each injected storage failure is one-shot.
emulator_errorreport, and a secret-bearing error thrown inside the router stays out of the response and the data point.emulator_unavailablereport with its flags. This holds while the object loads, through the service router during reset, and through seed and credentials. It used to be rethrown.other.Core
routePatternreturns the matched template.createServerwithrethrowUnexpectedErrorsrethrows errors without a status and keeps API errors.ControlPlaneRejectionwith a 400 and send any other error to the error handler (500, or a rethrow withrethrowUnexpectedErrors).Mutation checks. Each of these reverts makes the listed tests fail:
worker_errortest + runtime probeidFromNamemoved outside the catcherrorClassaccepts any alphanumeric namemethodaccepts any uppercase wordconsole.error(report)put backobservability.enabledset totrueResults:
@emulators/cloudflare40 passed and@emulators/core88 passed. With the exact-event wait, the runtime probe passed 32 of 32 reruns: 20 one at a time and 12 with four running at once. One run of the cloudflare suite failed once, and its output wasn't captured. It did not recur in 85 reruns: 30 of the runtime probe alone, 31 of the full suite, and 24 with four suites running at once.pnpm build,format:check,type-check,lintandtestpass across the repo.