Repository navigation
storage: support AWS IAM authentication for PostgreSQL connections - #40911
val-materialize wants to merge 6 commits into
Conversation
The RDS IAM auth token generator lived in mz-mysql-util, but nothing in it is specific to MySQL: it signs a request for a host, port, and database user, and RDS and Aurora accept the same token for MySQL and PostgreSQL. Move it to mz-aws-util behind a new `rds` feature and make it public, so that other crates (notably mz-postgres-util) can use it for IAM authentication. mz-mysql-util now calls the shared function. No behavior change. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Allow `AWS CONNECTION` on `CREATE CONNECTION ... TO POSTGRES`, so that a connection can authenticate to Amazon RDS or Aurora PostgreSQL with an IAM authentication token in place of a password. This matches what MySQL connections already support. * Planner: accept the `AWS CONNECTION` option for PostgreSQL. Reject it together with `PASSWORD`, and require SSL, because the token is sent as the password. * `PostgresConnection` gains an optional `aws_connection` and loads the AWS SDK config when it builds the connection config. * `mz_postgres_util::Config` takes an optional `SdkConfig`. When set, every new connection signs a fresh token for the database's own host and port, including when it connects through an SSH tunnel or AWS PrivateLink. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…postgres-iam-auth
Materialize ingests over a replication connection, and Amazon RDS only accepts IAM authentication on replication connections when rds.iam_auth_for_replication is set. Also list the rds_replication role. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
martykulma
left a comment
There was a problem hiding this comment.
This ends up causing mz cli and mz_debug to pull in the AWS crate.
Not sure if that's a problem, but it can be avoided:
- set
default-features=falseformz-postgres-utilin those crates - make
mz-orea non-optional dependency ofmz-postgres-utiland drop it from the tunnel list (every feature pulls in tunnel, so they all requiredmz-ore)
It needs to be tested against an actual RDS PG and Aurora to make sure things are working correctly before merge. I'll hold off on approving until we can get an E2E test.
mz-postgres-util's tunnel feature now pulls in the AWS crates for IAM authentication. mz and mz-debug only use the query helpers, so depend on mz-postgres-util without default features there. To make that build, mz-ore becomes a non-optional dependency of mz-postgres-util: the crate re-exports mz_ore::sql unconditionally, so it never compiled without it. The tunnel feature now only adds mz-ore's async feature. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Thanks, both make sense. AWS crates in E2E test: agreed, this should not merge without it. I will try to test the CI build against RDS PG and Aurora myself, but I might need some help getting it deployed. The steps are on SS-593; the main thing I want to confirm is that a source stays healthy past the 15-minute token lifetime. |
QA LLM Review1. MEDIUM -- Validation succeeds when IAM replication is disabled
With IAM database authentication and logical replication enabled, leaving DetailsThe validator opens an ordinary SQL session and checks the WAL level, WAL senders, and available slots. All three checks can pass. The replication worker then calls 2. MEDIUM -- Role credential fetching runs on the Timely worker
With assume-role authentication, opening a PostgreSQL connection now polls a network credential request directly inside the source operator. On a busy Timely worker, delays between polls can exhaust the STS request timeout and restart a source whose credentials and database are healthy. DetailsThe replication worker requests 3. MEDIUM -- The verified TLS example omits the database CA
The example selects DetailsThe production image ships the distribution CA bundle ( |
|
Is medium… good? |
|
Running e2e tests. Will keep you posted |
E2E test against RDS PostgreSQL and Aurora PostgreSQL: passedTested the CI-built image for this PR's head ( The database user (
Planner rejections behave as intended: What this answers
Things worth knowing
Not covered
The test resources (EC2 machine, RDS instance, Aurora cluster, IAM roles) were torn down right after the run. Full log: every statement, its output, and UTC timestamps (AWS account ID redacted) |
martykulma
left a comment
There was a problem hiding this comment.
nice - thanks for running those tests!
Are those doc changes going to go live after the merge? You might have to split them out so they go out with the next release.
The rest looks good 🐟
Merging to main publishes the docs immediately, before the release that contains this change is live. The docs now live in a separate PR that merges when that release is out. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
@martykulma tysm for the review! Splitting out docs to another PR then merging this |
|
docs in #40923 to be merged during v26.52.0 (next week) |
Motivation
Part of SS-593. IAM authentication works today for RDS MySQL connections only. Users with PostgreSQL on Amazon RDS or Aurora have asked for the same, so they can stop managing a database password.
Builds on #40909 (merged), which moved the token signer into
mz-aws-util.Description
CREATE CONNECTION ... TO POSTGRESnow acceptsAWS CONNECTION:CREATE CONNECTION pg TO POSTGRES ( HOST 'instance.foo000.us-west-1.rds.amazonaws.com', DATABASE 'postgres', USER 'materialize', AWS CONNECTION aws_rds_postgres, SSL MODE 'verify_full' );PASSWORD, and requires SSL, because the token is sent as the password. Same rules and error messages as MySQL.PostgresConnection. Gains an optionalaws_connection, and loads the AWS SDK config when it builds the connection config.mz_postgres_util::Config. Takes an optionalSdkConfig. When set, every call to connect signs a fresh token and uses it as the password. A token is valid for 15 minutes and is only checked when a connection opens.mainpublishes docs immediately. That PR merges when the release containing this change is live.Things for reviewers to weigh in on:
PostgresConnectiongains a field. I believe this is safe in the same way the MySQL field was, but please check version skew betweenenvironmentdandclusterd.Verification
test/pg-cdc/create-alter-iam-connection.tdcovers planning: create, the twoALTER CONNECTIONrejections, and the create-time errors.rust-cargo-test, and thepostgres-cdcandmysql-cdcsuites.Cargo.lockwas edited by hand, and CI accepted it.2900dbc14. Passed, including replication past the 15-minute token lifetime and reconnecting with a fresh token. Receipt: storage: support AWS IAM authentication for PostgreSQL connections #40911 (comment) . The only change since that commit removes the docs from this PR. SSH tunnel and PrivateLink were not tested.🤖 Generated with Claude Code