Skip to content

test: add KMS Encryption plugin integration tests - #1349

Draft
JuanLeee wants to merge 1 commit into
feat/kms-stack-4-pluginfrom
feat/kms-stack-5-integration-tests
Draft

JuanLeee wants to merge 1 commit into
feat/kms-stack-4-pluginfrom
feat/kms-stack-5-integration-tests

Conversation

@JuanLeee

@JuanLeee JuanLeee commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds the KMS Encryption plugin's integration tests, on PostgreSQL and MySQL servers with LocalStack's KMS, a real KMS key, or Aurora.
It changes no product code.

It is the fifth PR in the KMS stack, after #1330, #1300, #1317 and #1331 (the plugin). Its base is #1331's branch, so its diff is the tests alone, and it stays a draft until the PRs below it merge.

  • Test modules: tests/integration/container/test_kms_encryption_*.py, sqlalchemy/test_sqlalchemy_kms_encryption.py and django/test_django_kms_encryption.py. They cover 66 planned cases: 57 on LocalStack, 6 that need a real key and 3 that need Aurora.
  • Harness: the RUN_KMS_ENCRYPTION_TESTS_ONLY and KMS_LOCALSTACK features, three Gradle tasks, a LocalStack container pinned by digest with no published port, PostgreSQL statement logging in KMS environments, and a JUnit XML report beside each HTML report. Outside a KMS environment the KMS modules skip, and inside one everything else does.
  • A compose lane, tests/integration/kms_local/, that runs the LocalStack modules on a laptop without Gradle, replacing the deleted tests/integration/test_kms_encryption_live.py.
  • A unit test keeping the guide's DDL equal to the tests' schema.

Lanes

  • Compose, no AWS: tests/integration/kms_local/run.sh up, test PG, test MYSQL, down.
  • Harness with LocalStack, no AWS: ./gradlew test-kms-encryption-docker.
  • Real key, with credentials for the test key only: run.sh real PG or real MYSQL, or ./gradlew test-kms-encryption-docker-kms.
  • Aurora, with the Aurora tasks' credentials: ./gradlew test-kms-encryption-aurora.

KMS_MANUAL_TESTS=1 adds three manual tests, which CI never runs.

What CI runs

The new kms_encryption_integration_tests.yml runs three jobs on PostgreSQL and MySQL:

  • kms-localstack: on pull requests that touch the wrapper, the integration tests, pyproject.toml, poetry.lock or the workflow, on pushes to main, and on dispatch. It needs no secrets, so forks run it too.
  • kms-real-key: on pushes to main and on dispatch, through OIDC to a role limited to the test key.
  • kms-aurora: on dispatch only, on two-instance clusters it deletes at the end.

Each job fails when no KMS test ran, as an all-skipped pytest run exits 0.

Results

  • Compose lane, four runs: PostgreSQL 77 passed, 13 skipped; MySQL 145 passed, 27 skipped, 1 strict xfail. After the last code change, the round-trip module reran with the same counts. The prose pass after that changed only comments, docstrings and docs.
  • Each of the 32 P0 tests failed when its behavior was deliberately broken. Most of these controls ran before the review fixes; after them, the controls for the new review checks and for test_mysql_cursor_kinds_and_binary_prefix reran and failed as intended.
  • Real key: PostgreSQL 3 passed, MySQL 4 passed, through run.sh real and the harness (on Finch, with two local workarounds). The three manual tests passed on PostgreSQL; test_via_service_condition_makes_every_statement_fail_closed hasn't rerun since a review fix changed it. A forced failure printed neither the account id nor the key id.
  • Aurora PostgreSQL 17.9 and MySQL 3.10.3, on an earlier commit, since the clusters were deleted before the final one: the auth and TLS test and the trigger test pass, and the MySQL failover test passed with this PR's 390-second reader wait.
  • Not run: test-kms-encryption-docker. Finch couldn't create its LocalStack container and Docker Desktop wasn't available, so this PR's kms-localstack job is its first run.

Expected failures

Both are strict xfails: a fix fails them until the mark goes.

  • test_fetchmany_without_a_size_reads_arraysize_rows[mysql-cext]: AwsWrapperCursor.fetchmany() defaults size to 0 and passes it on, so mysql-connector's C extension returns no rows.
  • test_failover_and_read_write_splitting_keep_encrypting on Aurora PostgreSQL: after a writer failover, the plugin's metadata lookup connects to the old writer and raises MetadataError above failover_v2, so failover never starts. mysql-connector fails earlier, at cursor(), which failover_v2 handles.

Administrator setup, once

  • One symmetric KMS key ($1 a month). CI never creates or changes keys.
  • A CI role trusted through GitHub OIDC with sub equal to repo:aws/aws-advanced-python-wrapper:ref:refs/heads/main (StringEquals) and aud equal to sts.amazonaws.com. It may call kms:GenerateDataKey, kms:Decrypt and kms:DescribeKey on the key and sts:AssumeRole on the Decrypt-only role, with an explicit deny on key and alias changes.
  • A Decrypt-only role, trusted by the CI role, allowed only kms:Decrypt on the key.
  • A maintainer role with the CI role's permissions, for real-key runs before merge, since the CI role trusts only main.
  • Repository variables KMS_KEY_ARN (a full key ARN, not an alias), KMS_REGION, KMS_IT_ROLE_ARN and KMS_DECRYPT_ONLY_ROLE_ARN.

What to check

  • This PR's kms-localstack run on both engines, including its "KMS tests run" count.
  • That other Gradle tasks still work (test-docker with FILTER='test_direct_connection' passed).
  • The report hook that masks passwords, the account id and the key id.
  • The two expected failures, and filing their defects.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Add container tests for the KMS Encryption plugin on PostgreSQL and
MySQL, including SQLAlchemy and Django, against LocalStack's KMS, a
real KMS key and Aurora. The harness gets the RUN_KMS_ENCRYPTION_TESTS_ONLY
and KMS_LOCALSTACK features, three Gradle tasks, a LocalStack container
pinned by digest, PostgreSQL statement logging and a JUnit XML report.
No product code changes.

Add a compose lane, tests/integration/kms_local/run.sh, that runs the
LocalStack modules without Gradle. It replaces
tests/integration/test_kms_encryption_live.py.

A new workflow runs the LocalStack job on pull requests, the real-key
job on pushes to main and on dispatch, and the Aurora job on dispatch
only. Each job fails when no KMS test ran.

Two strict xfails track product defects: fetchmany() without a size
returns no rows on mysql-connector's C extension, and on Aurora
PostgreSQL the plugin's metadata lookup after a writer failover stops
failover_v2 from running.

The real-key jobs need one-time administrator setup: a symmetric key, an
OIDC role trusted only for main and limited to that key, a Decrypt-only
role, and the KMS_KEY_ARN, KMS_REGION, KMS_IT_ROLE_ARN and
KMS_DECRYPT_ONLY_ROLE_ARN repository variables.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant