Merge pull request #79 from gmr/feature/new-object-models

Model EXCLUDE constraints, replica identity and default privileges

Author: gmr

PLAN-coverage.md

@@ -1,6 +1,7 @@
 # Coverage plan: silent schema loss, RLS, and PostgreSQL 18
 
-Status: Phases 0 to 5 complete. Phases 6-8 proposed.
+Status: Phases 0 to 5 complete, and Phase 6 items 1 to 3. The rest of
+Phase 6, and Phases 7 and 8, proposed.
 Written 2026-09-21.
 
 Every claim below was verified against PostgreSQL 18.4 (the version
@@ -547,7 +548,37 @@ enabled state.
 → verify: pull a database containing all seven; `remaining.yaml` is
 absent, and `just round-trip` passes with them in `fixtures/schema.sql`.
 
-### Phase 6 — New object models (~4 days, ranked)
+### Phase 6 — New object models (~4 days, ranked) — items 1 to 3 **DONE**
+
+**Done:** `EXCLUDE` constraints (item 1), `REPLICA IDENTITY` (item 2)
+and default privileges (item 3), each pulled, built, deployed and in
+`fixtures/schema.sql`. The coverage gate list is down from 5 entries
+to 4. Items 4 to 7 are still open.
+
+Where the work departs from the items below:
+
+- **An unmatched comment now fails the pull.** Pull logged a warning
+  for a comment it could not attach and dropped it, which the
+  governing invariant forbids. It now keeps the entry in
+  `remaining.yaml`. The model has a comment for an exclusion
+  constraint only, so a comment on a primary key, unique, check or
+  foreign key constraint now fails the pull, and the coverage fixture
+  records that gap.
+- **Default privileges are a new top-level object type**,
+  `default_privileges/<role>.yaml`, not part of `src/build/acls.rs`:
+  they are keyed by the role the defaults are `FOR`, and the role files
+  are not written for a pull from `--dump`. Deploy matches them by
+  existence, like the Phase 5 types. `REVOKE GRANT OPTION FOR` does not
+  parse into the model; pg_dump folds a grant and a revoke of its
+  option into a plain grant, so the fixture could not produce one.
+- **`USING INDEX` is written with its index.** pg_dump puts `REPLICA
+  IDENTITY USING INDEX` in the index's entry, and build does the same,
+  since the index has to exist first.
+- **Deviation 27.** The shared index column renderer now writes
+  `COLLATE`, where the Python wrote `COLLATION`, and quotes column
+  names.
+
+The original items follow.
 
 Do now, in this order:
 

bin/deploy-gates

@@ -97,6 +97,8 @@ echo "==> Gate 2: convergence (mutate, deploy --apply, re-deploy is empty)"
 # NOT NULL there outright
 # row-security drift: turned off and unforced, one policy dropped, one
 # edited, one database-only, and a changed policy comment
+# replica identity reset and changed, an exclusion constraint's comment
+# changed, and another one dropped
 psql -d "${TARGET_DB}" -q -v ON_ERROR_STOP=1 <<'SQL'
 ALTER TABLE test.users ADD COLUMN scratch text;
 ALTER TABLE test.users ALTER COLUMN display_name SET NOT NULL;
@@ -114,6 +116,11 @@ ALTER POLICY tenant_notes_own ON test.tenant_notes USING (true);
 CREATE POLICY stray ON test.tenant_notes AS RESTRICTIVE USING (true);
 COMMENT ON POLICY tenant_notes_own ON test.tenant_notes IS 'drift';
 ALTER TABLE test.sealed_notes DISABLE ROW LEVEL SECURITY;
+ALTER TABLE test.replica_full REPLICA IDENTITY DEFAULT;
+ALTER TABLE test.replica_index REPLICA IDENTITY FULL;
+COMMENT ON CONSTRAINT room_bookings_no_overlap ON test.room_bookings
+    IS 'drift';
+ALTER TABLE test.handles DROP CONSTRAINT handles_unique_lower;
 SQL
 ./target/debug/pglifecycle deploy --apply --allow-drop -d "${TARGET_DB}" \
     "${WORKDIR}/project"

bin/generate-schema-docs.py

@@ -63,7 +63,14 @@
         'type',
         'view',
     ],
-    'Roles and privileges': ['acls', 'group', 'role', 'user', 'user_mapping'],
+    'Roles and privileges': [
+        'acls',
+        'default_privileges',
+        'group',
+        'role',
+        'user',
+        'user_mapping',
+    ],
     'Shared definitions': [
         'argument',
         'column',

docs/commands.md

@@ -80,8 +80,10 @@ Objects that differ between the project and the database are
 reconciled in place where PostgreSQL can express it:
 
 - **Tables** — add column, set/drop default, set/drop not-null,
-  add/drop check constraints and foreign keys, primary-key and unique
-  additions, index and trigger create/drop, and comment changes.
+  add/drop check constraints, foreign keys and exclusion constraints,
+  primary-key and unique additions, index and trigger create/drop,
+  `REPLICA IDENTITY`, and comment changes. A changed comment on an
+  exclusion constraint is set alone, without rebuilding its index.
   Identity columns are added, and their `ALWAYS`/`BY DEFAULT` behavior
   and sequence options changed, with `ALTER COLUMN`; renaming an
   identity's sequence falls back. A `NOT VALID` check, foreign key or
@@ -159,7 +161,8 @@ Ownership is not managed (the script behaves like
 `pg_restore --no-owner`), and roles, users, groups, and tablespaces are
 skipped entirely — they are cluster-level objects a single-database
 dump cannot capture. Aggregates, casts, collations, conversions, event
-triggers, publications and text search objects are created when
+triggers, publications, text search objects and default privileges are
+created when
 missing but otherwise only existence-checked: `pull` models them, but
 `deploy` does not compare their definitions yet, so a changed one is
 left as the database has it. An aggregate is matched by its name and
@@ -219,6 +222,9 @@ The entries were preserved in ./project/remaining.yaml; re-run with
 --allow-unsupported to accept the project as it is.
 ```
 
+A `COMMENT` entry counts as unmodeled when the model has no place for
+it, such as a comment on a primary key or foreign key constraint.
+
 The project directory is written either way, so `remaining.yaml` is
 there to inspect. `--allow-unsupported` downgrades the failure to a
 warning for the cases where an incomplete project is what you want.

docs/project-format.md

@@ -164,6 +164,45 @@ policies:
   managed: `deploy` leaves its row security and policies as the
   database has them. Pull the project again to record them.
 
+- An exclusion constraint lists each element as an index column (a
+  `name` or an `expression`, with its `collation`, `opclass`,
+  `direction` and `null_placement`) and the `operator` two rows are
+  compared with. `replica_identity` is `FULL`, `NOTHING`, or
+  `{index: name}`; absent is `DEFAULT`, the primary key.
+
+```yaml
+exclude_constraints:
+  - name: room_bookings_no_overlap
+    method: gist
+    elements:
+      - name: room
+        operator: =
+      - name: during
+        operator: '&&'
+    where: (status <> 'cancelled'::text)
+replica_identity: FULL
+```
+
+- Default privileges live in `default_privileges/<role>.yaml`, one file
+  for each role whose new objects they apply to (`ALTER DEFAULT
+  PRIVILEGES FOR ROLE`). Global and per-schema declarations compose in
+  PostgreSQL, so each is kept as written; `build` emits the
+  revocations first, then the grants, as `pg_dump` does.
+
+```yaml
+---
+name: app_owner
+grants:
+  - schema: reporting
+    object_type: TABLES
+    grantee: analyst
+    privileges: [SELECT]
+revocations:
+  - object_type: FUNCTIONS
+    grantee: PUBLIC
+    privileges: [EXECUTE]
+```
+
 - A cast has no schema of its own. `pull` files it in the
   `casts/<schema>.yaml` of the first schema its function or types name,
   or `public` when they are all built-in.

fixtures/schema.sql

@@ -484,3 +484,44 @@ ALTER EVENT TRIGGER pglifecycle_drops DISABLE;
 CREATE EVENT TRIGGER pglifecycle_replica ON ddl_command_end
     EXECUTE FUNCTION test.note_ddl();
 ALTER EVENT TRIGGER pglifecycle_replica ENABLE REPLICA;
+
+-- Exclusion constraints: gist over a range, with a predicate and a
+-- comment, and btree over an expression with its operator class and
+-- order, INCLUDE and deferral. btree_gist, created above, provides =
+-- for integers in gist.
+
+CREATE TABLE test.room_bookings (
+    room   INTEGER,
+    during TSRANGE,
+    status TEXT,
+    CONSTRAINT room_bookings_no_overlap
+        EXCLUDE USING gist (room WITH =, during WITH &&)
+        WHERE (status <> 'cancelled')
+);
+COMMENT ON CONSTRAINT room_bookings_no_overlap ON test.room_bookings IS
+    'One booking per room at a time';
+
+CREATE TABLE test.handles (
+    handle TEXT,
+    owner  INTEGER,
+    CONSTRAINT handles_unique_lower
+        EXCLUDE USING btree (lower(handle) text_pattern_ops DESC NULLS LAST WITH =)
+        INCLUDE (owner) DEFERRABLE INITIALLY DEFERRED
+);
+
+-- Replica identity: the whole row, none, and a unique index. pg_dump
+-- writes the index form in the index's own entry.
+CREATE TABLE test.replica_full (id INTEGER);
+ALTER TABLE test.replica_full REPLICA IDENTITY FULL;
+CREATE TABLE test.replica_nothing (id INTEGER);
+ALTER TABLE test.replica_nothing REPLICA IDENTITY NOTHING;
+CREATE TABLE test.replica_index (id INTEGER NOT NULL);
+CREATE UNIQUE INDEX replica_index_id ON test.replica_index (id);
+ALTER TABLE test.replica_index REPLICA IDENTITY USING INDEX replica_index_id;
+
+-- Default privileges, global and per schema. PUBLIC as the grantee
+-- keeps the fixture free of a cluster-wide role. These come last, so
+-- the objects above are created under the built-in defaults.
+ALTER DEFAULT PRIVILEGES IN SCHEMA test GRANT SELECT ON TABLES TO PUBLIC;
+ALTER DEFAULT PRIVILEGES IN SCHEMA test GRANT USAGE ON SEQUENCES TO PUBLIC;
+ALTER DEFAULT PRIVILEGES REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;

fixtures/unsupported-descs.txt

@@ -1,5 +1,4 @@
-CONSTRAINT	unsupported.reservations reservations_no_overlap
-DEFAULT ACL	unsupported.DEFAULT PRIVILEGES FOR TABLES
+COMMENT	unsupported.CONSTRAINT payloads_pkey ON payloads
 RULE	unsupported.append_only append_only_no_delete
 STATISTICS	unsupported.measurements_stats
 TABLE	unsupported.payloads

fixtures/unsupported.sql

@@ -16,28 +16,21 @@
 -- and no provider ships with the standard server, so the statement
 -- fails on the gate's cluster.
 
-CREATE EXTENSION btree_gist;
-
 CREATE SCHEMA unsupported;
 SET search_path = unsupported, public, pg_catalog;
 
 CREATE ROLE pglifecycle_coverage_reader;
 
--- Exclusion constraint
-CREATE TABLE reservations (
-    room   INT,
-    during DATERANGE,
-    CONSTRAINT reservations_no_overlap
-        EXCLUDE USING gist (room WITH =, during WITH &&)
-);
-
--- Column storage and compression, and the table's replica identity
+-- Column storage and compression
 CREATE TABLE payloads (
     id   INT PRIMARY KEY,
     body TEXT COMPRESSION lz4
 );
 ALTER TABLE payloads ALTER COLUMN body SET STORAGE EXTERNAL;
-ALTER TABLE payloads REPLICA IDENTITY FULL;
+
+-- A comment on a primary key, unique, check or foreign key constraint:
+-- only an exclusion constraint carries one in the model
+COMMENT ON CONSTRAINT payloads_pkey ON payloads IS 'The payload id';
 
 -- Extended statistics
 CREATE TABLE measurements (a INT, b INT);
@@ -48,7 +41,3 @@ CREATE STATISTICS measurements_stats (ndistinct, dependencies)
 CREATE TABLE append_only (id INT);
 CREATE RULE append_only_no_delete AS
     ON DELETE TO append_only DO INSTEAD NOTHING;
-
--- Default privileges
-ALTER DEFAULT PRIVILEGES IN SCHEMA unsupported
-    GRANT SELECT ON TABLES TO pglifecycle_coverage_reader;

schemata/default_privileges.yml

@@ -0,0 +1,69 @@
+---
+$schema: https://json-schema.org/draft/2020-12/schema
+$id: https://gmr.github.io/pglifecycle/schemata/default_privileges.json
+title: Default Privileges
+description: >
+  The privileges objects a role creates start with (ALTER DEFAULT
+  PRIVILEGES FOR ROLE). Global and per-schema declarations compose, so
+  they are kept as written.
+$defs:
+  declaration:
+    type: object
+    properties:
+      schema:
+        title: Schema
+        description: The schema the declaration is limited to; absent for every schema
+        type: string
+      object_type:
+        title: Object Type
+        enum: [TABLES, SEQUENCES, FUNCTIONS, ROUTINES, TYPES, SCHEMAS, LARGE OBJECTS]
+      grantee:
+        title: Grantee
+        description: The role given or denied the privileges, or PUBLIC
+        type: string
+      privileges:
+        title: Privileges
+        type: array
+        items:
+          type: string
+        minItems: 1
+      with_grant_option:
+        title: With Grant Option
+        description: PostgreSQL does not give grant options to PUBLIC
+        type: boolean
+    required: [object_type, grantee, privileges]
+    additionalProperties: false
+    not:
+      properties:
+        grantee:
+          pattern: '^[Pp][Uu][Bb][Ll][Ii][Cc]$'
+        with_grant_option:
+          const: true
+      required: [grantee, with_grant_option]
+type: object
+properties:
+  name:
+    title: Role
+    description: The role whose new objects the defaults apply to
+    type: string
+  grants:
+    title: Grants
+    type: array
+    items:
+      $ref: '#/$defs/declaration'
+  revocations:
+    title: Revocations
+    description: >
+      Privileges taken away from the built-in defaults, such as EXECUTE on
+      functions from PUBLIC. REVOKE GRANT OPTION FOR is not supported, so
+      a revocation cannot set with_grant_option to true.
+    type: array
+    items:
+      $ref: '#/$defs/declaration'
+      not:
+        properties:
+          with_grant_option:
+            const: true
+        required: [with_grant_option]
+required: [name]
+additionalProperties: false