Draft: Direct tables

architecture/client-architecture.mdx

@@ -36,6 +36,7 @@ The PowerSync Client SDK automatically maintains the following tables in the SQL
 | `ps_data__<table>` | This contains the data for each "table", in JSON format. Since JSON is being used, this table's schema does not change when columns are added, removed or changed in the Sync Streams/Rules and client-side schema. |
 | `ps_data_local__<table>` | Same as the previous point, but for [local-only](/client-sdks/advanced/local-only-usage) tables. |
 | `<table>` (`VIEW`) | These are views on the above `ps_data` tables, with each defined column in the client-side schema extracted from the JSON. For example, a `description` text column would be `CAST(data ->> '$.description' as TEXT)`. |
+| `<table>` (`TABLE`) | For [direct tables](/client-sdks/advanced/performance-optimization#direct-tables), PowerSync replaces the `ps_data__` storage table and view with a single table matching your schema. |
 | `ps_untyped` | Any synced table that is not defined in the client-side schema is placed here. If the table is added to the schema at a later point, the data is then migrated to `ps_data__<table>`. |
 | `ps_oplog` | This is the operation history data as received from the [PowerSync Service](/architecture/powersync-service), grouped per bucket. |
 | `ps_crud` | The client-side upload queue (see [Writing Data](#writing-data-via-sqlite-database-and-upload-queue) below) |

client-sdks/advanced/performance-optimization.mdx

@@ -6,13 +6,13 @@
 PowerSync client SDKs use SQLite, and apply reasonable defaults to optimize performance:
 
 1. For all native targets, [write-ahead logging](https://www.sqlite.org/wal.html) (WAL) is enabled to support concurrent reads and writes. PowerSync also dispatches queries to multiple threads for parallelism. With the JavaScript web SDK, a WAL-like option is [available on Chromium browsers](/client-sdks/reference/javascript-web#2-opfs-based-alternatives).
 2. SDKs also configure connections for performance by enabling a large page cache of around 50MB and setting `pragma synchronous = normal` to avoid frequent fsync operations.
 
 ## Debugging Query Performance
 
 Slow queries can have two causes:
 
 1. The query itself is slow, e.g. because it reads a lot of rows or uses unoptimized joins not backed by an index. The [Fixing Slow Queries](#fixing-slow-queries) section describes how these queries can be optimized.
 2. The database is blocked. There can only be a single writer at a time, and the number of concurrent readers is bound by an option set when opening the database. So a query that appears slow might actually be fast, having run at an unfortunate time when the database was busy.
 
 To understand the problem, having a trace of query runtimes available helps.
@@ -50,6 +50,73 @@
 
 For contention on the write connection, note that the PowerSync client processes changes from the PowerSync Service in a single write transaction after they've been downloaded. Due to consistency requirements, this transaction cannot be split into multiple steps, and especially for a large initial sync it can hold a write lock for several seconds.
 
+## Direct Tables
+
+<Note>
+  **Availability**
+
+  Direct tables are experimental. They were introduced in the following versions of our client SDKs:
+
+  - **JavaScript** (React Native: `2.3.4`, Web: `2.6.0`, Node.js: `1.2.1`)
+  - **Dart**: Version `2.6.0`.
+  - **Kotlin**: Version `1.16.0`.
+  - **Swift**: Version `1.17.0`.
+  - **.NET**: Not supported yet.
+  - **Rust**: Version `0.1.2`.
+</Note>
+
+By default, PowerSync stores all data as [JSON rows](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure): synced rows are copied into local tables without modifications, and local updates modify an existing JSON object. This format is highly flexible and avoids schema migrations, but comes with a performance cost:
+
+- Every column read forces SQLite to decode the entire JSON object, making queries slower than queries on "real" SQLite tables.
+- JSON objects occupy more storage space than a native SQLite row, making databases slightly larger and increasing writes during large sync downloads.
+
+[Raw tables](/client-sdks/advanced/raw-tables) fix this by letting PowerSync sync into actual tables you define. With raw tables, you are responsible for creating and migrating them. Direct tables are raw tables that PowerSync manages for you, like the default JSON-based tables. In supported SDKs, you enable direct tables with a flag on the table definition:
+
+<CodeGroup>
+
+```typescript TypeScript
+const schema = new Schema({
+  users: new Table({ name: column.text }, { direct: true })
+});
+```
+
+```dart Dart/Flutter
+const schema = Schema([
+  Table.direct('users', [Column.text('name')])
+]);
+```
+
+```kotlin Kotlin
+val schema = Schema(
+  Table("users", listOf(Column.text("name")), direct = true)
+)
+```
+
+```swift Swift
+let schema = Schema(
+  Table(name: "users", columns: [.text("name")], direct: true)
+)
+```
+
+```rust Rust
+let mut schema = Schema::default();
+schema
+  .tables
+  .push(Table::create("users", vec![Column::text("name")], |tbl| {
+    tbl.direct = true
+  }));
+```
+
+</CodeGroup>
+
+Direct tables aren't based on views, so you can't set the `viewName` option on them. In the JavaScript SDKs, [High Performance Diffs](/client-sdks/high-performance-diffs) don't support direct tables yet.
+
+### Migrations
+
+To switch a table between the JSON-based format and a direct table, change the flag in your schema. PowerSync SDKs then migrate the existing data automatically. Schema changes on direct tables can take longer than on JSON-based tables: PowerSync must migrate the table itself, whereas JSON-based tables only need their views re-created.
+
+When you add or remove columns on an existing direct table, PowerSync detects the change and runs `ALTER TABLE` statements automatically. New columns contain `NULL` values until the next completed sync.
+
 ## Fixing Slow Queries
 
 If a query itself is expensive and takes a long time to run, several options can improve performance. Some of these require a restructuring of your app's schema.

client-sdks/advanced/raw-tables.mdx

@@ -4,10 +4,6 @@ description: "Access raw SQLite tables to bypass JSON view limitations for nativ
 sidebarTitle: "Raw SQLite Tables"
 ---
 
-<Warning>
-  Raw tables are not currently compatible with the [High Performance Diffs](/client-sdks/high-performance-diffs) feature available in our JavaScript SDKs. We have a [public proposal](https://docs.google.com/document/d/12bvZDJF2aaTOqkxHyo3W-TYGQTaS9FQFdu-MX2yOuqI/edit) open to discuss this addition - your feedback is welcome!
-</Warning>
-
 By default, PowerSync uses a [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) where data is stored schemalessly in JSON format and then presented through SQLite views based on the client-side schema. Raw tables allow you to define native SQLite tables in the client-side schema, bypassing this.
 
 This eliminates overhead associated with extracting values from the JSON data and provides access to advanced SQLite features like foreign key constraints and custom indexes.
@@ -32,7 +28,9 @@ Consider raw tables when you need:
 - **Indexes** - PowerSync's default schema has basic support for indexes on columns, while raw tables give you complete control to create indexes on expressions, use `GENERATED` columns, etc.
 - **Improved performance** for complex queries (e.g., `SELECT SUM(value) FROM transactions`) - raw tables more efficiently get these values directly from the SQLite column, instead of extracting the value from the JSON object on every row.
 - **Reduced storage overhead** - eliminate JSON object overhead for each row in `ps_data__<table>.data` column.
-- **To manually create tables** - Sometimes you need full control over table creation, for example when implementing custom triggers.
+- **To manually create tables** - Sometimes you need full control over table creation, for example when implementing custom triggers or if you use an existing library like [Room](/client-sdks/orms/kotlin/room) managing the schema for you.
+
+If you only need raw tables for their performance and reduced storage overhead, [direct tables](/client-sdks/advanced/performance-optimization#direct-tables) (experimental) may be a better option. They use the same storage format as raw tables, but PowerSync creates and migrates them for you.
 
 <Warning>
   **Advanced SQLite features** like `FOREIGN KEY` and `ON DELETE CASCADE` constraints need [special consideration](#using-foreign-keys).

client-sdks/cascading-delete.mdx

@@ -7,9 +7,9 @@ keywords:
   - "delete"
 ---
 
-PowerSync [uses SQLite views](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) instead of standard tables, so SQLite features like foreign key constraints and cascading deletes are not available.
+By default, PowerSync [uses SQLite views](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) instead of standard tables, so SQLite features like foreign key constraints and cascading deletes are not available.
 
-There's no built-in support for cascading deletes on the client, but you can achieve this in two ways:
+There's no built-in support for cascading deletes on the client, but you can achieve this in three ways:
 
 1. Manual deletion in a transaction - Delete all related records in a single transaction (recommended for most cases)
    <Note>
@@ -19,6 +19,7 @@ There's no built-in support for cascading deletes on the client, but you can ach
    <Note>
      You create triggers on the internal tables (not the views defined by the client schema), similar to what is done [here](https://github.com/powersync-ja/powersync-js/blob/e77b1abfbed91988de1f4c707c24855cd66b2219/demos/react-supabase-todolist/src/app/utils/fts_setup.ts#L50).
    </Note>
+3. Raw tables - PowerSync can sync into existing tables defined by your app, and those tables can use [foreign key constraints](/client-sdks/advanced/raw-tables#using-foreign-keys).
 
 ## Example: Manual Transaction
 

client-sdks/high-performance-diffs.mdx

@@ -22,6 +22,8 @@ While [basic/incremental watch queries](/client-sdks/watch-queries) enable react
   - API design and developer experience
   - Additional features or optimizations needed
 
+  The API does not support [raw](/client-sdks/advanced/raw-tables) or [direct](/client-sdks/advanced/performance-optimization#direct-tables) tables yet. We have a [public proposal](https://docs.google.com/document/d/12bvZDJF2aaTOqkxHyo3W-TYGQTaS9FQFdu-MX2yOuqI/edit) open to discuss this addition - your feedback is welcome!
+
   Join our [Discord community](https://discord.gg/powersync) to share your experience and get help.
 </Warning>