Skip to content

[AMORO-4498][Docs] Document table format capabilities and catalog setup - #4515

Open
hutiefang76 wants to merge 1 commit into
apache:masterfrom
hutiefang76:frank/docs-amoro4498-table-formats-20261113
Open

hutiefang76 wants to merge 1 commit into
apache:masterfrom
hutiefang76:frank/docs-amoro4498-table-formats-20261113

Conversation

@hutiefang76

Copy link
Copy Markdown
Contributor

Why are the changes needed?

Fixes #4498.

The overview lists only four formats and lacks a capability matrix. Hudi and Lance users need catalog properties and the boundaries between metadata browsing, native-format compaction, AMS self-optimizing and AMS maintenance.

This documents capabilities available in the current development version. Screenshots remain coordinated with #4497; existing catalog-management documentation is linked rather than rewritten.

Brief change log

  • Cover all six formats in the overview, with links and a matrix distinguishing metadata, self-optimizing and format-specific AMS maintenance; qualify Iceberg V1/V2 and V3 limits.
  • Add Hudi and Lance guides with five plain Catalog Properties examples, the separate metastore-selection/configuration-upload steps, a Hudi Hive XML example, existing dataset layouts and source-build prerequisites.
  • Document current limits, including native compaction information versus AMS scheduling, Lance filesystem databases, Hive warehouse/Kerberos restrictions and the Lance Namespace REST requirement.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible
  • Add screenshots for manual tests if appropriate
  • Run test locally before making a pull request

Documentation-only change; Java tests and new UI screenshots are not needed.

  • Built both sites with official Hugo Extended 0.169.0 (darwin/arm64), before and after the patch on base ffc2b713bf03ad642c569e8bad54f958a95b81fe. The sources and combined-site base URLs follow site/README.md: site/amoro-site at http://localhost:5500/, and site/amoro-docs at http://localhost:5500/docs/latest/. Each build used fresh external output/resource/cache directories.
  • Audited 82 generated HTML pages and 4,365 references. The one existing catalog-link finding is unchanged, with zero new findings. The 892 external, historical-version and special references were explicitly left unverified.
  • Served generated output over actual local HTTP: verified nine link/anchor pairs, all six matrix rows, five rendered JSON property maps, one Hive XML example and both new aliases. Historical-version URL sets are unchanged on the previous 78 pages; the temporary server was stopped.
  • Checked capability and configuration literals against current catalog/format/maintenance source. git diff --check passes.

The examples were checked against source and rendered JSON/XML; no live Hudi, Lance or Hive connection was executed. This does not deploy the official documentation site.

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable; this documents existing format and catalog behavior.

AI assistance: DeepSeek and OpenAI Codex assisted drafting, source verification, validation tooling and automated review.

Cover six formats and add Hudi/Lance catalog property examples with current
metadata, optimizing and maintenance limits.

AI assistance: DeepSeek and OpenAI Codex.
@github-actions github-actions Bot added the type:docs Improvements or additions to documentation label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:docs Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement]: Update table format documentation

1 participant