Skip to content

Add a Native CPU engine backed by CPUPlan (#23935) - #23935

Merged
meta-codesync[bot] merged 1 commit into
pytorch:mainfrom
JakeStevens:export-D125540986
Oct 29, 2026
Merged

meta-codesync[bot] merged 1 commit into
pytorch:mainfrom
JakeStevens:export-D125540986

Conversation

@JakeStevens

@JakeStevens JakeStevens commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary:

Add CPUEngineHost and CPUEngineContext using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with Model/Session, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. current_execution_context moves from CPUBackend.cpp into cpu_runtime so CPUBackend and the engine read the threadpool the same way, and the unused ExecutionContext::avx2_fma flag, which no provider reads, is removed. tensor_bytes no longer checks dtypes; CPUBackend keeps its own check for the FP32, integer, and boolean dtypes it supports.

Alias handling lives in the shared functionalize_aliases pass (runtime/graph/utils/Aliases.h), which also hosts the alias_root/has_mutated_alias_use helpers that Vulkan's MaterializeViewCopies now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through views alone with the same shape, so the decision does not depend on activation dim-order hints. as_strided accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in aliases_test.

Add CMake libraries and installed targets for the CPU runtime, providers, and engine. EXECUTORCH_BUILD_NATIVE_CPU requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through executorch_shared. Include the CPU engine integration suite in CTest.

Authored with OpenAI Codex and Claude Code.

Reviewed By: digantdesai

Differential Revision: D125540986

@pytorch-bot

pytorch-bot Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23935

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit 3475bdd with merge base a12abb9 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 16, 2026
@meta-codesync

meta-codesync Bot commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

@JakeStevens has exported this pull request. If you are a Meta employee, you can view the originating Diff in D125540986.

@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

JakeStevens added a commit to JakeStevens/executorch that referenced this pull request Oct 26, 2026
Summary:
Pull Request resolved: pytorch#23935

Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics.

Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation.

Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest.

Authored with OpenAI Codex and Claude Code.

Differential Revision: D125540986

@digantdesai digantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

JakeStevens added a commit to JakeStevens/executorch that referenced this pull request Oct 28, 2026
Summary:
Pull Request resolved: pytorch#23935

Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports.

Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`.

Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest.

Authored with OpenAI Codex and Claude Code.

Reviewed By: digantdesai

Differential Revision: D125540986
@meta-codesync meta-codesync Bot changed the title Add a Native CPU engine backed by CPUPlan Add a Native CPU engine backed by CPUPlan (#23935) Oct 28, 2026
JakeStevens added a commit to JakeStevens/executorch that referenced this pull request Oct 28, 2026
Summary:
Pull Request resolved: pytorch#23935

Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports.

Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`.

Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest.

Authored with OpenAI Codex and Claude Code.

Reviewed By: digantdesai

Differential Revision: D125540986
JakeStevens added a commit to JakeStevens/executorch that referenced this pull request Oct 28, 2026
Summary:
Pull Request resolved: pytorch#23935

Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports.

Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`.

Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest.

Authored with OpenAI Codex and Claude Code.

Reviewed By: digantdesai

Differential Revision: D125540986
JakeStevens added a commit to JakeStevens/executorch that referenced this pull request Oct 29, 2026
Summary:
Pull Request resolved: pytorch#23935

Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports.

Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`.

Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest.

Authored with OpenAI Codex and Claude Code.

Reviewed By: digantdesai

Differential Revision: D125540986
Summary:
Pull Request resolved: pytorch#23935

Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports.

Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`.

Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest.

Authored with OpenAI Codex and Claude Code.

Reviewed By: digantdesai

Differential Revision: D125540986
@meta-codesync
meta-codesync Bot merged commit 1d55f9c into pytorch:main Oct 29, 2026
254 of 255 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants