Repository navigation
Add a Native CPU engine backed by CPUPlan (#23935) - #23935
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23935
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit 3475bdd with merge base a12abb9 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@JakeStevens has exported this pull request. If you are a Meta employee, you can view the originating Diff in D125540986. |
This PR needs a
|
Summary: Pull Request resolved: pytorch#23935 Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest. Authored with OpenAI Codex and Claude Code. Differential Revision: D125540986
digantdesai
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
Summary: Pull Request resolved: pytorch#23935 Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports. Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`. Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest. Authored with OpenAI Codex and Claude Code. Reviewed By: digantdesai Differential Revision: D125540986
CPUPlanCPUPlan (#23935)
834180f to
9b6a985
Compare
Summary: Pull Request resolved: pytorch#23935 Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports. Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`. Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest. Authored with OpenAI Codex and Claude Code. Reviewed By: digantdesai Differential Revision: D125540986
Summary: Pull Request resolved: pytorch#23935 Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports. Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`. Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest. Authored with OpenAI Codex and Claude Code. Reviewed By: digantdesai Differential Revision: D125540986
Summary: Pull Request resolved: pytorch#23935 Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports. Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`. Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest. Authored with OpenAI Codex and Claude Code. Reviewed By: digantdesai Differential Revision: D125540986
9b6a985 to
d0f1851
Compare
Summary: Pull Request resolved: pytorch#23935 Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports. Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`. Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest. Authored with OpenAI Codex and Claude Code. Reviewed By: digantdesai Differential Revision: D125540986
d0f1851 to
3475bdd
Compare
Summary:
Add
CPUEngineHostandCPUEngineContextusing the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates withModel/Session, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics.current_execution_contextmoves fromCPUBackend.cppintocpu_runtimesoCPUBackendand the engine read the threadpool the same way, and the unusedExecutionContext::avx2_fmaflag, which no provider reads, is removed.tensor_bytesno longer checks dtypes;CPUBackendkeeps its own check for the FP32, integer, and boolean dtypes it supports.Alias handling lives in the shared
functionalize_aliasespass (runtime/graph/utils/Aliases.h), which also hosts thealias_root/has_mutated_alias_usehelpers that Vulkan'sMaterializeViewCopiesnow uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value throughviews alone with the same shape, so the decision does not depend on activation dim-order hints.as_stridedaccepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests inaliases_test.Add CMake libraries and installed targets for the CPU runtime, providers, and engine.
EXECUTORCH_BUILD_NATIVE_CPUrequires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups throughexecutorch_shared. Include the CPU engine integration suite in CTest.Authored with OpenAI Codex and Claude Code.
Reviewed By: digantdesai
Differential Revision: D125540986