6.2.4. ExecuTorch¶
Introduction¶
ExecuTorch Overview¶
Executorch是一款端到端解决方案，用于在移动的和边缘设备（包括可穿戴设备、嵌入式设备和微控制器）上实现设备上的推理功能。它是PyTorch Edge生态系统的一部分，可以将PyTorch模型高效部署到边缘设备。
主要价值主张(key value propositions):
- Portability
- Productivity
- Performance
How ExecuTorch Works¶
从高层次上讲，在边缘设备（如笔记本电脑、移动的手机、可穿戴设备和物联网设备）上使用Executorch运行PyTorch模型有三个步骤:
1. Export the model
2. Compile the exported model to an ExecuTorch program
3. Run the ExecuTorch program on a target device
Key Benefits:
1. Export that is robust and powerful
2. Operator standardization
3. Standardization for compiler interfaces (aka delegates) and the OSS ecosystem
4. First-party SDK and toolchain
5. No intermediate conversions necessary
6. Ease of customization
7. Low overhead runtime and execution
High-level Architecture and Components of ExecuTorch¶
将PyTorch模型部署到设备上有三个阶段：程序准备(Program Preparation)，运行时准备(Runtime Preparation)和程序执行(Program Execution)¶
Getting Started¶
Setting Up ExecuTorch¶
install:
pip install executorch
Export a Program:
# ExecuTorch provides APIs to compile a PyTorch nn.Module to a .pte binary
1. torch.export
2. exir.to_edge
3. exir.to_executorch
4. Save the result as a .pte binary to be consumed by the ExecuTorch runtime.
导出代码示例:
import torch
from torch.export import export
from executorch.exir import to_edge
# Start with a PyTorch model that adds two input tensors (matrices)
class Add(torch.nn.Module):
def __init__(self):
super(Add, self).__init__()
def forward(self, x: torch.Tensor, y: torch.Tensor):
return x + y
# 1. torch.export: Defines the program with the ATen operator set.
aten_dialect = export(Add(), (torch.ones(1), torch.ones(1)))
# 2. to_edge: Make optimizations for Edge devices
edge_program = to_edge(aten_dialect)
# 3. to_executorch: Convert the graph to an ExecuTorch program
executorch_program = edge_program.to_executorch()
# 4. Save the compiled .pte program
with open("add.pte", "wb") as file:
file.write(executorch_program.buffer)
Run Your Program:
executor_runner --model_path ./add.pte