ExecuTorch

ExecuTorch runs PyTorch models on mobile and edge devices. Export your Transformers models to the ExecuTorch format with Optimum ExecuTorch with the command below.

optimum-cli export executorch \
    --model "HuggingFaceTB/SmolLM2-135M-Instruct" \
    --task "text-generation" \
    --recipe "xnnpack" \
    --use_custom_sdpa \
    --use_custom_kv_cache \
    --qlinear 8da4w \
    --qembedding 8w \
    --output_dir="hf_smollm2"

Run optimum-cli export executorch --help to see all export options. For detailed export instructions, check the README.

Update on GitHub