IBM brings its Spyre accelerator into PyTorch workflows

IBM says its Spyre AI accelerator now works as a PyTorch backend, allowing developers to use familiar model code on the device. A Spyre plugin for vLLM also connects it to llm-d, which schedules distributed inference across accelerator types.

Spyre is an inference accelerator designed for teams running AI alongside applications and data on IBM Z, LinuxONE and Power systems. IBM and the PyTorch Foundation outlined plans for deeper Spyre support in October 2025. IBM says the latest software updates put those plans into practice.

PyTorch code compiles for Spyre

The torch-spyre backend turns ordinary PyTorch code into an executable plan for Spyre. IBM says most model logic passes through the PyTorch compiler unchanged. For operations the stack cannot yet compile effectively, an adapter can substitute a mathematically equivalent form.

IBM has also moved the backend to PyTorch’s Inductor compiler. It says the resulting compiler is roughly seven times smaller than the system it replaced. In a demonstration using Granite 3.3 8B at batch size 1 and sequence length 1024, changes to graph construction and compute dispatch made prefill 1.7 times faster and decode 2.4 times faster.

The vLLM plugin extends model support

IBM says the Spyre plugin uses vLLM extension points to adapt the software for its hardware. Work on hardware-agnostic layers in vLLM aims to keep model implementations portable as other paths become more specialized. Early support has landed in vLLM, and IBM says it has validated the approach with the Spyre plugin on Gemma 4, Qwen3 and Granite 4.2.

In tests on NVIDIA H100 GPUs, those layers delivered total token throughput within 3.4% of native implementations, measured as the geometric mean across three recent models. IBM also says its engineers are using end-to-end model tests and continuous integration to catch compatibility problems caused by changes to upstream frameworks.

According to IBM, torch-spyre supplies the PyTorch backend that lets the vLLM plugin expose Spyre to llm-d. IBM, Red Hat and collaborators in the PyTorch Foundation, including Meta and Hugging Face, pushed the software ecosystem updates in the second half of 2026.