Poster Type: Research Posters
Author: Paul Mifsud (Forschungszentrum Jülich), Stefan Kesselheim (Forschungszentrum Jülich)
Abstract: The recent developments in AI have prompted an interest in tensor compilers, but their use is not universal and often they are coupled with hand optimized kernels in real deployments. In this work, we dive deep into tensor compilers and adjacent systems to build a holistic systems understanding of tensor program optimization. We define our own tensor operator taxonomy. Using a modern GPU as an example, we condense GPU kernel optimization into two principles, from which many optimization techniques emerge. Using the PyTorch 2.0 compiler backend, TorchInductor, and the Triton compiler as examples, we explain in detail what intermediate representations and optimization techniques are used during the compilation process. We find that kernel fusion is the most consequential optimization. In the end, we are able to understand the value proposition of tensor compilers, the type of optimizations they apply, their limitations and how they can be used effectively and augmented.
Best Poster Finalist (BP): no
Poster: PDF
Poster Summary: PDF