Native Sparse Attention PyTorch PyTorch implementation of Native Sparse Attention. Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention.