No Module Named 'torch'

[MilesQLi]

When I run pip install flash-attn, it says that. But obviously, it is wrong. See screenshot.

[ulysses500]

Same issue for me.

[ulysses500]

Workaround: install the previous version pip install flash_attn==1.0.5

[smeyerhot]

I am seeing the same problem on every flash_attn version. I am using Cuda 12.1 on the new g2 vm instance from gcp. https://cloud.google.com/compute/docs/accelerator-optimized-machines#g2-vms. The underlying GPU is the Nvidia L4 which uses Ada.

[smeyerhot]

Workaround: install the previous version pip install flash_attn==1.0.5

This might work in some scenarios but not all.

[tridao]

Can you try python -m pip install flash-attn?
It's possible that pip and python -m pip refer to different environments.

Getting the dependencies right for all setup is hard. We had torch in the dependency in 1.0.5, but for some users it would download a new version of torch instead of using the existing one. So for 1.0.6 we leave torch out of the dependency.

[official-elinas]

Getting the same issue. I also tried python -m pip install flash-attn as you suggested with the same failure.

[nrailg]

same problem here.

[tridao]

I don't know a right solution that works for all setups, happy to hear suggestions.

We recommend the Pytorch container from Nvidia, which has all the required tools to install FlashAttention.

[smeyerhot]

I believe this is an incompatibility issue with cuda 12.1 version of torch.

Using the following torch version solves my probem.

torch==2.0.0+cu117

[MilesQLi]

@smeyerhot I use the exact version, but it doesn't work. See the screenshot.

[smeyerhot]

@MilesQLi

I believe this is an incompatibility issue with cuda 12.1 version of torch.

Using the following torch version solves my probem.

torch==2.0.0+cu117

Sorry! This didn't fix things... apologies on the false hope.

[MilesQLi]

@smeyerhot No problem. Thanks a lot anyway!

[jzsbioinfo]

same problem

[quant-cracker]

same problem

[leucocyte123]

pip install flash-attn==1.0.5 might help. I am using torch 1.13 and cuda 12.0.