fix HPU could not handle float16 in attention.py.
#68
by
sywangyi
- opened
No description provided.
@yao-matrix
sywangyi
changed pull request status to
open
mpt model need to be changed in Attention for the following error. https://huggingface.co/mosaicml/mpt-7b/blob/main/attention.py#L49
causal_mask = attn_weight.new_ones(s, s, dtype=torch.float16)
RuntimeError: float16/half is not supported on Gaudi.