RuntimeError: CUDA error: an illegal memory access was encountered CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. 這個(gè)問(wèn)題困擾了我很久，而且僅僅通過(guò)提供的報(bào)錯(cuò)信息無(wú)法查找到錯(cuò)誤的根源。我通過(guò)不斷地嘗試，定位到了問(wèn)題可能出現(xiàn)的地方。 ? ? ? ? if attn_bias is not None: ? ? ? ? ? ? x = x + attn_bias

加上上述代碼就會(huì)報(bào)錯(cuò)，去掉就正常。 改成這樣就可以解決問(wèn)題了。 ? ? ? ? if attn_bias is not None: ? ? ? ? ? ? x = x + attn_bias.detach()