"import error: oneflow.cuda.amp.GradScaler is not implemented" and "TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not Tensor" about onediff

[Alokia]

我根据报错信息中的这句话 from transformers import CLIPTextModel, CLIPFeatureExtractor，做了一些尝试

• 执行 import diffusers，也报同样的错误
• 执行 from transformers import CLIPTextModel, CLIPFeatureExtractor 没有报错误
• 执行 from transformers import CLIPTextModel, CLIPFeatureExtractor 后，再次执行 import diffusers 不再报错误，于是我继续尝试 from diffusers import OneFlowStableDiffusionPipeline 也不再报错。
• 当我关闭这个 ipython，开启新的后，import diffusers 继续报错

在导入diffusers之前先导入transformers可以解决报错的问题，但不知道原因

下面附上测试过程：

In [1]: import diffusers

RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

In [1]: import oneflow

In [2]: with oneflow.mock_torch.enable():
   ...:     from transformers import CLIPTextModel, CLIPFeatureExtractor

RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

In [1]: from transformers import CLIPTextModel, CLIPFeatureExtractor

In [2]: import oneflow

In [3]: with oneflow.mock_torch.enable():
   ...:     from transformers import CLIPTextModel, CLIPFeatureExtractor
   ...: 

In [1]: from transformers import CLIPTextModel, CLIPFeatureExtractor

In [2]: import diffusers

In [3]: from diffusers import OneFlowStableDiffusionPipeline

[jackalcooper]

python3 -m pip uninstall accelerate

[Alokia]

solved : )

[strint]

可以先安装下 accelerate，复现问题；

然后改下：
diffusers/src/diffusers/utils/import_utils.py

def is_accelerate_available():
    return _accelerate_available

注释下这里，让该函数返回 false，看是否能解决问题。如果可以的话，把改动可以提交到 oneflow 的 diffusers 仓库。

@Alokia

[Alokia]

更改后，重新执行 python3 -m pip install -e .[oneflow]，导入diffuser依旧报错

In [1]: import diffusers
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1110, in _LazyModule._get_module(self, module_name)
   1109 try:
-> 1110     return importlib.import_module("." + module_name, self.__name__)
   1111 except Exception as e:

File ~/miniconda3/lib/python3.10/importlib/__init__.py:126, in import_module(name, package)
    125         level += 1
--> 126 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1050, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1027, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:1006, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:688, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:883, in exec_module(self, module)

File <frozen importlib._bootstrap>:241, in _call_with_frames_removed(f, *args, **kwds)

File ~/miniconda3/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py:27
     26 from ...modeling_outputs import BaseModelOutput, BaseModelOutputWithPooling
---> 27 from ...modeling_utils import PreTrainedModel
     28 from ...utils import (
     29     ModelOutput,
     30     add_start_docstrings,
   (...)
     33     replace_return_docstrings,
     34 )

File ~/miniconda3/lib/python3.10/site-packages/transformers/modeling_utils.py:83
     82 if is_accelerate_available():
---> 83     from accelerate import __version__ as accelerate_version
     84     from accelerate import dispatch_model, infer_auto_device_map, init_empty_weights

File ~/miniconda3/lib/python3.10/site-packages/accelerate/__init__.py:7
      5 __version__ = "0.16.0"
----> 7 from .accelerator import Accelerator
      8 from .big_modeling import (
      9     cpu_offload,
     10     disk_offload,
   (...)
     14     load_checkpoint_and_dispatch,
     15 )

File ~/miniconda3/lib/python3.10/site-packages/accelerate/accelerator.py:29
     27 import torch.utils.hooks as hooks
---> 29 from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
     30 from .data_loader import DataLoaderDispatcher, prepare_data_loader, skip_first_batches

File ~/miniconda3/lib/python3.10/site-packages/accelerate/checkpointing.py:22
     21 import torch
---> 22 from torch.cuda.amp import GradScaler
     24 from .utils import (
     25     MODEL_NAME,
     26     OPTIMIZER_NAME,
   (...)
     32     save,
     33 )

File ~/oneflow/python/oneflow/mock_torch/__init__.py:44, in ModuleWrapper.__getattr__(self, name)
     43         return [attr for attr in dir(self.module) if not attr.startswith("_")]
---> 44     raise ModuleNotFoundError(self.module.__name__ + "." + name + error_msg)
     45 attr = getattr(self.module, name)

ModuleNotFoundError: oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
Cell In[1], line 1
----> 1 import diffusers

File ~/diffusers/src/diffusers/__init__.py:22
     20 import oneflow
     21 with oneflow.mock_torch.enable():
---> 22     from transformers import CLIPTextModel, CLIPFeatureExtractor
     23 from .modeling_utils import ModelMixin
     24 from .models import AutoencoderKL, Transformer2DModel, UNet1DModel, UNet2DConditionModel, UNet2DModel, VQModel

File <frozen importlib._bootstrap>:1075, in _handle_fromlist(module, fromlist, import_, recursive)

File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1101, in _LazyModule.__getattr__(self, name)
   1099 elif name in self._class_to_module.keys():
   1100     module = self._get_module(self._class_to_module[name])
-> 1101     value = getattr(module, name)
   1102 else:
   1103     raise AttributeError(f"module {self.__name__} has no attribute {name}")

File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1100, in _LazyModule.__getattr__(self, name)
   1098     value = self._get_module(name)
   1099 elif name in self._class_to_module.keys():
-> 1100     module = self._get_module(self._class_to_module[name])
   1101     value = getattr(module, name)
   1102 else:

File ~/miniconda3/lib/python3.10/site-packages/transformers/utils/import_utils.py:1112, in _LazyModule._get_module(self, module_name)
   1110     return importlib.import_module("." + module_name, self.__name__)
   1111 except Exception as e:
-> 1112     raise RuntimeError(
   1113         f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its"
   1114         f" traceback):\n{e}"
   1115     ) from e

RuntimeError: Failed to import transformers.models.clip.modeling_clip because of the following error (look up to see its traceback):
oneflow.cuda.amp.GradScaler is not implemented, please submit an issue at  
'https://github.com/Oneflow-Inc/oneflow/issues' including the log information of the error, the 
minimum reproduction code, and the system information.

[strint]

File ~/miniconda3/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py:27
     26 from ...modeling_outputs import BaseModelOutput, BaseModelOutputWithPooling
---> 27 from ...modeling_utils import PreTrainedModel
     28 from ...utils import (
     29     ModelOutput,
     30     add_start_docstrings,
   (...)
     33     replace_return_docstrings,
     34 )

File ~/miniconda3/lib/python3.10/site-packages/transformers/modeling_utils.py:83
     82 if is_accelerate_available():
---> 83     from accelerate import __version__ as accelerate_version
     84     from accelerate import dispatch_model, infer_auto_device_map, init_empty_weights

尴尬了，这个路径是 transformers 库里面的，我们现在让用户自己安装的官方 transformers，这样就改不了它的逻辑了。

那就还是沿用上面的方法吧：

python3 -m pip uninstall accelerate