Comments (7)
I have managed to run the below TTS code on XPU, but it takes 23 seconds , seriously, it takes 3 second on CPU :
Please can we check what issue on XPU:
`(tts) spandey2@imu-nex-nuc13x2-arc770-dut:~/tts$ cat andrej_code_tts.py
In case of proxies, remove .intel.com from no proxies:
import os
#os.environ['no_proxy'] = '10.0.0.0/8,192.168.0.0/16,localhost,127.0.0.0/8,134.134.0.0/16'
#os.environ['NO_PROXY'] = '10.0.0.0/8,192.168.0.0/16,localhost,127.0.0.0/8,134.134.0.0/16'
IPEX
#import subprocess
#subprocess.run(["python", "-m", "pip", "install", "torch==2.1.0.post2", "torchvision==0.16.0.post2", "torchaudio==2.1.0.post2",
"intel-extension-for-pytorch==2.1.30+xpu", "oneccl_bind_pt==2.1.300+xpu",
"--extra-index-url", "https://pytorch-extension.intel.com/release-whl/stable/xpu/us/"])
TTS dependency. Do it on TERMINAL:
sudo apt install espeak-ng
let's check also python can see the device
import torch
import intel_extension_for_pytorch as ipex
print(torch.version)
print(ipex.version)
[print(f'[{i}]: {torch.xpu.get_device_properties(i)}') for i in range(torch.xpu.device_count())]
from TTS.utils.manage import ModelManager
from TTS.utils.synthesizer import Synthesizer
#from IPython.display import Audio
import numpy as np
import soundfile as sf
model_manager = ModelManager()
model_path, config_path, model_item = model_manager.download_model("tts_models/en/vctk/vits")
synthesizer = Synthesizer(model_path, config_path, use_cuda=False)
Move the model to GPU and optimize it using IPEX
synthesizer.tts_model.to('xpu')
synthesizer.tts_model.eval() # Set the model to evaluation mode for inference
synthesizer.tts_model = ipex.optimize(synthesizer.tts_model, dtype=torch.float32)
speaker_manager = synthesizer.tts_model.speaker_manager
speaker_names = list(speaker_manager.name_to_id.keys())
print("Available speaker names:", speaker_names)
speaker_name = "p229" # Replace with the actual speaker name you want to use
text = "Your last lap time was 117.547 seconds. That's a bit slower than your best, but you're still doing well. Keep pushing, a really good lap is around 100 seconds. You've got this, let's keep improving."
Move input data to GPU and run inference with autocast for potential mixed precision
with torch.no_grad(), torch.xpu.amp.autocast(enabled=False):
wavs = synthesizer.tts(text, speaker_name=speaker_name)
if isinstance(wavs, list):
# Convert each NumPy array or scalar in the list to a PyTorch tensor
tensor_list = [torch.tensor(wav, dtype=torch.float32).unsqueeze(0) if np.isscalar(wav) else torch.tensor(wav, dtype=torch.float32) for wav in wavs]
# Concatenate the tensor list into a single tensor
wav_concatenated = torch.cat(tensor_list, dim=0)
else:
# If 'wavs' is already a tensor, use it directly
wav_concatenated = wavs
#Move the tensor to CPU and convert to NumPy array
wav_concatenated = wav_concatenated.cpu().numpy()
#Save the output to a WAV file
output_path = "output_vctk_vits.wav"
sf.write(output_path, wav_concatenated, synthesizer.tts_config.audio['sample_rate'])
`
from bigdl.
Seems related to this issue. intel/intel-extension-for-pytorch#317
@gc-fu Can we solve this issue in conda env?
from bigdl.
The issue intel/intel-extension-for-pytorch#317 was caused by self-built torch with pre-built intel-extension-for-pytorch.
I have tried install these two dependencies in conda and then import intel_extension_for_pytorch as ipex
. Everything works fine.
Can you try these through conda?
from bigdl.
Hi @shailesh837
Can you share your OS and kernel version? This error is raised by intel-extension-for-pytorch. In most cases, it's caused by OS or glibc out of date.
from bigdl.
I have ubuntu 22.04 LTS and Linux imu-nex-nuc13x2-arc770-dut 6.5.0-26-generic .
@gc-fu : Did you managed to run the code :
import torch
from TTS.api import TTS
from ipex_llm import optimize_model
#Get device
device = 'xpu' # Use 'xpu' to indicate Intel GPU
#List availableTTS models
print(TTS().list_models())
Init TTS
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v2")
tts = optimize_model(tts, cpu_embedding=True)
tts = tts.to(device)
wav = tts.tts(text="Hello world!", speaker_wav="./audio.wav", language="en").to('cpu') # Move output to CPU if needed
tts.tts_to_file(text="Hello world!", speaker_wav="./audio.wav", language="en", file_path="output.wav")
from bigdl.
After some investigation, the issue was caused not installing the correct torchaudio version.
Install using these instructions:
pip install TTS
pip install --pre --upgrade ipex-llm[xpu] --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
# Then pip install torchaudio
pip install torchaudio==2.1.0a0 --extra-index-url
https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
The code results:
The unknown symbol issue should have been addressed.
The error here should be caused by the fact that ipex-llm not supporting TTS models for now.
from bigdl.
The code you provided is using ipex, not ipex-llm. Please do check again.
The intel-extension-for-pytorch repo address is here:
https://github.com/intel/intel-extension-for-pytorch
Your related code snippet:
import intel_extension_for_pytorch as ipex
synthesizer.tts_model = ipex.optimize(synthesizer.tts_model, dtype=torch.float32)
from bigdl.
Related Issues (20)
- [Feature]internlm-xcomposer2-vl-7b support HOT 2
- Qwen-7B TypeError: qwen_attention_forward() got an unexpected keyword argument 'registered_causal_mask' HOT 2
- ipex-llm(0517) Failed to Run 'baichuan-inc/Baichuan2-7B-Chat' in batch_size==2 and batch_size==4 with 32-32, 1024-128, 2048-256 input_length HOT 1
- Qwen-7B-Chat fail with larger 6.7k for second or 3rd time HOT 2
- Ollama Linux No Response Issue with IPEX-LLM HOT 2
- Qwen1.5-4b and Qwen1.5-7b model cannot be loaded correctly in ipex-llm version 20240522 HOT 9
- [inference]: fine tuned model fails to do inferencing HOT 1
- ModuleNotFoundError: No module named 'ipex_llm.vllm.xpu' while using docker and installation HOT 1
- [integration]: merging bfloat16 model failed HOT 2
- all-in-one with version 2.1.0b1 failed HOT 3
- need an easy way to roll back driver installs HOT 3
- all-in-one benchmark llama-3-8b-instruct issue with version 2.1.0b1 HOT 3
- about test 3 Gpu with ipex HOT 1
- install issue HOT 2
- bug for inference qwen1.5-7b-chat with SYM_INT4 on Windows platform HOT 3
- Evaluation on if MiniCPM-2B-sft-bf16 need model based optimization on ipex-llm HOT 1
- finetune chatGLM6B using LoRA on arc HOT 3
- XEON and MAX with Kernel 5.15 configuration HOT 1
- transformers 4.38.1 gives bad llama3 performance on MTL iGPU HOT 2
- phi3 medium - garbage output in webui or generated by ollama HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bigdl.