This is a video translation and dubbing tool that can translate videos in one language into videos in a specified language, and automatically generate and add subtitles and dubbing in that language. and supports API calls
Speech recognition supports faster-whisper
and openai-whisper
local offline models and OpenAI SpeechToText API
GoogleSpeech
阿里中文语音识别模型
and bean bag model, and supports custom speech recognition API.
Text translation supports微软翻译|Google翻译|百度翻译|腾讯翻译|ChatGPT|AzureAI|Gemini|DeepL|DeepLX|字节火山|离线翻译OTT
Text synthesis speech supports Microsoft Edge tts
Google tts
Azure AI TTS
Openai TTS
Elevenlabs TTS
自定义TTS服务器api
GPT-SoVITS
clone-voice ChatTTS-ui Fish TTS CosyVoice
Allows to retain background accompaniment music, etc. (based on uvr5)
Supported languages: Chinese Simplified and Traditional, English, Korean, Japanese, Russian, French, German, Italian, Spanish, Portuguese, Vietnamese, Thai, Arabic, Turkish, Hungarian, Indian, Ukrainian, Kazakh, Indonesian, Malay, Czech, Polish, Dutch, Swedish/other languages optional automatic detection
[Sponsor]
302.AI is an AI supermarket that brings together the world's top brands, with pay-as-you-go, zero monthly fees, and zero threshold for using various types of AI.
Comprehensive functions/easy to use/zero threshold for paying on demand/separation of managers and users
[Automatically translate video and dub] Translate the sound in the video into dubbing in another language and embed subtitles in that language
[Speech recognition/Convert audio and video to subtitles] Can batch recognize human voices in audio and video files as text and export them as srt subtitle files
[Speech synthesis/subtitle dubbing] Create dubbing based on local existing srt subtitle files, supporting single or batch subtitles
[Translate subtitle files] Translate one or more srt subtitle files into subtitle files in other languages
[Merge video and audio] Merge video files and audio files in batches in one-to-one correspondence
[Merge video and srt subtitles] Batch merge video files and srt subtitle files one by one
[Add image watermark to videos] Embed image watermarks into video files in batches
【Extract audio from video】Separate video into audio files and silent video
[Audio and video format conversion] Convert audio and video formats in batches
[Subtitle editing and exporting to multiple formats] Supports importing subtitles in srt, vtt, and ass formats. After editing, you can set the font style, color, etc. to export subtitles in the corresponding format.
[Subtitle format conversion] Convert subtitle files to srt/ass/vtt formats in batches
[Download YouTube videos] Videos can be downloaded from YouTube
[Separation of vocal background music]
[API call] Supports speech synthesis, language recognition, subtitle translation, and video translation interface calls
Use pyinstaller to package without anti-virus and signature. Anti-virus software may report viruses. Please add it to the trust list or use source code for deployment.
[Click to download the prepackaged version, unzip it to an English directory without spaces, and double-click sp.exe (https://github.com/jianchang512/pyvideotrans/releases)
Unzip it to an English path, and the path does not contain spaces. After unzipping, double-click sp.exe (if you encounter permission problems, you can right-click and open with administrator rights)
Note: It must be decompressed before use. It cannot be double-clicked directly in the compressed package, nor can the sp.exe file be moved to other locations after decompression.
Open a terminal window and execute the following commands respectively:
Make sure Homebrew is installed before executing. If you do not have Homebrew installed, you need to install it first.
Execute the command to install Homebrew:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
After the installation is complete, execute:
eval $(brew --config)
brew install libsndfile brew install ffmpeg brew install git brew install [email protected]
Continue execution
export PATH="/usr/local/opt/[email protected]/bin:$PATH" source ~/.bash_profile source ~/.zshrc
Create a folder without spaces and Chinese characters, and enter the folder in the terminal.
Execute the command git clone https://github.com/jianchang512/pyvideotrans
in the terminal
Execute the command cd pyvideotrans
Continue executing python -m venv venv
Continue to execute the command source ./venv/bin/activate
. After execution, check and confirm that the terminal command prompt has changed to start with (venv)
. The following commands must make sure that the terminal prompt starts with (venv)
.
Execute pip install -r requirements.txt
. If the prompt fails, execute the following 2 commands to switch the pip image to the Alibaba image.
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ pip config set install.trusted-host mirrors.aliyun.com
Then re-execute it. If you have switched to the Alibaba image source and it still fails, please try to execute pip install -r requirements.txt
python sp.py
opens the software interface
CentOS/RHEL system execute the following commands in order to install python3.10
sudo yum update sudo yum groupinstall "Development Tools" sudo yum install openssl-devel bzip2-devel libffi-devel cd /tmp wget https://www.python.org/ftp/python/3.10.4/Python-3.10.4.tgz tar xzf Python-3.10.4.tgz cd Python-3.10.4 ./configure — enable-optimizations sudo make && sudo make install sudo alternatives — install /usr/bin/python3 python3 /usr/local/bin/python3.10 1 sudo yum install -y ffmpeg
For Ubuntu/Debian system, execute the following command to install python3.10
apt update && apt upgrade -y apt install software-properties-common -y add-apt-repository ppa:deadsnakes/ppa apt update sudo apt-get install libxcb-cursor0 apt install python3.10 curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10 sudo update-alternatives --install /usr/bin/python python /usr/local/bin/python3.10 1 sudo update-alternatives --config python apt-get install ffmpeg
Open any terminal and execute python3 -V
. If "3.10.4" is displayed, the installation is successful, otherwise it fails.
Create a folder without spaces and Chinese characters, and open the folder from the terminal.
Execute the command git clone https://github.com/jianchang512/pyvideotrans
in the terminal
Continue executing the command cd pyvideotrans
Continue executing python -m venv venv
Continue to execute the command source ./venv/bin/activate
. After execution, check and confirm that the terminal command prompt has changed to start with (venv)
. The following commands must make sure that the terminal prompt starts with (venv)
.
Execute pip install -r requirements.txt
. If the prompt fails, execute the following 2 commands to switch the pip image to the Alibaba image.
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ pip config set install.trusted-host mirrors.aliyun.com
Then re-execute it. If you have switched to the Alibaba image source and still fail, please try pip install -r requirements.txt
If you want to use CUDA acceleration, execute it separately
pip uninstall -y torch torchaudio
pip install torch==2.2.0 torchaudio==2.2.0 --index-url https://download.pytorch.org/whl/cu118
pip install nvidia-cublas-cu11 nvidia-cudnn-cu11
linux If you want to enable cuda acceleration, you must have an NVIDIA graphics card and configure the CUDA11.8+ environment. Please search for "Linux CUDA installation"
python sp.py
opens the software interface
Open https://www.python.org/downloads/ to download windows 3.10. After downloading, double-click and click next. Make sure to select "Add to PATH"
Open a cmd and execute python -V
. If the output is not 3.10.4
, it means there is an installation error or Add to PATH
is not added. Please reinstall.
Open https://github.com/git-for-windows/git/releases/download/v2.45.0.windows.1/Git-2.45.0-64-bit.exe, download git, and double-click the next step after downloading. .
Find a folder without spaces and Chinese characters, enter cmd
in the address bar and press Enter, open the terminal, and execute the following commands in the terminal
Execute the command git clone https://github.com/jianchang512/pyvideotrans
Continue executing the command cd pyvideotrans
Continue executing python -m venv venv
Continue to execute the command .venvscriptsactivate
. After execution, please check to confirm that the beginning of the command line has changed to (venv)
, otherwise an error has occurred.
Execute pip install -r requirements.txt
. If the prompt fails, execute the following 2 commands to switch the pip image to the Alibaba image.
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ pip config set install.trusted-host mirrors.aliyun.com
Then re-execute it. If you have switched to the Alibaba image source and still fail, please try pip install -r requirements.txt
If you want to use CUDA acceleration, execute it separately
pip uninstall -y torch torchaudio
pip install torch==2.2.0 torchaudio==2.2.0 --index-url https://download.pytorch.org/whl/cu118
Windows If you want to enable CUDA acceleration, you must have an NVIDIA graphics card and configure the CUDA11.8+ environment. For specific installation, see CUDA acceleration support.
Unzip ffmpeg.zip to the current source code directory. If prompted to overwrite, overwrite. After decompression, ensure that ffmpeg.exe ffprobe.exe ytwin32.exe can be seen in the ffmepg folder under the source code.
python sp.py
opens the software interface
The 4.x version of ctranslate2 is used by default and only supports CUDA12.x version. If your cuda is lower than 12 and cannot upgrade cuda to 12.x, please execute the command to uninstall ctranslate2 and then reinstall it.
pip uninstall -y ctranslate2 pip install ctranslate2==3.24.0
You may encounter errors such as xx module not found
. Please open requirements.txt, search for the xx module, and then remove the == after xx and the version number after a while.
Source code deployment/b station under Mac
How to set up video translation using Gemini Api/station b
How to download and install
ChatTTS-ui: UI interface using ChatTTS to synthesize sounds
OTT: Local offline text translation tool
Voice cloning tool: synthesize speech with any timbre
Speech recognition tool: local offline speech recognition to text tool
Vocal background music separation: Vocal and background music separation tool
Improved version of api.py of GPT-SoVITS
api.py adapted to CosyVoice
Some open source projects that this program mainly relies on
ffmpeg
PySide6
edge-tts
faster-whisper
openai-whisper
pydub