CTC強制對準器
v0.2
如果您感謝我對社區的貢獻,請在 github 上為該專案加註星標(請參見右上角)!
此 Python 套件提供了一種使用 Hugging Face 的預訓練模型在文字和音訊之間執行強制對齊的有效方法。它利用 Wav2Vec2、HuBERT 和 MMS 模型的強大功能進行精確對齊,使其成為創建語音語料庫的強大工具。
標記插入的頻率、段合併的合併閾值等。pip install git+https://github.com/MahmoudAshraf97/ctc-forced-aligner.git
ctc-forced-aligner --audio_path " path/to/audio.wav " --text_path " path/to/text.txt " --language " eng " --romanize
爭論 | 描述 | 預設 |
---|---|---|
--audio_path | 音訊檔案的路徑 | 必需的 |
--text_path | 文字檔案的路徑 | 必需的 |
--language | ISO 639-3 代碼中的語言 | 必需的 |
--romanize | 為非拉丁腳本或多語言模型啟用羅馬化,無論使用預設模型時需要什麼語言 | 錯誤的 |
--split_size | 對齊粒度:“sentence”、“word”或“char” | “單字” |
--star_frequency | 標記的頻率:“段”或“邊” | “邊緣” |
--merge_threshold | 段合併的合併閾值 | 0.00 |
--alignment_model | 對齊模型的名稱 | MahmoudAshraf/mms-300m-1130-強制對準器 |
--compute_dtype | 計算用於推理的 dtype | “浮動32” |
--batch_size | 用於推理的批量大小 | 4 |
--window_size | 音訊分塊的視窗大小(以秒為單位) | 30 |
--context_size | 區塊之間的重疊(以秒為單位) | 2 |
--attn_implementation | 注意執行 | “渴望的” |
--device | 用於推理的設備:“cuda”或“cpu” | “cuda”(如果可用),否則“cpu” |
# Align an English audio file with the text file
ctc-forced-aligner --audio_path " english_audio.wav " --text_path " english_text.txt " --language " eng " --romanize
# Align a Russian audio file with romanized text
ctc-forced-aligner --audio_path " russian_audio.wav " --text_path " russian_text.txt " --language " rus " --romanize
# Align on a sentence level
ctc-forced-aligner --audio_path " audio.wav " --text_path " text.txt " --language " eng " --split_size " sentence " --romanize
# Align using a model with native vocabulary
ctc-forced-aligner --audio_path " audio.wav " --text_path " text.txt " --language " ara " --alignment_model " jonatasgrosman/wav2vec2-large-xlsr-53-arabic "
import torch
from ctc_forced_aligner import (
load_audio ,
load_alignment_model ,
generate_emissions ,
preprocess_text ,
get_alignments ,
get_spans ,
postprocess_results ,
)
audio_path = "your/audio/path"
text_path = "your/text/path"
language = "iso" # ISO-639-3 Language code
device = "cuda" if torch . cuda . is_available () else "cpu"
batch_size = 16
alignment_model , alignment_tokenizer = load_alignment_model (
device ,
dtype = torch . float16 if device == "cuda" else torch . float32 ,
)
audio_waveform = load_audio ( audio_path , alignment_model . dtype , alignment_model . device )
with open ( text_path , "r" ) as f :
lines = f . readlines ()
text = "" . join ( line for line in lines ). replace ( " n " , " " ). strip ()
emissions , stride = generate_emissions (
alignment_model , audio_waveform , batch_size = batch_size
)
tokens_starred , text_starred = preprocess_text (
text ,
romanize = True ,
language = language ,
)
segments , scores , blank_token = get_alignments (
emissions ,
tokens_starred ,
alignment_tokenizer ,
)
spans = get_spans ( tokens_starred , segments , blank_token )
word_timestamps = postprocess_results ( text_starred , spans , stride , scores )
對齊結果將儲存到包含以下 JSON 格式資訊的檔案中:
text
:對齊的文字。segments
:段落列表,每個段落包含對應文字段的開始和結束時間。{
"text" : " This is a sample text to be aligned with the audio. " ,
"segments" : [
{
"start" : 0.000 ,
"end" : 1.234 ,
"text" : " This "
},
{
"start" : 1.234 ,
"end" : 2.567 ,
"text" : " is "
},
{
"start" : 2.567 ,
"end" : 3.890 ,
"text" : " a "
},
{
"start" : 3.890 ,
"end" : 5.213 ,
"text" : " sample "
},
{
"start" : 5.213 ,
"end" : 6.536 ,
"text" : " text "
},
{
"start" : 6.536 ,
"end" : 7.859 ,
"text" : " to "
},
{
"start" : 7.859 ,
"end" : 9.182 ,
"text" : " be "
},
{
"start" : 9.182 ,
"end" : 10.405 ,
"text" : " aligned "
},
{
"start" : 10.405 ,
"end" : 11.728 ,
"text" : " with "
},
{
"start" : 11.728 ,
"end" : 13.051 ,
"text" : " the "
},
{
"start" : 13.051 ,
"end" : 14.374 ,
"text" : " audio. "
}
]
}
歡迎貢獻!請隨時提出問題或提交拉取請求。
該專案在 BSD 許可證下獲得許可,請注意,預設模型具有 CC-BY-NC 4.0 許可證,因此請確保使用不同的模型進行商業用途。
此專案基於 FAIR MMS 團隊的工作。