PromptExtractionEval

從定制的大語言模型中提取提示

本文由紙的源代碼組成： Why Are My Prompts Leaked? Unraveling Prompt Extraction Threats in Customized Large Language Models 。

源代碼說明

• PEAD數據集：提取prompt/指令/benchmark_collections/themus_data_benchmark.json
• 所有實驗的源代碼：提取prompt/
  • 廣義評估
    • 香草：提取prompt/1.run_prompt_extraction.py
    • 功能呼叫比較：deftractingPrompt/5.funcall_comparison.py
  • 及時提取法律規律
    • 型號大小：deftractingprompt/2.model_size_prompt_extraction_experiments.py
    • 序列長度：提取prompt/4.varying_sequence_length.py
  • 經驗分析
    • 令人信服的前提：提取prompt/6.ppl_comparison.py
    • 平行翻譯：提取prompt/7.ATTENTION_VISUALIZE.PY
    • 並行翻譯：提取prompt/coative_visualize.py
  • 國防策略
    • 防禦方法：Defacting Prompt/Deferding/PPL_HIGH2_CONFUSINGBEGINNINGS.PY
    • 防禦的性能下降實驗：提取啟示/防禦/2.drops_of_defending.py
    • 可視化：提取/辯護/防禦/防禦_Visualization.py
  • 閉合實驗
    • 香草提示提取：提取prompt/api_related_experiments/1.run_prompt_extraction.py
    • 軟提取：提取prompt/api_related_experiments/2.soft_extraction_experiments.py
    • 防禦的性能下降：deftractingprompt/api_related_experiments/3.1.drops_of_defense.py

實驗環境

跑步

pip install -r re.txt

或手動安裝以下密鑰軟件包：

datasets
numpy
pandas
peft
safetensors
scipy
tensorboard
tensorboardX
tiktoken
tokenizers
torch
tqdm
transformers
matplotlib
scikit-learn
thefuzz
einops
sentencepiece

聯繫作者

如果存在任何問題，請隨時打開問題，或將電子郵件發送到[email protected] 。

引用：

 @misc { liang2024promptsleakedunravelingprompt ,
      title = { Why Are My Prompts Leaked? Unraveling Prompt Extraction Threats in Customized Large Language Models } , 
      author = { Zi Liang and Haibo Hu and Qingqing Ye and Yaxin Xiao and Haoyang Li } ,
      year = { 2024 } ,
      eprint = { 2408.02416 } ,
      archivePrefix = { arXiv } ,
      primaryClass = { cs.CL } ,
      url = { https://arxiv.org/abs/2408.02416 } , 
}