Team RSLLM SaTML CTF Competition

Submission evaluation and evaluation scripts for the Large Language Model Capture-the-Flag (LLM CTF) Competition @ SaTML 2024 of team RSLLM.

Usage • Docs • Credits

---

Usage

Set your API key

export API_KEY="your_api_key"

Defense Testing Script

Script to automate the defense testing process. Goes through all the attacks and evalutes them for a given defense.

python scripts/testing_defense.py

Competition Attack Script

Script to automate the attack competition. Starts an attack and allows for interactive prompting in the competition.

python scripts/competition_attack.py

Docs

• attacks/sample_attacks.yaml: contains a list of sample attacks
• defenses/gpt.json: GPT defense (prompt and filters)
• defenses/llama.json: LLAMA defense (prompt and filters)

Credits

Robin Schmid  ·  GitHub @RobinSchmid7  ·  Email [email protected]

Takahiro Miki  ·  GitHub @mktk1117  ·  Email [email protected]

Victor Klemm  ·  GitHub @vklemm  ·  Email [email protected]

Chenhao Li  ·  GitHub @breadli428  ·  Email [email protected]

Stefan Kraft  ·  GitHub @stekra  ·  Email [email protected]