v0.4.0

What's Changed

1. New Advanced Attack Techniques: Expanded orchestrators with advanced attack techniques, including PAIR, tree of attacks, and crescendo strategies.
2. New Targets: Crucible target, Prompt Shield Target, Azure OpenAI GPT-4o target
3. New Converters: Added Tense, Emoji, image to text, and Character Space converters.
4. New Scorers: Scale Scorer, Prompt Shield, and True/False Inverter Scorer
5. Automatic Scoring & Memory Labels: Introduced automatic scoring in the PromptSendingOrchestrator. Added support for scoring with user-provided memory labels.
6. Delegation SAS Authentication: Supported delegation SAS authentication for secure interactions with Azure Blob Storage targets.
7. Improved Resiliency: Enhanced the resiliency of targets, converters, and orchestrators with robust error handling mechanisms.
8. Bug Fixes & Performance: Various bug fixes, added support for Python 3.12, speedup unit tests
9. Fetch functionality: Introduced functionality to fetch adversarial datasets, such as SecLists, XStest etc.,
10. Updated Demo Codes: Replaced demo code examples with the GPT-4o target.

Full List of Changes

• FIX: Fixing policheck bug by @rlundeen2 in #261
• release v0.3.0 by @jbolor21 in #265
• DOC: Adding Guidance on Incorporating Research by @rlundeen2 in #268
• FEAT: Adding Tense Converter by @rlundeen2 in #273
• [FEAT] Add Scoring to PromptSendingOrchestrator by @nina-msft in #262
• FIX Fixed mypy Type Failures by @elgertam in #269
• FEAT: Adding Crucible Target by @rlundeen2 in #277
• FIX ValueError with Azure TTS Target in Single Turn Conversation Using PromptSendingOrchestrator by @nina-msft in #278
• FEAT: Converter Tokens by @rlundeen2 in #279
• [FIX] Add flake8-copyright check to pre-commit hooks by @nina-msft in #281
• FIX Exclude Morse Converter from Flake8 Precommit by @nina-msft in #284
• [DRAFT] [FIX] Replace Orchestrator ID with UUID by @nina-msft in #285
• DOC update citation for past tense paper by @romanlutz in #288
• FEAT Add scale scorer by @romanlutz in #274
• FEAT Add Delegation SAS-Based Auth, Update Storage Plugins, and Async Blob Download by @rdheekonda in #286
• FEAT add (back) Gandalf scorer by @romanlutz in #287
• MAINT clean up copyright by @romanlutz in #297
• FEAT: Add Error Handling to AML Chat Target by @nina-msft in #299
• FIX: bug with multi-modal image responses by @rlundeen2 in #301
• MAINT: Improving some LLM Converters by @rlundeen2 in #300
• [FIX][Issue #302] update language version enforcement to fix black-pre-commit installation incompatibility by @shivenchawla in #303
• FEAT return ID in conversation duplication code by @romanlutz in #296
• [FEAT] Implement PAIR by @dlmgary in #255
• FEAT add float scale threshold scorer by @romanlutz in #294
• FEAT: Add GPT4-o chat target by @shivenchawla in #293
• FEAT: Adding Emoji Converter by @rlundeen2 in #306
• DOC: Doc Reorg by @rlundeen2 in #304
• MAINT: Removing asyncio sleep by @rlundeen2 in #309
• MAINT add support for Python 3.12 and fix tests that started breaking by @romanlutz in #305
• FEAT Add print_conversation method to prompt sending orchestrator by @romanlutz in #312
• FEAT Add many-shot jailbreaking feature implementation by @KutalVolkan in #254
• FEAT: Add tree of attacks with pruning by @salmazainana in #210
• FEAT Add Space Converter by @rdheekonda in #316
• FEAT Add Flexible Memory Labels and Scoring to Orchestrators by @rdheekonda in #315
• FEAT: Crescendo Orchestrator by @SafwanA02 in #275
• Feat: Adding multi-turn promptSendingOrchestrator by @rlundeen2 in #317
• DOC Fix README.md link by @romanlutz in #319
• MAINT: Fixing data serializer ability to properly raise errors by @rlundeen2 in #318
• FEAT: Add fetch function for SecLists AI LLM Bias Testing datasets (#267) by @KutalVolkan in #280
• FEAT: Adding true_false inverter scorer by @rlundeen2 in #321
• FIX: fixing check links by @rlundeen2 in #323
• FEAT: Add Exception Handling to Azure TTS Target by @nina-msft in #322
• DOC - replacing gpt4 with gpt4o in example notebooks by @jsong468 in #313
• [MAINT] Changing Examples from stop signs by @jbolor21 in #325
• FEAT Prompt Shield by @ValbuenaVC in #271
• FEAT: add xstest dataset by @KutalVolkan in #320
• [FEAT] Created add_image_text_converter and unit tests by @jsong468 in #328
• DOC: Adding Notebook to document re-sending previous prompts by @rlundeen2 in #332
• MAINT: speeding up crescendo tests by @rlundeen2 in #333
• FIX Move pillow from dev to core dependency by @rdheekonda in #334
• FIX add sample image classifier file by @jbolor21 in #336
• FEAT: Add deterministic flag and custom substitutions to LeetspeakConverter by @KutalVolkan in #329
• MAINT Remove Duplicate Module by @rdheekonda in #337
• MAINT Restructure pyrit.models module and prune by @romanlutz in #338
• [MAINT] Speeding up unit tests by @jbolor21 in #335
• FIX Crescendo backtrack with same orchestrator ID and handling responses with markdown syntax by @romanlutz in #340

New Contributors

• @shivenchawla made their first contribution in #303
• @KutalVolkan made their first contribution in #254
• @salmazainana made their first contribution in #210
• @jsong468 made their first contribution in #313
• @ValbuenaVC made their first contribution in #271

Full Changelog: v0.3.0...v0.4.0