To obtain such certificates, we present a simple, scalable and powerful algorithm: VALID. Remarkably, for each unwanted response it provides a **global bound in prompt space** 🚀

(6/7)

A Domain Certificate bounds the adversarial risk of the model producing out-of-domain responses:

(5/7)

Example: Can't afford Github Copilot? 💡 Use the Amazon Shopping App.

(3/7)

Consider an LLM deployed for a specific purpose like a medical chatbot. Such model should **only** respond to medical questions.

⚠️ Problem: LLMs are very capable and vulnerable to respond to **any** queries: how to build a bomb, organize tax fraud etc.

(2/7)

Our method enables strong LLM performance while providing adversarial guarantees on out-of-domain behaviour.

(4/6)

It is known that fine-tuned foundation models are adversarially vulnerable to provide responses to questions they should not answer.

(2/6)

For instance: Can't afford ChatGPT Plus? Use a shopping app instead.