What books is this? 👀

Will do!

Very cool!

This was the last paper of my PhD and was in collaboration with my very dear advisors @kaiwei_chang and @guyvdb

We evaluate our approach, which we call Gen C (like Gen Z, get it?) on several tasks such as LLM detoxification, Sudoku as well as shortest-path prediction and and show that our approach outperforms the baselines. We plan on adding even more tasks very soon.

More importantly, we can efficiently condition this approximate distribution on our constraint such that any sample provably satisfies the constraint. We can reweigh our samples using the LLM to correct for any bias introduced by our approximate distribution.

To do so, we construct a first-order approximation of the LLM centered at the unconstrained sample. This approximation naturally does not constitute the best LM, but allows us to efficiently represent a distribution over all sentences of bounded length.

Now imagine we want to ban a bad expression, say "full of sh!t". We start by taking a sample from the LLM. The sample, shown in red, violates the constraint. What we want to do now is project the sample onto the support of the LLM distribution satisfying the constraint, m(alpha).

Constrained decoding typically uses a DFA to mask invalid tokens at every step of generation. This ensures constraint satisfaction* but can introduce significant bias in the generated output.

*This is not strictly true due to tokenization. See paper for more on this.

More importantly, we can efficiently condition this approximate distribution on our constraint such that any sample provably satisfies the constraint. We can reweigh our samples using the LLM to correct for any bias introduced by our approximate distribution.

To do so, we construct a first-order approximation of the LLM centered at the unconstrained sample. This approximation naturally does not constitute the best LM, but allows us to efficiently represent a distribution over all sentences of bounded length.

Now imagine we want to ban a bad expression, say "full of sh!t". We start by taking a sample from the LLM. The sample, shown in red, violates the constraint. What we want to do now is project the sample onto the support of the LLM distribution satisfying the constraint, m(alpha).

Constrained decoding typically uses a DFA to mask invalid tokens at every step of generation. This ensures constraint satisfaction* but can introduce significant bias in the generated output.

*This is not strictly true due to tokenization. See paper for more on this.

Can't wait to finally meet you and hopefully @mniepert.bsky.social in person! :)

Hi! I work on probabilistic ML and Neuro-Symbolic AI