Special thanks to Biogen and CIFAR for the support, and
@proceduralia.bsky.social + @pierrelucbacon.bsky.social
for their valuable supervision, and to the entire Mila community for their feedback, discussions, and support. Code, paper, and models are public: github.com/ddidacus/mol...

Mol-MoE improves with more property experts with a larger gain than classic merging and overall, it achieves the highest scores. Simple reward scalarization here does not work. We aim at further calibrating Mol-MoE and testing the performance on larger sets of objectives.

The model we obtain does achieve a smaller mean absolute error in generating compounds according to the provided properties, surpassing the alternative methods. Arguably, the learned routing functions can tackle task interference.

But the relationship between interpolation coefficients and properties isn’t strictly linear, needing a calibration function. Mol-MoE addresses this by training only the routers to predict optimal merging weights from prompts, enabling more precise control and less interference.

Think, think, think... what if we trained experts on single properties separately and leveraged model merging techniques to obtain a multi-property model? We re-implement rewarded soups and obtain a robust baseline capable of generating high-quality, out-of-distribution samples.

In our ablation studies, instruction-tuned models struggle with higher property values due to lack of explicit optimization. Even RL fine-tuning on multiple objectives can hit performance plateaus or declines, and balancing objectives requires re-training, limiting steerability.

Drug discovery inherently involves multi-objective optimization, requiring candidate molecules to not only bind effectively to target proteins, triggering a specific function, but also to meet safety and compatibility criteria to become drugs. Is supervised learning sufficient?

Finally, LOgically COnsistent (LoCo) LLaMas can outperform solver-based baselines and SFT! I thank @nolovedeeplearning.bsky.social and @looselycorrect.bsky.social for the guidance in realizing this project, get in touch or come to chat in Singapore!

arxiv.org/abs/2409.13724

Our method makes LLaMa's knowledge more consistent to any given knowledge graph, by seeing only a portion of it! It can transfer logical rules to similar or derived concepts. As proposed by @ekinakyurek.bsky.social et al., you can use a LLM-generated KB to reason over its knowledge.

Yes! We propose to leverage the Semantic Loss as a regularizer: it maximizes the likelihood of world (model) assignments satisfying any given logical rule. We thus include efficient solvers in the training pipeline to efficiently perform model counting on the LLM's own beliefs.

Various background works focus on instilling single consistency rules, e.g. A and not A can't be both true (negation, Burns et al.), A true and A implies B, thus B true (modus ponens). Can we derive a general objective function that combines logical rules dynamically?

I guess it also depends on the field/subfield?