The answer? Leverage the knowledge built into VLMs (benefit from zero-shot and few-shot capabilities), use common off-the-shelf models and methods like fine-tuning and parameter-efficient tuning, and tap into multi-task capabilities

He then explores the shift from handwriting recognition to stroke interactions. While handwriting recognition is well-established, large models are driving a technological transformation. So how can we integrate handwriting, digital ink understanding & freeform capabilities into these models?

Claudiu Musat, researcher at Google DeepMind Switzerland, introduces strokes as a new modality for VLMs. Strokes enhance online expression and offer robustness against visual distractions. 🖌️

When should you use context caching?
✅ Chatbots with extensive system instructions
✅ Repetitive analysis of lengthy video files
✅ Recurring queries over large document sets
✅ Frequent code repository analysis & bug fixing

Corpus-in-Context can be slow and potentially costly due to its quadratic complexity. This is where context caching comes to the rescue! When the corpus doesn't change much, computations can be performed once and reused multiple times. This significantly boosts speed and reduces costs. ⚡️