Also, we are currently at NeurIPS in Vancouver! We will be presenting this work in the RBFM workshop on Saturday! Come say hi, and let’s spark some collaborations! 🚀

This was a monumental collaboration, and a huge thank you to all the co-authors, ServiceNow Research, Mila, and all the institutions involved for their incredible support! 🙏

We hope this effort aids the community in building more robust models for these tasks while emphasizing the importance of open and transparent data usage and release.

We evaluated several VLM models—both open and closed source—on BigDocs-Bench to build a leaderboard.

📊 Models trained on BigDocs outperformed all models on BigDocs-Bench tasks and delivered rebust performance on established benchmarks.
✅ Human evaluations confirmed their strong performance!

To validate the quality of the BigDocs datasets, we trained several VLMs on BigDocs-7.5M and evaluated their performance on document-specific and general VLM benchmarks.

The results? Training on BigDocs provides significant boosts compared to training on other datasets! 📈✨

We introduce BigDocs-Bench, a set of benchmarks that focus on:

📄 Document Understanding
🌐 Web and GUI reasoning
👨‍💻 Code Generation

We also tackle complex outputs like SVG, LaTeX code, Markdown, and HTML, including very long and structured formats. Here are some examples

By sharing this journey, we aim to bring more transparency to how datasets are built—especially as data remains the most opaque aspect of model performance in today’s fast-moving AI landscape. 🌟

Building BigDocs was no small feat! We curated a large-scale dataset from diverse, license-friendly sources and documented the entire process.