Basic torchgfn usage follows standard pytorch workflows, allowing the user to swap in any modified components to support the development of new methods:

Great job gang! And thanks to @boussifo.bsky.social for being such a stellar lead. (Also depicted: @jainmoksh.bsky.social)

These chunks also generalize - they’re transferable across samplers and tasks!

Chunks learned in one environment improve exploration and sampling efficiency in unseen settings, suggesting the method abstracts high order general principles that are robust & adaptable to new envs!

For mode discovery, our approach also significantly speeds up discovering diverse high-reward states.

For example, in FractalGrid, vanilla GFlowNets get stuck in a single mode, but armed with ActionPiece, it unlocks new exploration paths!

Chunking helps!

Across synthetic and real-world tasks (e.g., RNA sequence generation, bit sequences, and FractalGrid), our approach improves especially for GFlowNets:

✅ Mode discovery
✅ Exploration
✅ Density estimation
✅ Interpretability

By applying BPE (which we're calling "ActionPiece" for learning chunked actions) to sampled trajectories, we extract meaningful high-level actions that naturally emerge during learning. For example, here are some learned chunks from sampler of RNA binders:

Ecstatic to show off some work my brilliant colleagues and I did at @iclr-conf.bsky.social this year! 🚀

We address the credit assignment challenge under long trajectories in RL or GFlowNets by constructing high order actions, or “chunks”, effectively compressing trajectory lengths!

This book radically influenced my way of looking at human civilization and the kinds of tech that must be built to move us into a post sacristy, sustainable future. His story (overcoming failure/depression to produce major intellectual contributions outside the academic system was also formative.