Daniel Marczak
@dmarczak.bsky.social
mostly trying to merge models | phd student @ warsaw university of technology & ideas
Pinned
Daniel Marczak
@dmarczak.bsky.social
· Feb 10
🚀 What happens when you modify the spectrum of singular values of the merged task vector? 🤔
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
🚀 What happens when you modify the spectrum of singular values of the merged task vector? 🤔
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
February 10, 2025 at 2:47 PM
🚀 What happens when you modify the spectrum of singular values of the merged task vector? 🤔
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
Apparently, you achieve 🚨state-of-the-art🚨 model merging results! 🔥
✨ Introducing “No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces”
Reposted by Daniel Marczak
Self-supervised Learning with Masked Autoencoders (MAE) is known to produce worse image representations than Joint-Embedding approaches (e.g. DINO). In our new paper, we identify new reasons for why that is and point towards solutions: arxiv.org/abs/2412.03215 🧵
December 5, 2024 at 7:56 PM
Self-supervised Learning with Masked Autoencoders (MAE) is known to produce worse image representations than Joint-Embedding approaches (e.g. DINO). In our new paper, we identify new reasons for why that is and point towards solutions: arxiv.org/abs/2412.03215 🧵