Raman Dutt
@ramandutt4.bsky.social
Generative AI @Noah's Ark Lab, Huawei & @TuringInstitute | PhD candidate in Biomedical AI @ University of Edinburgh | Efficient Fine-Tuning in Medical AI, Diffusion Models, Autoregressive Image Generation
Not an ML-related post but I am just as happy to share this
December 6, 2024 at 1:48 PM
Not an ML-related post but I am just as happy to share this
Hence, we devise MemControl - a framework that searches for the optimal parameters to be fine-tuned to:
(1) Improve image generation quality
(2) Reduce Memorization!
MemControl leads to optimal model capacity that should be used during fine-tuning: Not more, not less!
(1) Improve image generation quality
(2) Reduce Memorization!
MemControl leads to optimal model capacity that should be used during fine-tuning: Not more, not less!
November 26, 2024 at 6:32 PM
Hence, we devise MemControl - a framework that searches for the optimal parameters to be fine-tuned to:
(1) Improve image generation quality
(2) Reduce Memorization!
MemControl leads to optimal model capacity that should be used during fine-tuning: Not more, not less!
(1) Improve image generation quality
(2) Reduce Memorization!
MemControl leads to optimal model capacity that should be used during fine-tuning: Not more, not less!
We also found that fine-tuning different subsets of parameters in a diffusion model can affect generative quality and memorization differently!
Each marker in the figure is a diffusion model finetuned on the same data but with different parameter subset.
Full FT (green) leads to high memorization!
Each marker in the figure is a diffusion model finetuned on the same data but with different parameter subset.
Full FT (green) leads to high memorization!
November 26, 2024 at 6:30 PM
We also found that fine-tuning different subsets of parameters in a diffusion model can affect generative quality and memorization differently!
Each marker in the figure is a diffusion model finetuned on the same data but with different parameter subset.
Full FT (green) leads to high memorization!
Each marker in the figure is a diffusion model finetuned on the same data but with different parameter subset.
Full FT (green) leads to high memorization!
The conventional way of fine-tuning models (full fine-tuning) can lead to replication of artifacts in X-Rays that can further lead to leakage of patient information, thus endangering patient privacy.
Artifact replication is shown in red boxes.
Artifact replication is shown in red boxes.
November 26, 2024 at 6:24 PM
The conventional way of fine-tuning models (full fine-tuning) can lead to replication of artifacts in X-Rays that can further lead to leakage of patient information, thus endangering patient privacy.
Artifact replication is shown in red boxes.
Artifact replication is shown in red boxes.
November 24, 2024 at 4:28 PM
Migration to BlueSky successful! Thank you @mmbronstein.bsky.social 🙏
November 24, 2024 at 3:12 PM
Migration to BlueSky successful! Thank you @mmbronstein.bsky.social 🙏