Simen
@eide.ai
AI statistician (is that something?) building algorithms for Schibsted news media during daytime, outdoors and always injured during other times. Based in Norway. Not sure what ill do here yet
The added benefit of being bayesian here is that tasks with less data will be more similar to the hierarchical mean parameter set Θ. And therefore learn from other tasks!
November 25, 2024 at 2:36 PM
The added benefit of being bayesian here is that tasks with less data will be more similar to the hierarchical mean parameter set Θ. And therefore learn from other tasks!
Effectively, each task adapter will be optimized both towards the training data, but also to be similar to the other adapters. And therefore the adapters will share knowledge between them!
November 25, 2024 at 2:36 PM
Effectively, each task adapter will be optimized both towards the training data, but also to be similar to the other adapters. And therefore the adapters will share knowledge between them!
It works by constructing a hierarchical LLM where each task adapter parameter set θ_d has a prior to a shared hierarchical mean parameter set Θ.
November 25, 2024 at 2:36 PM
It works by constructing a hierarchical LLM where each task adapter parameter set θ_d has a prior to a shared hierarchical mean parameter set Θ.
It outperforms both the case when you train a shared adapter for all tasks, and the case when you train one adapter per task indepedently on our dataset.
November 25, 2024 at 2:36 PM
It outperforms both the case when you train a shared adapter for all tasks, and the case when you train one adapter per task indepedently on our dataset.