Joel Ye
@joelye.bsky.social
NeuroAI PhD student @ Carnegie Mellon. Being a brain-computer interface.
NDT3 is brought to you by a huge team of data contributors from @corticalbionics.bsky.social
@raeedcho.bsky.social
@jenpitt.bsky.social
. ? (Apologies for the shortlist, not wired into the bsky community yet)
@raeedcho.bsky.social
@jenpitt.bsky.social
. ? (Apologies for the shortlist, not wired into the bsky community yet)
February 7, 2025 at 2:21 PM
NDT3 is brought to you by a huge team of data contributors from @corticalbionics.bsky.social
@raeedcho.bsky.social
@jenpitt.bsky.social
. ? (Apologies for the shortlist, not wired into the bsky community yet)
@raeedcho.bsky.social
@jenpitt.bsky.social
. ? (Apologies for the shortlist, not wired into the bsky community yet)
Link: doi.org/10.1101/2025...
Code: github.com/joel99/ndt3
. Weights are posted on HuggingFace, includes a notebook for fine-tuning NDT3 on a new dataset.
Code: github.com/joel99/ndt3
. Weights are posted on HuggingFace, includes a notebook for fine-tuning NDT3 on a new dataset.
February 7, 2025 at 2:21 PM
Link: doi.org/10.1101/2025...
Code: github.com/joel99/ndt3
. Weights are posted on HuggingFace, includes a notebook for fine-tuning NDT3 on a new dataset.
Code: github.com/joel99/ndt3
. Weights are posted on HuggingFace, includes a notebook for fine-tuning NDT3 on a new dataset.
Lowlights:
- NDT3 scales weakly, likely due to cross-subject transfer issues (but there's some rzn to believe cross-attention encoders, like in POYO, wouldn't fare better)
- NDT3 will still overfit some types of stereotypy in neural data, e.g. reach angle. Post-training likely needed to resolve
- NDT3 scales weakly, likely due to cross-subject transfer issues (but there's some rzn to believe cross-attention encoders, like in POYO, wouldn't fare better)
- NDT3 will still overfit some types of stereotypy in neural data, e.g. reach angle. Post-training likely needed to resolve
February 7, 2025 at 2:21 PM
Lowlights:
- NDT3 scales weakly, likely due to cross-subject transfer issues (but there's some rzn to believe cross-attention encoders, like in POYO, wouldn't fare better)
- NDT3 will still overfit some types of stereotypy in neural data, e.g. reach angle. Post-training likely needed to resolve
- NDT3 scales weakly, likely due to cross-subject transfer issues (but there's some rzn to believe cross-attention encoders, like in POYO, wouldn't fare better)
- NDT3 will still overfit some types of stereotypy in neural data, e.g. reach angle. Post-training likely needed to resolve
Highlights:
- Pretrained on 2000 hours. By data/params, NDT3 brings motor BCI to the BERT era!
- Evaluated on 8 datasets, (including 3 from FALCON, go beat NDT3 snel.ai/falcon
!)
- Gains ~generalize to distribution shifts in neural data -- e.g. in time, posture
- Pretrained on 2000 hours. By data/params, NDT3 brings motor BCI to the BERT era!
- Evaluated on 8 datasets, (including 3 from FALCON, go beat NDT3 snel.ai/falcon
!)
- Gains ~generalize to distribution shifts in neural data -- e.g. in time, posture
February 7, 2025 at 2:21 PM
Highlights:
- Pretrained on 2000 hours. By data/params, NDT3 brings motor BCI to the BERT era!
- Evaluated on 8 datasets, (including 3 from FALCON, go beat NDT3 snel.ai/falcon
!)
- Gains ~generalize to distribution shifts in neural data -- e.g. in time, posture
- Pretrained on 2000 hours. By data/params, NDT3 brings motor BCI to the BERT era!
- Evaluated on 8 datasets, (including 3 from FALCON, go beat NDT3 snel.ai/falcon
!)
- Gains ~generalize to distribution shifts in neural data -- e.g. in time, posture
What it is:
- NDT3 is "just" an autoregressive Transformer, for decoding behavior from neural spiking activity
- We tokenize behavior timeseries by splitting and neural data by patching (a la NDT2/ViT). Thus, the model can ingest most motor brain-computer interface datasets
- NDT3 is "just" an autoregressive Transformer, for decoding behavior from neural spiking activity
- We tokenize behavior timeseries by splitting and neural data by patching (a la NDT2/ViT). Thus, the model can ingest most motor brain-computer interface datasets
February 7, 2025 at 2:21 PM
What it is:
- NDT3 is "just" an autoregressive Transformer, for decoding behavior from neural spiking activity
- We tokenize behavior timeseries by splitting and neural data by patching (a la NDT2/ViT). Thus, the model can ingest most motor brain-computer interface datasets
- NDT3 is "just" an autoregressive Transformer, for decoding behavior from neural spiking activity
- We tokenize behavior timeseries by splitting and neural data by patching (a la NDT2/ViT). Thus, the model can ingest most motor brain-computer interface datasets