n1o_c0rTx
@n1o-cortx.bsky.social
From machine learning to vunerability research.
For more general ML stuff: https://n1o.github.io/
For more ML focus on Vunerability Research: https://codebreakers.re/
Github: https://github.com/n1o
For more general ML stuff: https://n1o.github.io/
For more ML focus on Vunerability Research: https://codebreakers.re/
Github: https://github.com/n1o
I am a huge fan of Graph Neural Networks and Large Language Models cannot be ignored, but both of them have some weaknesses and strengths. What if we could combine those two to get the best of both worlds:
n1o.github.io/posts/graph-...
#LLM #Graphs #GNN #AI
n1o.github.io/posts/graph-...
#LLM #Graphs #GNN #AI
Graph Neural Networks meet Large Language Models
Abstract Link to heading I am a huge fan of Graph Neural Networks (GNNs), and I am (a bit less) a fan of Large Language Models (LLMs), however they are hard to ignore. Both have different strengths, w...
n1o.github.io
December 17, 2024 at 10:22 AM
I am a huge fan of Graph Neural Networks and Large Language Models cannot be ignored, but both of them have some weaknesses and strengths. What if we could combine those two to get the best of both worlds:
n1o.github.io/posts/graph-...
#LLM #Graphs #GNN #AI
n1o.github.io/posts/graph-...
#LLM #Graphs #GNN #AI
At code:Breakers, I do a lot of research about source code and AI. One thing that stands out is that source code is mostly a Graph/Tree; however, most LLMs work with sequences of tokens.
codebreakers.re/articles/llm...
#AI #LLM #GNN #Graphs
codebreakers.re/articles/llm...
#AI #LLM #GNN #Graphs
GALLa: Graph Aligned Large Language Models for Improved Source Code Understanding
Merging Graph Neural Networks with LLMs for improved Source Code Understanding! GALLa is an excellent example of how we can extend LLMs and their superior Sequence Processing capabilities with somethi...
codebreakers.re
December 10, 2024 at 8:44 AM
At code:Breakers, I do a lot of research about source code and AI. One thing that stands out is that source code is mostly a Graph/Tree; however, most LLMs work with sequences of tokens.
codebreakers.re/articles/llm...
#AI #LLM #GNN #Graphs
codebreakers.re/articles/llm...
#AI #LLM #GNN #Graphs
@sirbayes.bsky.social Just droped the following paper:
arxiv.org/abs/2412.05265
I guess this is my christmas present. I got a hardcover copy of his first book more than 10 years ago also for christmas, and it hade an huge impact on my life. I can't wait to finaly dive deep into RL!
#AI #RL
arxiv.org/abs/2412.05265
I guess this is my christmas present. I got a hardcover copy of his first book more than 10 years ago also for christmas, and it hade an huge impact on my life. I can't wait to finaly dive deep into RL!
#AI #RL
Reinforcement Learning: An Overview
This manuscript gives a big-picture, up-to-date overview of the field of (deep) reinforcement learning and sequential decision making, covering value-based RL, policy-gradient methods, model-based met...
arxiv.org
December 9, 2024 at 7:43 AM
@sirbayes.bsky.social Just droped the following paper:
arxiv.org/abs/2412.05265
I guess this is my christmas present. I got a hardcover copy of his first book more than 10 years ago also for christmas, and it hade an huge impact on my life. I can't wait to finaly dive deep into RL!
#AI #RL
arxiv.org/abs/2412.05265
I guess this is my christmas present. I got a hardcover copy of his first book more than 10 years ago also for christmas, and it hade an huge impact on my life. I can't wait to finaly dive deep into RL!
#AI #RL
TLDR; FuseGPT
Large Language Models are huge, to reduce their resource requirements an common approach is to prune uneccesary layers. However this is not without cost, it requires expensive retraining to retain lost performance.
#AI #LLM #GPT #ModelPruning
Large Language Models are huge, to reduce their resource requirements an common approach is to prune uneccesary layers. However this is not without cost, it requires expensive retraining to retain lost performance.
#AI #LLM #GPT #ModelPruning
December 3, 2024 at 1:21 PM
TLDR; FuseGPT
Large Language Models are huge, to reduce their resource requirements an common approach is to prune uneccesary layers. However this is not without cost, it requires expensive retraining to retain lost performance.
#AI #LLM #GPT #ModelPruning
Large Language Models are huge, to reduce their resource requirements an common approach is to prune uneccesary layers. However this is not without cost, it requires expensive retraining to retain lost performance.
#AI #LLM #GPT #ModelPruning
A couple of days ago, Nvidia released a new breed of Attention-SSM hybrids called Hymba:
n1o.github.io/posts/hymba-...
Since their approach is really groundbreaking, I compiled a somewhat exhaustive but beginner-friendly summary:
#LLM #AI #SSM #Attention #NVIDIA
n1o.github.io/posts/hymba-...
Since their approach is really groundbreaking, I compiled a somewhat exhaustive but beginner-friendly summary:
#LLM #AI #SSM #Attention #NVIDIA
Hymba, a new breed of SSM-Attention Hybrids
Abstract Link to heading State space models are really close to my heart, I even have a dedicated page about them. But when it comes to Language Models they lack some performance and that gave rise to...
n1o.github.io
December 2, 2024 at 8:50 AM
A couple of days ago, Nvidia released a new breed of Attention-SSM hybrids called Hymba:
n1o.github.io/posts/hymba-...
Since their approach is really groundbreaking, I compiled a somewhat exhaustive but beginner-friendly summary:
#LLM #AI #SSM #Attention #NVIDIA
n1o.github.io/posts/hymba-...
Since their approach is really groundbreaking, I compiled a somewhat exhaustive but beginner-friendly summary:
#LLM #AI #SSM #Attention #NVIDIA
Currios how you can turn any LLM into an embeding model?
It is possible with Echo Embeding. Start with a prompt to guide the embeding, and repeat what you want to embed. And you done. Here is an example for Qwen2.5-Coder from Hugging Face.
#LLM #AI #Embedding #pytorch
It is possible with Echo Embeding. Start with a prompt to guide the embeding, and repeat what you want to embed. And you done. Here is an example for Qwen2.5-Coder from Hugging Face.
#LLM #AI #Embedding #pytorch
November 28, 2024 at 12:24 PM
Currios how you can turn any LLM into an embeding model?
It is possible with Echo Embeding. Start with a prompt to guide the embeding, and repeat what you want to embed. And you done. Here is an example for Qwen2.5-Coder from Hugging Face.
#LLM #AI #Embedding #pytorch
It is possible with Echo Embeding. Start with a prompt to guide the embeding, and repeat what you want to embed. And you done. Here is an example for Qwen2.5-Coder from Hugging Face.
#LLM #AI #Embedding #pytorch
Mamba(2) is great but not without flaws. Combining with Attention can aleviate its shortcommings. If you want to learn more about current SSM-Attention Hybrids I compiled a extensive review with cons and pros:
n1o.github.io/posts/ssm-tr...
#AI #Mamba #StateSpaceModels #LLM
n1o.github.io/posts/ssm-tr...
#AI #Mamba #StateSpaceModels #LLM
Mamba(2) and Transformer Hybrids: An Overview
Abstract Link to heading We have already looked into Mamba and Mamba2. In terms of efficiency, with their linear complexity and the absence of Key-Value cache, they are a significant improvement over ...
n1o.github.io
November 27, 2024 at 9:15 AM
Mamba(2) is great but not without flaws. Combining with Attention can aleviate its shortcommings. If you want to learn more about current SSM-Attention Hybrids I compiled a extensive review with cons and pros:
n1o.github.io/posts/ssm-tr...
#AI #Mamba #StateSpaceModels #LLM
n1o.github.io/posts/ssm-tr...
#AI #Mamba #StateSpaceModels #LLM
Cant be more happy to discover Paged Out, so much cool things
November 26, 2024 at 12:31 PM
Cant be more happy to discover Paged Out, so much cool things
During a productive weekend I dug into the topic of creating powerful embedding models from pretrained LLMs.
n1o.github.io/posts/from-l...
#AI #LLM #BERT #NLP #EMBEDDINGS
n1o.github.io/posts/from-l...
#AI #LLM #BERT #NLP #EMBEDDINGS
Transform any LLMs to a powerful Encoder
Abstract Link to heading In the last two years there has been a surge of Large Language Models. This is understandable, since LLMs are amazing at generating text, and a lot of things can be viewed as ...
n1o.github.io
November 25, 2024 at 9:39 AM
During a productive weekend I dug into the topic of creating powerful embedding models from pretrained LLMs.
n1o.github.io/posts/from-l...
#AI #LLM #BERT #NLP #EMBEDDINGS
n1o.github.io/posts/from-l...
#AI #LLM #BERT #NLP #EMBEDDINGS