Piotr Skalski
@skalskip92.bsky.social
Open-source Lead @roboflow. VLMs. GPU poor. Dog person. Coffee addict. Dyslexic. | GH: https://github.com/SkalskiP | HF: https://huggingface.co/SkalskiP
that's all the code you need to run detection and tracking
"how to track objects with SORT tracker" notebook: colab.research.google.com/github/robof...
"how to track objects with SORT tracker" notebook: colab.research.google.com/github/robof...
April 25, 2025 at 1:03 PM
that's all the code you need to run detection and tracking
"how to track objects with SORT tracker" notebook: colab.research.google.com/github/robof...
"how to track objects with SORT tracker" notebook: colab.research.google.com/github/robof...
it's build on top of supervision package allowing you to take advantage of all the tools we already created
April 25, 2025 at 1:03 PM
it's build on top of supervision package allowing you to take advantage of all the tools we already created
trackers v2.0.0 is out
combo object detectors from top model libraries with multi-object tracker of your choice
for now we support SORT and DeepSORT; more trackers coming soon
link: github.com/roboflow/tra...
combo object detectors from top model libraries with multi-object tracker of your choice
for now we support SORT and DeepSORT; more trackers coming soon
link: github.com/roboflow/tra...
April 25, 2025 at 1:03 PM
trackers v2.0.0 is out
combo object detectors from top model libraries with multi-object tracker of your choice
for now we support SORT and DeepSORT; more trackers coming soon
link: github.com/roboflow/tra...
combo object detectors from top model libraries with multi-object tracker of your choice
for now we support SORT and DeepSORT; more trackers coming soon
link: github.com/roboflow/tra...
you need to prepare your dataset in JSONL format; dataset includes three subsets: train, test, and valid
each subset contains images and annotations.jsonl file where each line of the file is a valid JSON object; each JSON object has three keys: image, prefix, and suffix
each subset contains images and annotations.jsonl file where each line of the file is a valid JSON object; each JSON object has three keys: image, prefix, and suffix
December 11, 2024 at 4:58 PM
you need to prepare your dataset in JSONL format; dataset includes three subsets: train, test, and valid
each subset contains images and annotations.jsonl file where each line of the file is a valid JSON object; each JSON object has three keys: image, prefix, and suffix
each subset contains images and annotations.jsonl file where each line of the file is a valid JSON object; each JSON object has three keys: image, prefix, and suffix
to limit the memory (VRAM) usage during the training, we can use LoRA, QLoRA, or freeze parts of the graph
December 11, 2024 at 4:58 PM
to limit the memory (VRAM) usage during the training, we can use LoRA, QLoRA, or freeze parts of the graph
PG2 offers 9 pre-trained models with sizes of 3B, 10B, and 28B parameters and resolutions of 224, 448, and 896 pixels.
to pick the right variant, you need to take into account the vision-language task you are solving, available hardware, amount of data, inference speed
to pick the right variant, you need to take into account the vision-language task you are solving, available hardware, amount of data, inference speed
December 11, 2024 at 4:58 PM
PG2 offers 9 pre-trained models with sizes of 3B, 10B, and 28B parameters and resolutions of 224, 448, and 896 pixels.
to pick the right variant, you need to take into account the vision-language task you are solving, available hardware, amount of data, inference speed
to pick the right variant, you need to take into account the vision-language task you are solving, available hardware, amount of data, inference speed
PG2 combines a SigLIP-So400m vision encoder with a Gemma 2 language model to process images and text. these tokens are then linearly projected and combined with input text tokens. Gemma 2 language model processes these combined tokens and generates output text tokens.
December 11, 2024 at 4:58 PM
PG2 combines a SigLIP-So400m vision encoder with a Gemma 2 language model to process images and text. these tokens are then linearly projected and combined with input text tokens. Gemma 2 language model processes these combined tokens and generates output text tokens.
new blog post is out: how to fine-tune PaliGemma 2
all I learned in a single blog
- PaliGemma 2 architecture
- dataset annotation and structure
- picking the right checkpoint
- memory optimization
- hyperparameters tuning
link: blog.roboflow.com/fine-tune-pa...
all I learned in a single blog
- PaliGemma 2 architecture
- dataset annotation and structure
- picking the right checkpoint
- memory optimization
- hyperparameters tuning
link: blog.roboflow.com/fine-tune-pa...
December 11, 2024 at 4:58 PM
new blog post is out: how to fine-tune PaliGemma 2
all I learned in a single blog
- PaliGemma 2 architecture
- dataset annotation and structure
- picking the right checkpoint
- memory optimization
- hyperparameters tuning
link: blog.roboflow.com/fine-tune-pa...
all I learned in a single blog
- PaliGemma 2 architecture
- dataset annotation and structure
- picking the right checkpoint
- memory optimization
- hyperparameters tuning
link: blog.roboflow.com/fine-tune-pa...
the paper suggests some nice strategies to increase the model's detection accuracy using fake boxes and <noise> special token; I plan to explore those in the coming days.
December 8, 2024 at 4:26 PM
the paper suggests some nice strategies to increase the model's detection accuracy using fake boxes and <noise> special token; I plan to explore those in the coming days.
PG2 offers 9 pre-trained models with sizes of 3B, 10B, and 28B parameters and resolutions of 224, 448, and 896 pixels.
we can see that PaliGemma2's object detection performance depends more on input resolution than model size. 3B 448 seems like a sweet spot.
we can see that PaliGemma2's object detection performance depends more on input resolution than model size. 3B 448 seems like a sweet spot.
December 8, 2024 at 4:26 PM
PG2 offers 9 pre-trained models with sizes of 3B, 10B, and 28B parameters and resolutions of 224, 448, and 896 pixels.
we can see that PaliGemma2's object detection performance depends more on input resolution than model size. 3B 448 seems like a sweet spot.
we can see that PaliGemma2's object detection performance depends more on input resolution than model size. 3B 448 seems like a sweet spot.
PG2 performs worse on the object detection task than specialized detectors; you can easily train a YOLOv11 model with 0.9 mAP on this dataset.
compared to PG1, it performs much better; datasets with a large number of classes were hard to fine-tune with previous version
compared to PG1, it performs much better; datasets with a large number of classes were hard to fine-tune with previous version
December 8, 2024 at 4:26 PM
PG2 performs worse on the object detection task than specialized detectors; you can easily train a YOLOv11 model with 0.9 mAP on this dataset.
compared to PG1, it performs much better; datasets with a large number of classes were hard to fine-tune with previous version
compared to PG1, it performs much better; datasets with a large number of classes were hard to fine-tune with previous version
PaliGemma2 for object detection on custom dataset
- used google/paligemma2-3b-pt-448 checkpoint
- trained on A100 with 40GB VRAM
- 1h of training
- 0.62 mAP on the validation set
colab with complete fine-tuning code: colab.research.google.com/github/robof...
- used google/paligemma2-3b-pt-448 checkpoint
- trained on A100 with 40GB VRAM
- 1h of training
- 0.62 mAP on the validation set
colab with complete fine-tuning code: colab.research.google.com/github/robof...
December 8, 2024 at 4:26 PM
PaliGemma2 for object detection on custom dataset
- used google/paligemma2-3b-pt-448 checkpoint
- trained on A100 with 40GB VRAM
- 1h of training
- 0.62 mAP on the validation set
colab with complete fine-tuning code: colab.research.google.com/github/robof...
- used google/paligemma2-3b-pt-448 checkpoint
- trained on A100 with 40GB VRAM
- 1h of training
- 0.62 mAP on the validation set
colab with complete fine-tuning code: colab.research.google.com/github/robof...
December 6, 2024 at 4:18 PM
PG2 offers 9 pre-trained models with sizes of 3B, 10B, and 28B parameters and resolutions of 224, 448, and 896 pixels.
it looks like OCR-related metrics ST-VQA, TallyQA, and TextCaps... benefit more from increased resolution than model size. that's why I went from 224 to 336.
it looks like OCR-related metrics ST-VQA, TallyQA, and TextCaps... benefit more from increased resolution than model size. that's why I went from 224 to 336.
December 6, 2024 at 4:18 PM
PG2 offers 9 pre-trained models with sizes of 3B, 10B, and 28B parameters and resolutions of 224, 448, and 896 pixels.
it looks like OCR-related metrics ST-VQA, TallyQA, and TextCaps... benefit more from increased resolution than model size. that's why I went from 224 to 336.
it looks like OCR-related metrics ST-VQA, TallyQA, and TextCaps... benefit more from increased resolution than model size. that's why I went from 224 to 336.
PaliGemma2 for image to JSON data extraction
- used google/paligemma2-3b-pt-336 checkpoint; I tried to make it happen with 224, but 336 performed a lot better
- trained on A100 with 40GB VRAM
- trained with LoRA
colab with complete fine-tuning code: colab.research.google.com/github/robof...
- used google/paligemma2-3b-pt-336 checkpoint; I tried to make it happen with 224, but 336 performed a lot better
- trained on A100 with 40GB VRAM
- trained with LoRA
colab with complete fine-tuning code: colab.research.google.com/github/robof...
December 6, 2024 at 4:18 PM
PaliGemma2 for image to JSON data extraction
- used google/paligemma2-3b-pt-336 checkpoint; I tried to make it happen with 224, but 336 performed a lot better
- trained on A100 with 40GB VRAM
- trained with LoRA
colab with complete fine-tuning code: colab.research.google.com/github/robof...
- used google/paligemma2-3b-pt-336 checkpoint; I tried to make it happen with 224, but 336 performed a lot better
- trained on A100 with 40GB VRAM
- trained with LoRA
colab with complete fine-tuning code: colab.research.google.com/github/robof...
how to prevent this in open-source projects?
- never allow github actions from first-time contributors.
- always require review for new contributors.
- never run important actions automatically via bots.
- protect release actions with unique cases and selected actors.
- never allow github actions from first-time contributors.
- always require review for new contributors.
- never run important actions automatically via bots.
- protect release actions with unique cases and selected actors.
December 5, 2024 at 9:21 PM
how to prevent this in open-source projects?
- never allow github actions from first-time contributors.
- always require review for new contributors.
- never run important actions automatically via bots.
- protect release actions with unique cases and selected actors.
- never allow github actions from first-time contributors.
- always require review for new contributors.
- never run important actions automatically via bots.
- protect release actions with unique cases and selected actors.
what happened?
malicious code was injected into the pypi deployment workflow (github action).
the source code itself wasn't infected. however, the resulting tar/wheel files were corrupted during the build process.
malicious code was injected into the pypi deployment workflow (github action).
the source code itself wasn't infected. however, the resulting tar/wheel files were corrupted during the build process.
December 5, 2024 at 9:21 PM
what happened?
malicious code was injected into the pypi deployment workflow (github action).
the source code itself wasn't infected. however, the resulting tar/wheel files were corrupted during the build process.
malicious code was injected into the pypi deployment workflow (github action).
the source code itself wasn't infected. however, the resulting tar/wheel files were corrupted during the build process.
popular computer vision package ultralytics (home of yolov8 and yolo11) was compromised.
a crypto miner was injected into versions 8.3.41 and 8.3.42.
link: github.com/ultralytics/...
a crypto miner was injected into versions 8.3.41 and 8.3.42.
link: github.com/ultralytics/...
December 5, 2024 at 9:21 PM
popular computer vision package ultralytics (home of yolov8 and yolo11) was compromised.
a crypto miner was injected into versions 8.3.41 and 8.3.42.
link: github.com/ultralytics/...
a crypto miner was injected into versions 8.3.41 and 8.3.42.
link: github.com/ultralytics/...
smart parking systems are just the beginning. roboflow workflows can be used for so much more. check out my clothes detection + sam2 + stabilityai inpainting workflow
November 27, 2024 at 5:43 PM
smart parking systems are just the beginning. roboflow workflows can be used for so much more. check out my clothes detection + sam2 + stabilityai inpainting workflow
custom python blocks in roboflow workflows are powerful. built a telegram bot connector for real-time alerts.
November 27, 2024 at 5:43 PM
custom python blocks in roboflow workflows are powerful. built a telegram bot connector for real-time alerts.
2 years ago i made a whole tutorial on coding this from scratch. now it's just 2 clicks in workflows.
link to my original line counting tutorial: www.youtube.com/watch?v=OS5q...
link to my original line counting tutorial: www.youtube.com/watch?v=OS5q...
November 27, 2024 at 5:43 PM
2 years ago i made a whole tutorial on coding this from scratch. now it's just 2 clicks in workflows.
link to my original line counting tutorial: www.youtube.com/watch?v=OS5q...
link to my original line counting tutorial: www.youtube.com/watch?v=OS5q...
I regret using OpenAI for license plate OCR.
expensive, slow, censors results, and refuses to read plates 20-30% of the time.
open-source models like florence2 are more reliable.
expensive, slow, censors results, and refuses to read plates 20-30% of the time.
open-source models like florence2 are more reliable.
November 27, 2024 at 5:43 PM
I regret using OpenAI for license plate OCR.
expensive, slow, censors results, and refuses to read plates 20-30% of the time.
open-source models like florence2 are more reliable.
expensive, slow, censors results, and refuses to read plates 20-30% of the time.
open-source models like florence2 are more reliable.
built a smart parking system with roboflow workflows.
- license plate detection and ocr
- object tracking with bytetrack.
- counting cars entering and leaving the lot.
- real-time alerts via telegram.
- license plate detection and ocr
- object tracking with bytetrack.
- counting cars entering and leaving the lot.
- real-time alerts via telegram.
November 27, 2024 at 5:43 PM
built a smart parking system with roboflow workflows.
- license plate detection and ocr
- object tracking with bytetrack.
- counting cars entering and leaving the lot.
- real-time alerts via telegram.
- license plate detection and ocr
- object tracking with bytetrack.
- counting cars entering and leaving the lot.
- real-time alerts via telegram.