Python4DataScience
LightNews LightNews
banner
python4data.science
Python4DataScience
@python4data.science
4.6K followers 4 following 76 posts
Teaching materials for the cusy training courses on a Python-based data science workflow: https://cusy.io/en/seminars
Posts Replies Media Videos
python4data.science
We analyse the memory consumption of our applications with memray and also monitor it continuously with pytest-memray: python4data.science/en/latest/pe...
#Python
Memray
Memory usage is difficult to control in Python projects because the language does not explicitly indicate where memory is allocated, module imports can significantly increase consumption, and it is...
python4data.science
February 11, 2026 at 6:58 AM
python4data.science
Git 2.53 provides faster insights into the repository structure with 'git repo structure'. However, to better understand this, it is helpful to be more familiar with the Git data and storage models: www.python4data.science/en/latest/pr...
#Git
Git Internals
So far, we have looked at how you can use Git to manage the different states of your code. Now we want to show you the data and storage models that underlie Git. Data Model: You will be able to use...
www.python4data.science
February 9, 2026 at 6:50 AM
python4data.science
The section on performance measurements and finding bottlenecks has been significantly expanded to include cProfile/profiling.tracing, tprof, and profiling.sampling/Tachyon: www.python4data.science/en/latest/pe...
#Python #Performance
Performance
Python can be used to write and test code quickly because it is an interpreted language that types dynamically. However, these are also the reasons it is slow when performing simple tasks repeatedl...
www.python4data.science
January 19, 2026 at 5:51 AM
python4data.science
We have updated the documentation section with references to README, CONTRIBUTING, CHANGELOG, etc.
python-basics-tutorial.readthedocs.io/en/latest/do...
#Python #Documentation
Document
In order for your software package to be useful, documentation is required that describes how your software can be installed, operated, used and improved: Those who want to use your package need in...
python-basics-tutorial.readthedocs.io
January 16, 2026 at 4:38 PM
python4data.science
We have updated the FastAPI extensions. It is very surprising to us that millions of extensions are being downloaded that have not been updated for over a year.
www.python4data.science/en/latest/da...
@fastapi.tiangolo.com
#Python #FastAPI #REST
Extensions
Administration: SQLAlchemy Admin for Starlette/FastAPI, Flexible admin interface for SQLAlchemy models. Downloads Contributors Commit activity Licence,, Piccolo Admin, Simple but powerful admin int...
www.python4data.science
January 7, 2026 at 2:39 PM
Reposted by Python4DataScience
veit-schiele.de
I took a look at the changes coming with Python 3.15 – and I can't wait to put them to productive use. I've already updated our tutorials:
• Performance measurements: www.python4data.science/en/latest/pe...
• Tachyon: www.python4data.science/en/latest/pe...
#Python
Unicode and character encodings
Special characters and escape sequences:\n stands for the newline character and\t for the tab character. Character sequences that begin with a backslash and are used to represent other characters a...
python-basics-tutorial.readthedocs.io
December 21, 2025 at 2:27 PM
python4data.science
We have updated the section on pytest with many exciting use cases
* on command line options
* on generating markers
* and on parameterising exceptions
python-basics-tutorial.readthedocs.io/en/latest/te...
#Python #Testing #pytest
pytest
pytest is an alternative to Python’s Unittest module that simplifies testing even further. pytest automatically recognises tests based on filenames and functions that start with test_, while unitte...
python-basics-tutorial.readthedocs.io
November 17, 2025 at 6:18 AM
python4data.science
We have updated our tutorial to data management with DVC. It also allows you to create lightweight data science and data modelling workflows and execute them in a parameterised manner: www.python4data.science/en/latest/pr...
#Data #Versioncontrol #Git #DataScience #Modeling #Python
Precision-Recall-Curve comparison between workspace and HEAD Receiver operating characteristic (ROC) comparison between workspace and HEAD Confusion Matrix comparison between workspace and HEAD
October 21, 2025 at 12:02 PM
python4data.science
We have now described how to create a configuration for Claude Code so that it uses uv reliably: python4data.science/en/latest/pr...
#ClaudeCode #Python #Packaging #uv
Configuring Claude Code for uv
How do we configure Claude Code to automatically use uv instead of pip for Python package management? Claude Code uses CLAUDE.md files to configure your project’s storage and context, ensuring a co...
python4data.science
September 24, 2025 at 12:52 PM
python4data.science
Since we have recently been asked frequently whether pandas is slow and whether we should use Polars, Dask or DuckDB instead, we have now provided an initial overview of the various technologies: www.python4data.science/en/latest/wo...
#Python #Performance #DuckDB
pandas
pandas is a Python library for data analysis that has become very popular in recent years. On the website, pandas is described thus: „pandas is a fast, powerful, flexible and easy to use open sourc...
www.python4data.science
September 23, 2025 at 12:53 PM
python4data.science
We have now completely switched to pathlib: python-basics-tutorial.readthedocs.io/en/latest/sa...
#Python
Files and directories
pathlib implements path operations using pathlib.PurePath and pathlib.Path objects. The os and os.path modules, on the other hand, offer functions that work at a low level with str- and bytes which...
python-basics-tutorial.readthedocs.io
September 16, 2025 at 6:07 AM
Reposted by Python4DataScience
veit-schiele.de
We have finally documented Ruff – the tool greatly simplifies static code analysis for Python projects: www.python4data.science/en/latest/pr...
#Python #Ruff
Ruff
Ruff is an extremely fast Python linter and code formatter written in Rust that can enforce the rules of flake8, isort, perflint, Black, Bandit, and others. In total, Ruff can check over 800 rules....
www.python4data.science
August 25, 2025 at 2:30 PM
python4data.science
We have now updated our packaging tutorial to include PEP 639, which enables SPDX-compliant licensing: python-basics-tutorial.readthedocs.io/en/latest/pa...
#Python #Packaging #SPDX #Licensing
Creating a distribution package
Distribution Packages are archives that can be uploaded to a package index such as pypi.org and installed with pip. Structure: A minimal distribution package can look like this, for example: pyproj...
python-basics-tutorial.readthedocs.io
August 22, 2025 at 4:07 PM
python4data.science
We have added a section on additional JSON tools: www.python4data.science/en/latest/da...
#Python #JSON
JSON
Overview:,,, Data structure support,+-, JSON supports array and map or object structures and many different data types including strings, numbers, boolean, null etc., but no date formats. However, ...
www.python4data.science
August 20, 2025 at 1:47 PM
python4data.science
We have added several geopython libraries: www.python4data.science/en/latest/da...
#Python #Geospatial #GeoPython
Geodata
File formats: PMTiles: PMTiles is a general format for tile data addressed by Z/X/Y coordinates. This can be cartographic vector tiles, remote sensing data, JPEG images or similar. HTTP Range Reque...
www.python4data.science
August 1, 2025 at 10:50 AM
python4data.science
We have significantly expanded the section on licences for AI systems: www.python4data.science/en/latest/pr...
#AI #Licensing #OpenData #OpenSource
Licensing
In order for others to use your software, it should have one or more licences that describe the terms of use. Otherwise, it is likely to be protected by copyright. Authors are those who have origin...
www.python4data.science
July 31, 2025 at 3:39 PM
Reposted by Python4DataScience
spackpm.bsky.social
💥Spack v1.0 is out!💥

This is a huge milestone. We reworked the core to add compiler dependencies, and we're introducing a stable package API.

🚀1.0 also adds concurrent builds, better includes, and much more -- read it all in the release notes!

github.com/spack/spack/...
github.com
July 20, 2025 at 10:45 AM
python4data.science
The XKCD comic on reproducible scientific results fits perfectly with our tutorial 🧐 😉
www.python4data.science/en/latest/pr...
XKCD #3117: Replication Crisis
July 19, 2025 at 12:12 PM
python4data.science
Almost more significant than the success of #Python is the growth of #Jupyter #Notebooks: “Data scientists and machine learning researchers commonly use the #OpenSource application for #MachineLearning, #DataViz, and more.”
jupyter-tutorial.readthedocs.io/en/latest/in...
Graph from GitHub’s Octoverse 2024 report showing a spike in utilization of Jupyter Notebooks across GitHub. This is calculated by looking at the distinct number of public repositories with at least one Jupyter Notebook by the year the repository was created. Since 2016, we have seen this number surge from near zero to more than 1.5 million repositories using Jupyter Notebooks.
July 15, 2025 at 7:53 AM
python4data.science
We have added a section on protomaps to our PyViz tutorial. Protomaps makes map visualisations so much easier.
pyviz-tutorial.readthedocs.io/en/latest/pr...
#Protomaps #Geography #World #Map @protomaps.com
Protomaps
Protomaps is an open source project for the creation and use of vector maps. It was developed as a lightweight alternative to conventional map providers and offers a number of advantages. Open Sour...
pyviz-tutorial.readthedocs.io
May 21, 2025 at 5:34 AM
python4data.science
We have expanded the section on geodata to include the most common (tile) file formats: www.python4data.science/en/latest/da...
#Geography #GIS
Geodata
File formats: PMTiles: PMTiles is a general format for tile data addressed by Z/X/Y coordinates. This can be cartographic vector tiles, remote sensing data, JPEG images or similar. HTTP Range Reque...
www.python4data.science
May 15, 2025 at 12:32 PM
python4data.science
We have updated our Python Basics tutorial to describe the guidelines for docstrings in more detail:
python-basics-tutorial.readthedocs.io/en/latest/do...
#Python #Documentation #DX
Docstrings
With the Sphinx extension sphinx.ext.autodoc, docstrings can also be included in the documentation. The following directives can be specified … for function-like objects: … for data and attributes:...
python-basics-tutorial.readthedocs.io
April 7, 2025 at 6:23 AM
python4data.science
We have expanded the section on open source hardware licences to include the TAPR and Solderpad Hardware Licence: www.python4data.science/en/latest/pr...
#OpenSource #Hardware #Licence
Licensing
In order for others to use your software, it should have one or more licences that describe the terms of use. Otherwise, it is likely to be protected by copyright. Authors are those who have origin...
www.python4data.science
April 3, 2025 at 8:19 AM
python4data.science
thoughtworks Technology Radar has now also adopted the tools uv and Renovate: www.thoughtworks.com/radar
Technology Radar | Guide to technology landscape
The Technology Radar is an opinionated guide to today's technology landscape. Read the latest here.
www.thoughtworks.com
April 2, 2025 at 3:21 PM
python4data.science
We have expanded our section on GitLab CI/CD pipelines with examples of
• GitLab Pages
• npm deployments with rsync
• building Docker containers
• multi-arch images with Buildah
• migrating GitHub Actions
www.python4data.science/en/latest/pr...
#GitLab #CICD #DevOps #DX
GitLab CI/CD
GitLab CI/CD can automatically build, test, deploy and monitor your applications during iterative code changes. This reduces the risk that you will develop new code based on buggy previous versions...
www.python4data.science
March 28, 2025 at 7:00 AM