@marcogorelli.bsky.social
Senior Software Engineer at Quansight Labs
Pretty focused on Narwhals and Polars these days, generally interested in dataframes and time series
Pretty focused on Narwhals and Polars these days, generally interested in dataframes and time series
✨ Narwhals now supports Daft!
🔌Just install the `narwhals-daft` plugin, and then `narwhals.from_native` will accept Daft DataFrame!
💡 No extra dependencies requires, computation will remain native (and distributed, and lazy!) until you call `collect`
🔌Just install the `narwhals-daft` plugin, and then `narwhals.from_native` will accept Daft DataFrame!
💡 No extra dependencies requires, computation will remain native (and distributed, and lazy!) until you call `collect`
November 10, 2025 at 3:33 PM
✨ Narwhals now supports Daft!
🔌Just install the `narwhals-daft` plugin, and then `narwhals.from_native` will accept Daft DataFrame!
💡 No extra dependencies requires, computation will remain native (and distributed, and lazy!) until you call `collect`
🔌Just install the `narwhals-daft` plugin, and then `narwhals.from_native` will accept Daft DataFrame!
💡 No extra dependencies requires, computation will remain native (and distributed, and lazy!) until you call `collect`
😲 Wes McKinney mentioned Narwhals on a podcast
November 4, 2025 at 5:01 PM
😲 Wes McKinney mentioned Narwhals on a podcast
✨ New @pola.rs feature: `list.agg`
💡 Compute any Expr aggregation on a list column, just use `pl.element()`
🔜 Coming soon, in version 1.35!
💡 Compute any Expr aggregation on a list column, just use `pl.element()`
🔜 Coming soon, in version 1.35!
October 23, 2025 at 4:09 PM
✨ New @pola.rs feature: `list.agg`
💡 Compute any Expr aggregation on a list column, just use `pl.element()`
🔜 Coming soon, in version 1.35!
💡 Compute any Expr aggregation on a list column, just use `pl.element()`
🔜 Coming soon, in version 1.35!
🥧🧵 Narwhals now tests against Python 3.14 (pi-Python!) free-threading build
👉 For more on free-threaded Python, my colleague Nathan's blog post is an excellent read, link in comments
👉 For more on free-threaded Python, my colleague Nathan's blog post is an excellent read, link in comments
October 20, 2025 at 6:52 PM
🥧🧵 Narwhals now tests against Python 3.14 (pi-Python!) free-threading build
👉 For more on free-threaded Python, my colleague Nathan's blog post is an excellent read, link in comments
👉 For more on free-threaded Python, my colleague Nathan's blog post is an excellent read, link in comments
✨ Narwhals now supports `format` across all backends
🧵 Create new string columns using placeholders and expressions
🌊🦄 Yet another piece of the @pola.rs API brought to pandas, @duckdb.org , PySpark, PyArrow, and more!
🧵 Create new string columns using placeholders and expressions
🌊🦄 Yet another piece of the @pola.rs API brought to pandas, @duckdb.org , PySpark, PyArrow, and more!
October 16, 2025 at 8:22 PM
✨ Narwhals now supports `format` across all backends
🧵 Create new string columns using placeholders and expressions
🌊🦄 Yet another piece of the @pola.rs API brought to pandas, @duckdb.org , PySpark, PyArrow, and more!
🧵 Create new string columns using placeholders and expressions
🌊🦄 Yet another piece of the @pola.rs API brought to pandas, @duckdb.org , PySpark, PyArrow, and more!
✨ NumPy's MaskedArray class is now type-complete
🏷️ Enjoy improved code completion and typing!
🙏 Thank you to Quansight Labs and Meta for having facilicated and funded the work (blog post in comments)
🏷️ Enjoy improved code completion and typing!
🙏 Thank you to Quansight Labs and Meta for having facilicated and funded the work (blog post in comments)
October 13, 2025 at 5:47 PM
✨ NumPy's MaskedArray class is now type-complete
🏷️ Enjoy improved code completion and typing!
🙏 Thank you to Quansight Labs and Meta for having facilicated and funded the work (blog post in comments)
🏷️ Enjoy improved code completion and typing!
🙏 Thank you to Quansight Labs and Meta for having facilicated and funded the work (blog post in comments)
🌝 The next @astropy.org release will support generic dataframe tables (like @pola.rs )
🌊🦄 Powered by Narwhals
🌊🦄 Powered by Narwhals
October 3, 2025 at 4:37 PM
🌝 The next @astropy.org release will support generic dataframe tables (like @pola.rs )
🌊🦄 Powered by Narwhals
🌊🦄 Powered by Narwhals
💡 Life-pro-tip: enable inline type hints
🏷️ Your editor will display inferred types for variables you haven't annotated
🕵️ Debugging those red squiggly lines will become much easier
🏷️ Your editor will display inferred types for variables you haven't annotated
🕵️ Debugging those red squiggly lines will become much easier
September 12, 2025 at 6:14 PM
💡 Life-pro-tip: enable inline type hints
🏷️ Your editor will display inferred types for variables you haven't annotated
🕵️ Debugging those red squiggly lines will become much easier
🏷️ Your editor will display inferred types for variables you haven't annotated
🕵️ Debugging those red squiggly lines will become much easier
✨🐼 Pandas 3.0 will have `pd.col` syntax
💡 You can use it for clean chaining in `assign` and `loc`
🚀 It's happening, the PR just got merged!
💡 You can use it for clean chaining in `assign` and `loc`
🚀 It's happening, the PR just got merged!
August 22, 2025 at 5:28 PM
✨🐼 Pandas 3.0 will have `pd.col` syntax
💡 You can use it for clean chaining in `assign` and `loc`
🚀 It's happening, the PR just got merged!
💡 You can use it for clean chaining in `assign` and `loc`
🚀 It's happening, the PR just got merged!
✨ New @pola.rs feature - you can now chain `name` expressions!
🌊🦄 As of Narwhals 2.0 (yes, we've released 2.0 🚀), this feature is available there as well! Little example below with our friend @duckdb.org
🌊🦄 As of Narwhals 2.0 (yes, we've released 2.0 🚀), this feature is available there as well! Little example below with our friend @duckdb.org
August 12, 2025 at 5:28 PM
✨ New @pola.rs feature - you can now chain `name` expressions!
🌊🦄 As of Narwhals 2.0 (yes, we've released 2.0 🚀), this feature is available there as well! Little example below with our friend @duckdb.org
🌊🦄 As of Narwhals 2.0 (yes, we've released 2.0 🚀), this feature is available there as well! Little example below with our friend @duckdb.org
💡 "There's nothing for permanent than a temporary fix"
Case in point, this pandas comment which was added [checks git log] ... 12 years ago 🙈
Case in point, this pandas comment which was added [checks git log] ... 12 years ago 🙈
August 7, 2025 at 8:16 PM
💡 "There's nothing for permanent than a temporary fix"
Case in point, this pandas comment which was added [checks git log] ... 12 years ago 🙈
Case in point, this pandas comment which was added [checks git log] ... 12 years ago 🙈
Reposted
"If the U.S. can't get its greenhouse emissions under control, they need to face real consequences" said the rest of the world.
"If Canada can't get these wildfires under control, they need to face real consequences," said Wisconsin state Rep. Calvin Callahan in a news release Wednesday. "We won't sit back while our air becomes a health hazard."
More Republican lawmakers demand legal action over Canadian wildfire smoke
The Republican lawmakers called for an investigation of Canada's wildfire management practices and for potential remedies under international law, saying Canada must face "real consequences."
www.nationalobserver.com
August 7, 2025 at 7:57 PM
"If the U.S. can't get its greenhouse emissions under control, they need to face real consequences" said the rest of the world.
💡 Today in open source: a 17-year-old NumPy commit promising that a new feature will be coming soon
Any day now..
Any day now..
July 29, 2025 at 4:35 PM
💡 Today in open source: a 17-year-old NumPy commit promising that a new feature will be coming soon
Any day now..
Any day now..
💡 Solution: use Narwhals for universal dataframe support 🌊🦄
July 18, 2025 at 10:56 AM
💡 Solution: use Narwhals for universal dataframe support 🌊🦄
July 15, 2025 at 6:41 PM
🎢 When the Q&A goes off-script 🤣
🇯🇵 Japanese math rock, 🅿️ porpoises and unfortunate aliases, and 🎤 Taylor Swift:
🥈All of these unexpectedly came up in this fun session, which is currently the 2nd most viewed of the conference
🙏 Thanks @pydatalondon.bsky.social for inviting me!
🇯🇵 Japanese math rock, 🅿️ porpoises and unfortunate aliases, and 🎤 Taylor Swift:
🥈All of these unexpectedly came up in this fun session, which is currently the 2nd most viewed of the conference
🙏 Thanks @pydatalondon.bsky.social for inviting me!
Marco Gorelli - How Narwhals brings Polars, DuckDB, PyArrow, & pandas together | PyData London 25
YouTube video by PyData
youtu.be
July 9, 2025 at 2:41 PM
🎢 When the Q&A goes off-script 🤣
🇯🇵 Japanese math rock, 🅿️ porpoises and unfortunate aliases, and 🎤 Taylor Swift:
🥈All of these unexpectedly came up in this fun session, which is currently the 2nd most viewed of the conference
🙏 Thanks @pydatalondon.bsky.social for inviting me!
🇯🇵 Japanese math rock, 🅿️ porpoises and unfortunate aliases, and 🎤 Taylor Swift:
🥈All of these unexpectedly came up in this fun session, which is currently the 2nd most viewed of the conference
🙏 Thanks @pydatalondon.bsky.social for inviting me!
🥳 The next version of @mlflow.org will support @pola.rs !
😩 Slightly disappointingly, they do so by converting to pandas
🌊🦄 Could Narwhals help make MLFlow gain native support for Polars? Link to the discussion below 👇
😩 Slightly disappointingly, they do so by converting to pandas
🌊🦄 Could Narwhals help make MLFlow gain native support for Polars? Link to the discussion below 👇
July 1, 2025 at 3:49 PM
🥳 The next version of @mlflow.org will support @pola.rs !
😩 Slightly disappointingly, they do so by converting to pandas
🌊🦄 Could Narwhals help make MLFlow gain native support for Polars? Link to the discussion below 👇
😩 Slightly disappointingly, they do so by converting to pandas
🌊🦄 Could Narwhals help make MLFlow gain native support for Polars? Link to the discussion below 👇
🙀 It's scary to merge a 2,000+ line PR!
😇 But, this one was so thorough that we just had to ship it
🏆 Thanks to an incredible effort by Ryan Whitten, Narwhals supports Ibis!
💡You can now use Narwhals with all of Ibis' backends, plus some extra, whilst keeping @pola.rs expressive API!
😇 But, this one was so thorough that we just had to ship it
🏆 Thanks to an incredible effort by Ryan Whitten, Narwhals supports Ibis!
💡You can now use Narwhals with all of Ibis' backends, plus some extra, whilst keeping @pola.rs expressive API!
June 19, 2025 at 6:59 PM
🙀 It's scary to merge a 2,000+ line PR!
😇 But, this one was so thorough that we just had to ship it
🏆 Thanks to an incredible effort by Ryan Whitten, Narwhals supports Ibis!
💡You can now use Narwhals with all of Ibis' backends, plus some extra, whilst keeping @pola.rs expressive API!
😇 But, this one was so thorough that we just had to ship it
🏆 Thanks to an incredible effort by Ryan Whitten, Narwhals supports Ibis!
💡You can now use Narwhals with all of Ibis' backends, plus some extra, whilst keeping @pola.rs expressive API!
✨ PySpark 4.0 is out
🌊🦄 And it's already supported by Narwhals, including PySpark connect (thanks Francesco Bruzzesi!)
🌊🦄 And it's already supported by Narwhals, including PySpark connect (thanks Francesco Bruzzesi!)
May 26, 2025 at 7:09 PM
✨ PySpark 4.0 is out
🌊🦄 And it's already supported by Narwhals, including PySpark connect (thanks Francesco Bruzzesi!)
🌊🦄 And it's already supported by Narwhals, including PySpark connect (thanks Francesco Bruzzesi!)
✨ Narwhals now supports `fill_null` with forwards and backwards strategies
🎬 Here's a demo of a fill-forward function which supports PySpark, @pola.rs , @duckdb.org , pandas, cuDF, and more!
🎁 Bonus: curious about what this looks like as DuckDB SQL? We got you covered 😎
🎬 Here's a demo of a fill-forward function which supports PySpark, @pola.rs , @duckdb.org , pandas, cuDF, and more!
🎁 Bonus: curious about what this looks like as DuckDB SQL? We got you covered 😎
May 22, 2025 at 2:55 PM
✨ Narwhals now supports `fill_null` with forwards and backwards strategies
🎬 Here's a demo of a fill-forward function which supports PySpark, @pola.rs , @duckdb.org , pandas, cuDF, and more!
🎁 Bonus: curious about what this looks like as DuckDB SQL? We got you covered 😎
🎬 Here's a demo of a fill-forward function which supports PySpark, @pola.rs , @duckdb.org , pandas, cuDF, and more!
🎁 Bonus: curious about what this looks like as DuckDB SQL? We got you covered 😎
✨ Narwhals now supports PySpark
🎇 If you have a dataframe-agnostic function, you can pass in `pyspark.sql.DataFrame`
📈 Here's a demo, made with @marimo.io
🎁 BONUS feature: combine with SQLFrame, to generate SQL from @pola.rs syntax 🪄
🎇 If you have a dataframe-agnostic function, you can pass in `pyspark.sql.DataFrame`
📈 Here's a demo, made with @marimo.io
🎁 BONUS feature: combine with SQLFrame, to generate SQL from @pola.rs syntax 🪄
May 9, 2025 at 1:41 PM
✨ Narwhals now supports PySpark
🎇 If you have a dataframe-agnostic function, you can pass in `pyspark.sql.DataFrame`
📈 Here's a demo, made with @marimo.io
🎁 BONUS feature: combine with SQLFrame, to generate SQL from @pola.rs syntax 🪄
🎇 If you have a dataframe-agnostic function, you can pass in `pyspark.sql.DataFrame`
📈 Here's a demo, made with @marimo.io
🎁 BONUS feature: combine with SQLFrame, to generate SQL from @pola.rs syntax 🪄
🛠️ NVIDIA's cuDF library now runs the Narwhals test suite as part of its CI process
🙀
🙀
May 7, 2025 at 12:02 PM
🛠️ NVIDIA's cuDF library now runs the Narwhals test suite as part of its CI process
🙀
🙀
✨ New Polars feature: `is_business_day`
🌴 Use this to determine which dates fall on holidays, blazingly fast
🇩🇪 Using Germany as an example because I was just there to give a talk at @pyconde.bsky.social - thanks for having me and for putting on such a fun event!
🌴 Use this to determine which dates fall on holidays, blazingly fast
🇩🇪 Using Germany as an example because I was just there to give a talk at @pyconde.bsky.social - thanks for having me and for putting on such a fun event!
April 25, 2025 at 5:12 PM
✨ New Polars feature: `is_business_day`
🌴 Use this to determine which dates fall on holidays, blazingly fast
🇩🇪 Using Germany as an example because I was just there to give a talk at @pyconde.bsky.social - thanks for having me and for putting on such a fun event!
🌴 Use this to determine which dates fall on holidays, blazingly fast
🇩🇪 Using Germany as an example because I was just there to give a talk at @pyconde.bsky.social - thanks for having me and for putting on such a fun event!