Eric Book
LightNews LightNews
erc-bk.fosstodon.org.ap.brid.gy
Eric Book
@erc-bk.fosstodon.org.ap.brid.gy
8 followers 7 following 79 posts
Pathologically curious about most things, but this space is mostly focused on data science and engineering.
#tfr

[bridged from https://fosstodon.org/@erc_bk on the fediverse by https://fed.brid.gy/ ]
Posts Replies Media Videos
erc-bk.fosstodon.org.ap.brid.gy
How spatial sampling methods affect the variance of your mean estimate when spatial autocorrelation is present.
https://dosull.github.io/posts/2025-11-14-gia-chapter-2A-spatial-autocorrelation/
Shows four density charts. Each chart represents a sampling method: poisson, grid, evenly spaced, and clustered.. Each chart shows two densities: random (no spatial autocorrelation) and smoothed (spatial autocorrelation). Grid and evenly spaced show too narrow variance estimates while clustered shows too wide of a variance estimate.
November 16, 2025 at 12:37 AM
erc-bk.fosstodon.org.ap.brid.gy
Just as I've started to get a pretty comfortable with all the coordinate representations, projections, etc., I see this. If I would've came across an older dataset with this before now, I'm not sure what I would've done. Tried an internet search out of […]

[Original post on fosstodon.org]
Shows text with geographical coordinates formatted in a number/character format, e.g. 51d56'N.
October 30, 2025 at 8:52 PM
erc-bk.fosstodon.org.ap.brid.gy
Hmm — an htmlwidget serializer. I'd like to see that out in the wild. #rstats
Shows plumber2 documentation with all available serializers listed. The htmlwidget serializer has been circled.
October 28, 2025 at 3:00 PM
erc-bk.fosstodon.org.ap.brid.gy
@erc_bk Decided to improve upon this a bit (with some Claude help). The code is in this gist, https://gist.github.com/ercbk/8d8803a3b22d21a4a871dc8ae967cd6a. Not as comprehensive as skimr, but takes advantage of arrow's lazy evaluation. This only uses around 214MB of RAM. #rstats
R code that shows the summary stats output of the skim_arrow function. Code for this function is at the gist link.
October 15, 2025 at 3:17 PM
erc-bk.fosstodon.org.ap.brid.gy
Didn't know that {skimr::skim} worked an arrow object. It's pretty cool, but it hits your RAM pretty hard. That's only a 70MB directory of parquet files, and my RAM usage went up by ~1.3GB. #rstats Code: https://ray.so/fCoXwso
NO2 pollution dataset. Schema of the dataset is shown and the descriptive output from skimr::skim. library(arrow) ds <- open_dataset("C:/Users/user/Documents/R/Data/no2/E1a") schema(ds) skimr::skim(ds)
October 14, 2025 at 8:26 PM
erc-bk.fosstodon.org.ap.brid.gy
I feel like ChatGPT is taking a jab at other types of models here.
Images shows the response of ChatGPT to a query about map artifacts generated by a random forest model. It describes some of the predictions of the model as hallucinations which is a common critique of large language models.
May 8, 2025 at 3:50 PM
erc-bk.fosstodon.org.ap.brid.gy
Another damn dynamic website bit me today. Tried to scrape some data with a POST request, but it was sneakily generated by secondary GET request that was triggered by a button I didn't even click.
April 8, 2025 at 2:51 AM
erc-bk.fosstodon.org.ap.brid.gy
Second time seeing these diamond plots. There's a little bit of visual friction with the rotated grid, but I don't think it's too bad of a trade-off for when you have long axis labels. Angled tick labels and region labeling/color coding can help as well […]

[Original post on fosstodon.org]
Scatter plot from 538 article that's rotated 45 degrees so that it's a diamond shape with 4 regions labelled. Subject matter is moderate voters and their positions on economics and immigration. Scatter plot from Owen Phillips substack that's rotated 45 degrees so that it's a diamond shape with 4 regions labelled and color coded along with rotated tick labels. Subject matter is 2024 NFL offensive and defensive EPA per play.
October 23, 2024 at 6:12 PM
erc-bk.fosstodon.org.ap.brid.gy
‘The data on extreme human ageing is rotten from the inside out’ – Ig Nobel winner Saul Justin Newman
https://theconversation.com/the-data-on-extreme-human-ageing-is-rotten-from-the-inside-out-ig-nobel-winner-saul-justin-newman-239023
I started getting interested in this topic when I debunked a couple of papers in Nature and Science about extreme ageing in the 2010s. In general, the claims about how long people are living mostly don’t stack up. I’ve tracked down 80% of the people aged over 110 in the world (the other 20% are from countries you can’t meaningfully analyse). Of those, almost none have a birth certificate. In the US there are over 500 of these people; seven have a birth certificate. Even worse, only about 10% have a death certificate. The epitome of this is blue zones, which are regions where people supposedly reach age 100 at a remarkable rate. For almost 20 years, they have been marketed to the public. They’re the subject of tons of scientific work, a popular Netflix documentary, tons of cookbooks about things like the Mediterranean diet, and so on. Okinawa in Japan is one of these zones. There was a Japanese government review in 2010, which found that 82% of the people aged over 100 in Japan turned out to be dead. The secret to living to 110 was, don’t register your death. The Japanese government has run one of the largest nutritional surveys in the world, dating back to 1975. From then until now, Okinawa has had the worst health in Japan. They’ve eaten the least vegetables; they’ve been extremely heavy drinkers. ‘What do you think explains most of the faulty data? It varies. In Okinawa, the best predictor of where the centenarians are is where the halls of records were bombed by the Americans during the war. That’s for two reasons. If the person dies, they stay on the books of some other national registry, which hasn’t confirmed their death. Or if they live, they go to an occupying government that doesn’t speak their language, works on a different calendar and screws up their age. According to the Greek minister that hands out the pensions, over 9,000 people over the age of 100 are dead and collecting a pension at the same time. In Italy, some 30,000 “living” pension recipients were found to be dead in 1997. Regions where people most often reach 100-110 years old are the ones where there’s the most pressure to commit pension fraud, and they also have the worst records. For example, the best place to reach 105 in England is Tower Hamlets. It has more 105-year-olds than all of the rich places in England put together. It's closely followed by downtown Manchester, Liverpool and Hull. Yet these places have the lowest frequency of 90-year-olds and are rated by the UK as the worst places to be an old person. The oldest man in the world, John Tinniswood, supposedly aged 112, is from a very rough part of Liverpool. The easiest explanation is that someone has written down his age wrong at some point.
September 16, 2024 at 2:41 PM
erc-bk.fosstodon.org.ap.brid.gy
nixla's TimeGPT is out of beta, and they even made a R package for it. It'll be interesting to see if it lives up to the hype. I like nixla, but I'm skeptical of any forecasting algos involving transformers. (Pics in reverse order. Sorry 😬) Thread: https://x.com/nixtlainc/status/1813917342965117015
Pic of Twitter thread. Claims that TimeGPT-1 ranks first among foundational models in accuracy and speed. Links to documentation. We are so grateful for our amazing community and all you've shared with us that helped us get to this release and broaden access to forecasting! [ Highlights in the new version: @ New Features: Introduction of timegpt-1-long-horizon for long-term forecasts. Enhanced uncertainty quantification and probabilistic features. Advanced model fine-tuning with diverse loss functions . Support for distributed computing (Spark, Ray, and Dask). Improved accuracy for hourly data. f Improvements: APl now supports much bigger files for large scale forecasting. Faster inference speed, higher reliability with retries and improved error messages. & TimeGPT in R: nixtlar now available on CRAN. i’ TimeGPT for Excel (Beta): forecast and perform anomaly detection in a few clicks in your Excel spreadsheets. TimeGEN in Azure Al Studio: TimeGEN-1 is an optimized version of TimeGPT for the Azure Al model catalog. It is offered as a Model as a Service (MaaS). & Documentation: Revamped layout, new tutorials and use cases including What-if scenarios, intermittent demand, web traffic and energy demand. @ nixtla @ @nixtlainc /7 We're thrilled to announce the release of the stable version of e This release takes TimeGPT out of beta and makes it broadly available to anyone for fast and easy time-series forecasting. We've added features and improvements, updated documentation, done more benchmarking, added an R package, created an Excel plug-in (in beta), made it available on Azure Al Studio as TimeGEN-1, and made it easy for anyone to get started with our free trial. During the beta phase, we were excited to see that TimeGPT and TimeGEN are working well for people across a wide set of use cases, helping with their forecasting by improving accuracy while reducing the complexity of deployment and time to results. For example, one of the biggest retailers in the world uses us for financial forecasting, a European energy distributor is using Nixtla software for advanced energy prediction models, a non-profit is using TimeGPT to enhance their ability to predict and respond to potential public health threats, and a small retail company is using TimeGPT to improve their staff cost planning. Since implementing TimeGEN, a major manufacturing organization has seen a significant improvement in forecasting accuracy, with errors reduced by nearly 30% on average during periods of high volatility, allowing them to streamline operations and optimize supply-chain efficiency.
July 18, 2024 at 6:06 PM
erc-bk.fosstodon.org.ap.brid.gy
I feel like I'm dreaming. R code of the proposed algorithm within the paper and a Quarto website with the resources. https://arxiv.org/abs/2406.06851
June 13, 2024 at 1:09 PM