TIL that over the years, I have added or updated 34,724 unique public authorities on WhatDoTheyKnow. That's 74% of the total. It's all my fault 😅

The decision by a number of local councils to run adverts on their websites didn't quite sit right with me. I couldn't quite work out why until I saw an advert for a credit card on the crisis loan page.

When I find failed redactions or accidental releases of PII in FOI responses I will usually notify the authority. The response is mixed, but far too often there is just silence. Often the only way I know they've got my message is by seeing if the file has disappeared. You'd think they'd want to know

Microsoft seems to have pulled the larger vibevoice TTS model from huggingface, and the github repo 404s github.com/microsoft/Vi.... It's not been out for long, but I can't be alone in having both downloaded and it's MIT licensed, so there is nothing to stop mirrors. I wonder what the issue is? 🤔

The SSD drive of shame has hit 2,752,077 files. Of course I haven't plugged in a new one rather than face clearing it 😅

As a bit of fun, I used some of the WDTK keywords that I created last week to create synthetic FOI requests using mistral-small. I then finetuned SmolLM2-360M-Instruct on those outputs to generate requests from 3 keywords and the authority name: huggingface.co/HMC83/reques...

I've uploaded descriptive keywords for over 1 million public FOI requests. I have left it at with request_id, keywords and the name of the public authority for now: github.com/FOIMonkey/fo...

As a side effect of this, I have generated keywords for over 1 million FOI requests. Figuring out how to do that well in the most lightweight way was a journey in and of itself. I've not looked yet, but combined with authority and outcome data, it should be possible to spot some interesting trends.

Turns out you don't need to read an FOI response to start to be able to guess the outcome. I trained a TF-IDF classifier with a 73% macro F1-score in predicting success using just 3 keywords about the request and metadata. Adding the full request text hits 76% & a snippet of the response email 84%.

I wrote up some quick notes on yesterday's journey to nowhere:
foimonkey.github.io/posts/12-hou...

Made it back to Cowes in 11 hours 39 minutes. Taking the floating bridge x 2, a double decker bus, a Hovercraft, a single decker bus, a tram, the overground, DLR X 2 (got on the wrong train), the cable car, a catamaran, the underground, an automatic people mover, 3 x trains, and the vehicle ferry.

Going to see how many different forms of public transport it is possible to take in one day today. First up Floating bridge.

I took 40,000 images from emails in the WhatDoTheyKnow archive and extracted the five most dominant colours from each (excluding monochrome/shades of grey). Behold the palette of the UK public sector.

Then I'm going to have another go at teaching it Welsh. Did the 1B version on the weekend, which showed promising results in terms of picking up the vocab/grammar vs the base model. A larger, more capable model and a more curated dataset seem like the way to go there if it is to be useful.

I'm starting a couple of new projects today. First up, trying to make OLMo-2 better at reasoning. I want to have a go at teaching it to generate <think> tokens showing its chain-of-thought as it answers. It will be a couple of days before I know if it has worked.