hrbrmstr's Daily Drop
LightNews LightNews
dailydrop.hrbrmstr.dev.web.brid.gy
hrbrmstr's Daily Drop
@dailydrop.hrbrmstr.dev.web.brid.gy
51 followers 0 following 360 posts
A daily dose of productivity-enhancing finds/info from the deep, dark tech corners of the internet. Expect delightful surprises whether you're into web tech, data […]

[bridged from https://dailydrop.hrbrmstr.dev/ on the web by https://fed.brid.gy/ ]
Posts Replies Media Videos
dailydrop.hrbrmstr.dev.web.brid.gy
Today's Typography-centric edition of the Drop includes details about "Typeset in the Future," a website and book delving into the typography of science fiction films. It also introduces "Wordmark," a tool for visualizing and comparing fonts, and […]

[Original post on dailydrop.hrbrmstr.dev]
<p>Typeset In The Future; Wordmark; The History Of Ligatures</p> <p>Folks should have noticed a double delivery of some Drops I had forgotten to push &#8220;publish&#8221; on. That, plus not being in a great headspace due to&nbsp;<em>gestures wildly</em>&nbsp;meant no Drop yesterday. And, I kinda wanted the 500th normal Drop to be on a Typography Tuesday <img alt="😎" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f60e.png" style="height: 1em;" />.</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139503535" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="tldr">TL;DR</h3> <p><em>(This is an AI-generated summary of today&#8217;s Drop using Sonnet via Perplexity.)</em></p> <p>Here&#8217;s a concise three-bullet summary of the blog post:</p> <ul class="wp-block-list"> <li><a href="https://typesetinthefuture.com/">Typeset in the Future</a>: Dave Addey&#8217;s website and book analyze typography in science fiction films, exploring how fonts contribute to world-building. The book covers seven classic sci-fi movies and includes interviews with industry experts.</li> <li><a href="https://wordmark.it/">Wordmark</a>: A web-based tool that allows users to visualize and compare text in all installed fonts on their computer. It offers features like size adjustments, typeface filtering, and Google Fonts integration.</li> <li><a href="https://blazetype.eu/blog/history-of-ligatures-part-1-2">History of Ligatures</a>: Tracing back to 3500 BCE, ligatures have evolved from practical space-saving techniques in ancient scripts to aesthetic elements in modern typography. Their use has been influenced by various writing systems, printing technologies, and design trends throughout history.</li> </ul> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <h3 class="wp-block-heading" id="typeset-in-the-future">Typeset In The Future</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503536" height="232" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-20.png?w=560" tabindex="0" width="560" /></figure> <p><em>(Keen-eyed readers will recognize that I hinted this would be in today&#8217;s TT Drop back on the 11th.)</em></p> <p>Dave Addey&#8217;s &#8220;<a href="https://typesetinthefuture.com/">Typeset in the Future</a>&#8221; site, which spawned a book of the same name, offers a unique lens through which to view science fiction cinema. Addey zeroes in on the often-overlooked role of typography in crafting the visual language of futuristic worlds.</p> <p>The site dissects the typographic choices in iconic/classic sci-fi films, revealing how fonts like&nbsp;<a href="https://fontsgeek.com/fonts/eurostile-extended-bold">Eurostile Bold Extended</a>&nbsp;in &#8220;Alien&#8221; contribute to the film&#8217;s sterile, corporate-dominated future. Similarly, Addey&#8217;s analysis of &#8220;Total Recall&#8221; shows how typeface selection enhances the film&#8217;s gritty yet fantastical atmosphere.</p> <p>Addey&#8217;s approach goes beyond just plain ol&#8217; font identification. He contextualizes typography within the broader visual design of each film, exploring its interplay with set design, costumes, and art direction. This holistic view offers readers a deeper appreciation for the craft of sci-fi world-building.</p> <p>The site/book serves as a valuable resource for designers, filmmakers, and we mere typography enthusiasts. By meticulously documenting fonts used in various productions, the author creates a reference guide for those of us seeking inspiration from these designs. His detailed breakdowns of letterforms and comparisons between similar typefaces provide practical insights for professionals across various creative fields.</p> <p>The website and book also feature interviews with industry experts, including directors like Paul Verhoeven and designers such as Mike Okuda, offering behind-the-scenes perspectives on the role of typography in shaping cinematic futures.</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <h3 class="wp-block-heading" id="wordmark">Wordmark</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503538" height="446" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-21.png?w=1024" tabindex="0" width="1024" /></figure> <p>If you&#8217;re like me you have&nbsp;<em>alot</em>&nbsp;of fonts on your system(s). I have various ways I test them out for various text contexts, but I recently came across&nbsp;<a href="https://wordmark.it/">Wordmark</a>, and it&#8217;s a pretty well-executed site/tool.</p> <p>The core functionality is pretty straightforward. You type in a word, phrase, or paragraph that you want to visualize, the app displays your text using&nbsp;<em>all</em>&nbsp;the fonts installed on your computer, arranged in a visual grid, you scroll through the options, comparing how your text looks in different typefaces, and then tap on promising fonts to select them, then use the filter feature to isolate and compare your favorites.</p> <p>This visual approach eliminates the need to squint at dropdown menus or individually apply fonts to see how they look, significantly speeding up the font selection process.</p> <p>It has many controls for size adjustments, typeface filtering, and even has Google Fonts integration (so you&#8217;re not limited to what you have on your system). Speaking of which, it uses different methods to access and display your installed fonts, depending on your browser. For Chromium-based browsers, it utilizes the Local Font Access API, requiring user permission to access installed fonts. That&#8217;s, unfortunately, the best way to experience the app/site (so if you&#8217;re on Safari/Firefox, you may want to look elsewhere).</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <h3 class="wp-block-heading" id="the-history-of-ligatures">The History Of Ligatures</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503541" height="576" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-22.png?w=1024" tabindex="0" width="1024" /></figure> <p>Ligatures have a rich history. From ancient cuneiform tablets to modern digital fonts, these typographic elements have served both practical and aesthetic purposes, shaping the way we write and read. A&nbsp;<a href="https://blazetype.eu/blog/history-of-ligatures-part-1-2">recent-ish article series by Sebastien Hayez</a>&nbsp;traces the origins and provides some great information, and includes ton of context and examples along the way.</p> <p>The origins of ligatures can be traced back to Sumerian cuneiform tablets around 3500 BCE. Scribes discovered that joining characters saved both space and effort, a practice that would be echoed in Egyptian hieratic script centuries later. This early form of ligature was born out of necessity, as scribes sought to economize their movements and maximize the use of limited writing surfaces.</p> <p>As writing systems evolved, so did the use of ligatures. In Greek handwriting, the fluid motion of the&nbsp;<a href="https://en.wikipedia.org/wiki/Reed_pen">calamus</a>&nbsp;on papyrus naturally led to connections between letters, creating a unique ductus — the direction, number and sequence of strokes necessary to create letterforms — that became characteristic of the script. This gestural economy not only increased writing speed but also gave rise to new forms of abbreviation. Greek culture saw the development of&nbsp;<a href="https://en.wikipedia.org/wiki/Monogram">monogrammatic</a>&nbsp;ligatures, where groups of letters were combined to create compact symbols representing words or concepts.</p> <p>The Roman Empire brought its own innovations to ligature use.&nbsp;<a href="https://en.wikipedia.org/wiki/Lapidary">Lapidary</a>&nbsp;inscriptions employed various techniques to create ligatures, including overlapping, merging, and interlocking letters. These methods not only saved space on stone surfaces but also added a decorative element to the text. The discretionary nature of these ligatures allowed engravers to exercise their artistic judgment, resulting in visually striking inscriptions that have endured for centuries.</p> <p>The Middle Ages saw the continued evolution of ligatures, with Gothic calligraphy introducing new forms and combinations. The dense, ornate style of Gothic script lent itself well to ligatures, particularly in cursive and formal variations. Common ligatures in this period often combined letters with identical stems, such as &#8216;b&#8217; with &#8216;o&#8217; or &#8216;p&#8217; with &#8216;c&#8217;. This practice not only enhanced the visual cohesion of the text but also improved the efficiency of copyists.</p> <p>The advent of movable type printing in the 15th century presented new challenges and opportunities for ligatures. Johannes Gutenberg&#8217;s revolutionary 42-line Bible, published in 1455, included an impressive array of ligatures and special characters. In his quest to faithfully reproduce handwritten text, Gutenberg created 79 ligatures among his 290 character dies. I did not know this before reading the posts and I continue to be amazed at the craftsmanship of that era. This attention to detail helped bridge the gap between handwritten and printed text, setting a standard for typographic excellence that would influence printing for centuries to come.</p> <p>Today, ligatures continue to play a role in typography, albeit in a more subtle form. Digital fonts often include ligatures to improve readability and aesthetic appeal, particularly in languages that use diphthongs or in decorative typefaces. The history of ligatures serves as a testament to the enduring relationship between function and form in written communication, reminding us that even the smallest details can have a profound impact on how we convey and perceive information.</p> <p>That barely touched on some of the even cooler details in Sebastien&#8217;s posts. Def carve out some time to pore over them!</p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 16, 2024 at 6:08 PM
dailydrop.hrbrmstr.dev.web.brid.gy
WinterJS is a new serverside Javascript option developed by Winter Community Group (WinterCG). It's written in Rust, uses Mozilla's SpiderMonkey engine, and can handle over 150,000 requests-per-second. It supports various JavaScript runtimes and can […]

[Original post on dailydrop.hrbrmstr.dev]
<p><em>WinterJS; Echo-Chamber; So. Many. JS. Cheatsheets</em></p> <p>Now that I&#8217;ve cranked out a couple JS/WASM chapters in the &#8220;Cooking With DuckDB&#8221; book, I&#8217;ve got JavaScript on the noggin&#8217; so, today, we focus on three different (and fun!) JS-resources.</p> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139502882" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="winterjs">WinterJS</h3> <p></p> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139502887" height="1258" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/03/pexels-photo-20681552.jpeg" tabindex="0" width="1880" /><figcaption class="wp-element-caption">Photo by Elena Golovchenko on <a href="https://www.pexels.com/photo/stream-in-a-coniferous-forest-covered-with-snow-20681552/" rel="nofollow">Pexels.com</a></figcaption></figure> <p>As if there weren&#8217;t enough choices in &#8220;server-side javascript&#8221;-land to already cause analysis paralysis, we have a new WASM-y entry into the field with the official release of&nbsp;<a href="https://github.com/wasmerio/winterjs">WinterJS</a>&nbsp;(<a href="https://wasmer.io/posts/winterjs-v1">announcement</a>).</p> <p>Before we talk aboot that, we need to kick back and learn about the Winter Community Group (<a href="https://wintercg.org/">WinterCG</a>) — an organization within the World Wide Web Consortium (W3C) that focuses on developing standards and technologies related to Web Integration Technologies. Some key areas the group works on include:</p> <ul class="wp-block-list"> <li>WebComponents: Encapsulated, reusable custom elements for web applications</li> <li>Service Workers: Scripts that run in the background, separate from web pages, enabling features like offline support, push notifications, background sync, etc.</li> <li>Web App Manifest: JSON files that provide metadata about web applications to enable app-like behaviors</li> <li>Import Maps: Mechanisms to control how JavaScript modules are resolved and loaded in a web page</li> </ul> <p>The goal is to enable more powerful, feature-rich, and superfast web applications through these interoperable technologies and standards. The group collaborates with browser vendors, web developers, and other stakeholders in the web community to define and evolve these web platform capabilities.</p> <p>There are plenty of WinterCG-compatible Javascript runtimes including Bun, Deno, and others (even Amazon&#8217;s llrt —&nbsp;mentioned in a recent Drop —&nbsp;has a long-term plan for WinterCG-compatibility). What&#8217;s so special about WinterJS?</p> <p>Well, for starters, it is written in Rust, uses Mozilla&#8217;s&nbsp;<a href="https://spidermonkey.dev/">SpiderMonkey</a>&nbsp;engine, and can handle over 150,000 requests-per-second (pretty sure that can handle the traffic for any app that I might write). One especially cool feature of it, though, is that it can be compiled down to WebAssembly —&nbsp;specically,&nbsp;<a href="https://wasix.org/">WASIX</a>. This means it can squeeze into places other JS app servers cannot.</p> <p>While I am loathe to encourage normalizing the use of&nbsp;<code>curl-bash</code>&nbsp;idioms, they do make it super easy to show you how to test this out:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate"> $ # install wasmer $ curl https://get.wasmer.io -sSfL | sh $ mkdir -p test $ cat&lt;&lt;EOF&gt;test/simple.js addEventListener(&#039;fetch&#039;, (req) =&gt; { req.respondWith(new Response(&quot;&#039;sup?&quot;)); }); EOF $ wasmer run wasmer/winterjs --net --mapdir=tests:tests tests/simple.js $ # SEPARATE TAB/WINDOW $ curl -s http://localhost:8080/ &#039;sup? </pre></div> <p>You can also&nbsp;<code>cargo install --git https://github.com/wasmerio/winterjs winterjs</code>&nbsp;to have a proper local binary</p> <p>It can run in a Cloudflare Workers environment or&nbsp;<a href="https://wasmer.io/products/edge">Wasmer Edge</a>, and also supports many frameworks:</p> <ul class="wp-block-list"> <li>Next.js</li> <li>Next.js React Server Components</li> <li>Hono (one I&#8217;m presently playing with off-and-on)</li> <li>Astro.build</li> <li>Remix.run</li> <li>Svelte</li> <li>Gatsby</li> <li>Nuxt</li> </ul> <p>It turns out that the migration to WordPress borked&nbsp;<a href="https://dailydrop.hrbrmstr.dev/2024/03/28/bonus-drop-36-2023-12-16-knowl-edge-drop/">Bonus Drop #36</a>, but I just restored it (yay for writing locally in Markdown first!). That covers a self-hosted similar thing by Supabase. It&#8217;s not paywalled, so give that a read to see where WinterJS fits into things, and check back for when I get more of a chance to kick the tyres.</p> <h3 class="wp-block-heading" id="echo-chamber">Echo-Chamber</h3> <p></p> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139502885" height="1300" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/03/pexels-photo-15172344.jpeg" tabindex="0" width="867" /><figcaption class="wp-element-caption">Photo by Magda Ehlers on <a href="https://www.pexels.com/photo/vaulted-ceiling-and-arched-window-of-corpus-christi-chapel-in-kutna-hora-czechia-15172344/" rel="nofollow">Pexels.com</a></figcaption></figure> <p>This resource is just bonkers fun/cool and anyone can get started with it right after reading this section. I&#8217;m going to quote the author directly as it says it pretty plainly:</p> <p><a href="https://github.com/tessalt/echo-chamber-js">Echochamber.js</a>&nbsp;is a 3rd-party script you can install to add a simple comment form to your blog post or website.</p> <p><em>Why not just use disqus?</em></p> <p>Because then there&#8217;d be a chance that someone would read the comments. You might have to read those comments. You don&#8217;t want that.</p> <p>When a user submits a comment,&nbsp;<code>echochamber.js</code>&nbsp;will save the comment to the user&#8217;s&nbsp;<code>LocalStorage</code>, so when they return to the page, they can be confident that their voice is being heard, and feel engaged with your very engaging content. It does not make any HTTP requests. Since LocalStorage is only local, you and your database need not be burdened with other people&#8217;s opinions.</p> <p>Super fun way to pwn verbose visitors!</p> <h3 class="wp-block-heading" id="so-many-js-cheatsheets">So. Many. JS. Cheatsheets</h3> <p></p> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139502884" height="1253" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/03/pexels-photo-357514.jpeg" tabindex="0" width="1880" /><figcaption class="wp-element-caption">Photo by Pixabay on <a href="https://www.pexels.com/photo/pile-of-folders-357514/" rel="nofollow">Pexels.com</a></figcaption></figure> <p>Codeacademy has a great set of&nbsp;<a href="https://www.codecademy.com/resources/cheatsheets/language/javascript">JavaScript cheatsheets</a>&nbsp;that are done with a twist. This isn&#8217;t some simple three-column JS language map. The sheets have themes like &#8220;<a href="https://www.codecademy.com/learn/intro-sql-back-end-development/modules/webdev-sql-manipulation/cheatsheet">Create a Back-End App with JavaScript &#8211; SQL for Back-End Development</a>&#8221; which give you a head start on digging into an area you&#8217;re not familiar with or get you started — quickly — in a new area.</p> <p>They aren&#8217;t all-in-one solutions articles; and, you will most certainly have to put in some work and thought. But, they may help you get unstuck or get you pointed in the right direction faster than trying to burn through a tome, or hack-and-pray your way through a new project.</p> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139502882" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 15, 2024 at 12:38 PM
dailydrop.hrbrmstr.dev.web.brid.gy
It's a 🕸️🕷️-focused Drop, today, with two tools (well, one tool and one tool/library/ecosystem) for slurping content up (responsibly!), and one TUI for helping craft perfect APIs
<p><em>Scout; posting; Crawlee</em></p> <p>Two scraping and one API-ing resources to help you wrangle HTTP ops more efficiently.</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139503501" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="tldr">TL;DR</h3> <p><em>(This is an AI-generated summary of today&#8217;s Drop using Sonnet via Perplexity.)</em></p> <ul class="wp-block-list"> <li>Scout (<a href="https://scout-lang.netlify.app/">https://scout-lang.netlify.app/</a>): A domain-specific language for web scraping and crawling, written in Rust. It offers a simple syntax and coder-friendly features for both basic and complex web scraping tasks.</li> <li>Posting (<a href="https://github.com/darrenburns/posting">https://github.com/darrenburns/posting</a>): A modern, terminal-based HTTP client that brings Postman-like functionality to the command line. It allows organizing and storing HTTP request collections in a git-friendly format.</li> <li>Crawlee (<a href="https://crawlee.dev/">https://crawlee.dev/</a>): A web scraping and browser automation library for Node.js, offering a unified API for HTTP and browser-based crawling. It includes features like proxy rotation, session management, and support for various crawling methods.</li> </ul> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <h3 class="wp-block-heading" id="scout">Scout</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503502" height="832" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-17.png?w=1024" tabindex="0" width="1024" /></figure> <p><a href="https://scout-lang.netlify.app/">Scout</a>&nbsp;(<a href="https://github.com/maxmindlin/scout-lang">GH</a>) is a domain-specific language (DSL) designed specifically for web scraping and crawling tasks. Written in Rust, Scout combines a simple, easy-to-learn syntax with coder-friendly web crawling capabilities. It aims to lower the barrier of entry for web scraping while providing advanced features for more complex scenarios. Please note that you need to have icky Firefox installed along with&nbsp;<a href="https://github.com/mozilla/geckodriver"><code>geckodriver</code></a>&nbsp;before trying to use Scout, as it performs all web-ops through that browser.</p> <p>The language spec&nbsp;<a href="https://scout-lang.netlify.app/docs/getting-started/fundamentals/">fits on a single web page</a>&nbsp;(which means nothing — since one page could scroll forever — but the main&nbsp;<code>div</code>&nbsp;of that page has ~200 lines and ~6K characters), and the operations are all fairly straightforward. For example, take this sample REPL session (just running&nbsp;<code>scout</code>&nbsp;in the terminal starts a REPL):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: jscript; title: ; notranslate"> &gt;&gt; goto &quot;https://example.com&quot; Null &gt;&gt; h1 = $&quot;h1&quot; Null &gt;&gt; h1 |&gt; textContent() &quot;Example Domain&quot; </pre></div> <p>That&#8217;s great, but we likely want to web scrape to get some data back that we can use. For that, Scout has a&nbsp;<code>results</code>&nbsp;JSON output structure which gets populated via&nbsp;<code>scrape</code>&nbsp;operations. Let&#8217;s grab all the links and titles from the main Hacker News site (NOTE: use their API if doing IRL &#8220;scraping&#8221; of their site):</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: jscript; title: ; notranslate"> goto &quot;https://news.ycombinator.com/&quot; items = $$&quot;tr.athing &gt; td.title &gt; span.titleline&quot; for item in items do scrape { text: item |&gt; textContent(), link: $(item)&quot;a&quot; |&gt; href(), } end </pre></div> <p>That will spit back something like:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: jscript; title: ; notranslate"> { "results": { "https://news.ycombinator.com/": &#91; { "link": "https://www.burn-heart.com/rulers-of-the-ancient-world", "text": "Rulers of the Ancient World: period correct measuring tools (burn-heart.com)" }, { "link": "https://typesetinthefuture.com/2018/12/04/walle/", "text": "The Typeset of Wall·E (2018) (typesetinthefuture.com)" }, { "link": "https://matttproud.com/blog/posts/x-window-system-boot-stipple.html", "text": "Iconography of the X Window System: The Boot Stipple (matttproud.com)" }, { "link": "https://payloadspace.com/habitable-worlds-observatory-and-the-future-of-space-telescopes-in-the-era-of-heavy-lift-launch/", "text": "Future of Space Telescopes in the Era of Super Heavy Lift Launch (payloadspace.com)" }, … ] } } </pre></div> <p>In debug mode, you can see the Firefox instrumentation (an example of that is in the section header). It also groks proxies and such, so if you&#8217;re behind one, you can still use it.</p> <p>I really like the language spec, so far, and learning the language has been very straightforward.</p> <p>Cannot wait to see how this evolves, and I&#8217;ll see if I can put together a larger, more real-life example over the coming weeks.</p> <h3 class="wp-block-heading" id="posting">posting</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503504" height="616" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-18.png?w=1024" tabindex="0" width="1024" /></figure> <p><a href="https://github.com/darrenburns/posting">Posting</a> is a modern, terminal-based HTTP client that brings <a href="https://www.postman.com/">Postman</a>-like functionality to the the terminal/CLI. It enables us to send and test HTTP requests directly from terminal, which means you can also use it over SSH (perhaps, from a different system that has better access to the resources in question).</p> <p>The tools let us organize and store collections of HTTP requests in a git-friendly format, using (ugh) YAML files with a&nbsp;<code>.posting.yaml</code>&nbsp;extension.</p> <p>As one might expect, it fully supports the use of environment variables and <code>.env</code> files, so we don&#8217;t have to hard-code different values for, say, dev/test/prod.</p> <p>Since it&#8217;s a modern TUI, it fully supports efficient navigation using both keyboard shortcuts and mouse input, including a &#8220;jump mode&#8221; for quick access to different parts of the interface. It&#8217;s also fully theme-able (including the layout), and lets us customize the response formatting as well. Said modernity is also shown via a built-in command palette, so you&#8217;re a quick keystroke-or-two away from every command.</p> <p>I&#8217;ve stolen one of their screenshots for the section header, but&nbsp;<em>please</em>&nbsp;head to the GH repo, as the there&#8217;s tons more info, and some animated screens that show off some of the features way better than any more blathering I could do.</p> <h3 class="wp-block-heading" id="crawlee">Crawlee</h3> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139503506" height="1254" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/spider-web-with-water-beads-network-dewdrop.jpg" tabindex="0" width="1880" /><figcaption class="wp-element-caption">Photo by Pixabay on <a href="https://www.pexels.com/photo/spider-web-34225/" rel="nofollow">Pexels.com</a></figcaption></figure> <p><em>(Given that it sure feels like they&#8217;re suggesting stealing content for use in AI training is OK, I almost did not include this, today, but I know Drop readers will use services/tools responsibly.)</em></p> <p><a href="https://crawlee.dev/">Crawlee</a>&nbsp;(<a href="https://github.com/apify/crawlee">GH</a>) is a &#8220;<em>web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.&#8221;</em></p> <p>The tool/library has several key features that make it a pretty compelling tool for web crawling. It offers a unified API that caters to both HTTP and browser-based crawling, which gives us plenty of flexibility, especially when a given website thinks it is too clever by half. The platform automatically scales based on system resources, and also integrates proxy rotation and session management, which helps keep you enhancing anonymous and reduces the likelihood of IP bans. <em>(Again, please consider using this tool responsibly!)</em></p> <p>It also has a persistent queue system, which helps manage URLs effectively. Configurable storage options are also available for storing scraped data. And, Crawlee comes with built-in error handling and retry mechanisms, ensuring robustness and reliability during the crawling process.</p> <p>Crawlee comes with various crawling methods to suit different needs. The &#8220;CheerioCrawler&#8221; provides fast HTTP-based crawling with Cheerio for HTML parsing, making it ideal for lightweight and speedy tasks. For more complex requirements, the &#8220;PuppeteerCrawler&#8221; utilizes headless Chrome automation via Puppeteer, offering extensive control over the browsing environment. The &#8220;PlaywrightCrawler&#8221; supports multi-browser automation with Playwright, compatible with Chrome, Firefox, and WebKit.</p> <p>By abstracting away common scraping challenges, this tool/library lets us focus on extracting and processing data rather than dealing with low-level crawling logistics. It seems super-flexibile, and the feature set looks fairly comprehensive.</p> <p>While I do my best to try things out before adding them to the Drop, this one is on the &#8220;TODO&#8221; list. If you kick the tyres before I do, def lemme know how you liked it!</p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 15, 2024 at 12:37 PM
dailydrop.hrbrmstr.dev.web.brid.gy
Today's drop is all about Observable Framework version 1.9.0! This June update introduces new features, including support for React & JSX and improved inline javascript expressions. The Observable HQ team replaced tags marking inline code blocks with […]

[Original post on dailydrop.hrbrmstr.dev]
<p>Observable Framework 1.9.0</p> <p>Back in June, the fine folks over at Observable HQ&nbsp;<a href="https://github.com/observablehq/framework/releases/tag/v1.9.0">bumped up the version to 1.9.0</a>, and gave us some goodies along the way.</p> <p>As longtime Drop readers know, I despise giant libraries like React and Vue, so you can investigate Framework&#8217;s new support for React &amp; JSX. It looks super easy, but I avoid React and JSX like we all should have the plague we&#8217;re still in.</p> <p>One of Framework&#8217;s superpowers are <a href="https://observablehq.com/framework/javascript#inline-expressions">inline javascript expressions</a>, which let us interpolate dynamic content anywhere on the page using <code>${…}</code>. An issue with them (before 1.9.0) was that they used <code>&lt;span&gt;</code> tags to mark the start/end of those inline blocks, which messes up all sorts of things HTML-wise.</p> <p>They switched to using comments (they also leveled up how they parse markdown files and JS code blocks/expressions), so now it&#8217;s possible to dynamically create cards, table rows, plots…anything, really.</p> <p>While they had some toy examples to explain this, I threw together&nbsp;<a href="https://codeberg.org/hrbrmstr/new-in-1-dot-9">a larger example</a>&nbsp;that showcases these new superpowers.</p> <p>Follow the instructions at that Codeberg link to see the magic behind:</p> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503520" height="637" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/tags.png?w=959" tabindex="0" width="959" /></figure> <p>and:</p> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503521" height="442" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/moar-tags.png?w=752" tabindex="0" width="752" /></figure> <p>and:</p> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503522" height="833" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/plots.png?w=734" tabindex="0" width="734" /></figure> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 14, 2024 at 8:52 PM
dailydrop.hrbrmstr.dev.web.brid.gy
Today's Drop highlights some Bash tricks, including process substitution, indented heredocs, and the use of readarray and jq for efficient data operations. It emphasizes the benefits and usage of these methods (with examples), and encourages readers […]

[Original post on dailydrop.hrbrmstr.dev]
<p><em>Some Quick Bash Tricks For Fun And Profit</em>¹</p> <p>I had to do some grungy work at the CLI this week. In doing so, I realized that I&#8217;d actually been keeping up jotting things down in <a href="https://bear.app/">Bear notes</a> when I went to it for some Bash references that I had managed to stash away in there, over the past few months (I&#8217;ve been increasingly relying on it as a brain extension due to the long covid stuff).</p> <p>So, rather than some third-party tool references, today&#8217;s Drop is just a collection of some Bash idioms I rarely see used in most scripts I come across, but that can be quite useful. Each section will start with an example and then work through it.</p> <p>Please note that some of these require Bash 4+. macOS folks will need to rely on Homebrew (et al.) for that.</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139503509" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <figure class="wp-block-image size-large is-resized"><img alt="" class="wp-image-139503513" height="580" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-19.png?w=580" style="width: 649px; height: auto;" tabindex="0" width="580" /></figure> <h3 class="wp-block-heading" id="process-substitution">Process Substitution</h3> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate"> while read line; do echo &quot;Processing: $line&quot; done &lt; &lt;(ls -1 *.txt) </pre></div> <p>I&#8217;m not sure why we don&#8217;t see the <code>&lt;(COMMAND)</code> / <code>&lt; &lt;(COMMAND)</code> idioms more. It&#8217;s pretty handy! It avoids creating temporary files; the <code>while</code> loop runs in the current shell, not a subshell, so variables set inside the loop persist after the loop ends; and, it allows for direct processing of command output in a line-by-line manner.</p> <p>Process substitution allows a command&#8217;s input or output to appear as a file, which enables direct reading from or writing to another program. It exits to make it possible to provide data to programs that only accept files as inputs/outputs.</p> <p>Let&#8217;s break down the example:</p> <ol class="wp-block-list"> <li><code>ls -1 *.txt</code>: This command lists all <code>.txt</code> files in the current directory, one file per line.</li> <li><code>&lt;(ls -1 *.txt)</code>: This process substitution runs the <code>ls</code> command and makes its output available as if it were a file.</li> <li><code>&lt; &lt;(ls -1 *.txt)</code>: The outer <code>&lt;</code> redirects the output of the process substitution as input to the <code>while</code> loop.</li> <li><code>while read line; do ... done</code>: This loop reads each line from the input (in this case, each filename output by <code>ls</code>).</li> <li><code>echo "Processing: $line"</code>: For each filename read, this line prints a message indicating its being processed. You can, of course, do anything you want with that data.</li> </ol> <p>Note the space between the&nbsp;<code>&lt; &lt;</code>&nbsp;is important, and the input version has a sibling output version:&nbsp;<code>&gt; &gt;(COMMAND)</code>. One more common use for the output version is something like:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate"> SOMEOTHERPROCESSTHATMAKESALOTOFBYTES | tee &gt;(gzip &gt; file.gz) &gt;(bzip2 &gt; file.bz2) &gt; file </pre></div> <p>This uses output process substitution with&nbsp;<code>tee</code>&nbsp;command to simultaneously output data to multiple destinations, including compressed files. Let&#8217;s break this example down, too:</p> <ol class="wp-block-list"> <li><code>tee</code>: This command reads from standard input and writes to both standard output and files.</li> <li><code>&gt;(gzip &gt; file.gz)</code>: This is process substitution. It runs the <code>gzip</code> command and redirects its output to <code>file.gz</code>. The <code>&gt;()</code> syntax makes this appear as a file to <code>tee</code>.</li> <li><code>&gt;(bzip2 &gt; file.bz2)</code>: Similar to the previous one, but uses <code>bzip2</code> compression and outputs to <code>file.bz2</code>.</li> <li><code>&gt; file</code>: This is a regular output redirection to <code>file</code>.</li> </ol> <p>What&#8217;s happening with the data is:</p> <ol class="wp-block-list"> <li>Some data is piped into this command (not shown in the example).</li> <li><code>tee</code> takes this input and: <ul class="wp-block-list"> <li>Writes it uncompressed to <code>file</code></li> <li>Sends it to <code>gzip</code>, which compresses it and writes to <code>file.gz</code></li> <li>Sends it to <code>bzip2</code>, which compresses it and writes to <code>file.bz2</code></li> <li>Also outputs it to <code>stdout</code></li> </ul> </li> </ol> <p>This allows you to create three versions of the same data simultaneously:</p> <ul class="wp-block-list"> <li>An uncompressed version in <code>file</code></li> <li>A gzip-compressed version in <code>file.gz</code></li> <li>A bzip2-compressed version in <code>file.bz2</code></li> </ul> <p>This is super useful when you want to process or store data in multiple formats without having to read it multiple times or create intermediate files. It&#8217;s an efficient way to create backups, compressed versions of data streams, or, say, turn some JSON into multiple formats all at once.</p> <h3 class="wp-block-heading" id="indented-heredocs">Indented Heredocs</h3> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate"> if true; then cat &lt;&lt;-EOF I am a heredoc is inside an if statement! I&#039;m indented to match the surrounding code. This makes the script more readable. EOF fi </pre></div> <p>An indented heredoc uses the&nbsp;<code>&lt;&lt;-</code>&nbsp;syntax instead of the standard&nbsp;<code>&lt;&lt;</code>&nbsp;for heredocs. The hyphen (<code>-</code>) tells Bash to ignore leading tab characters in the heredoc content and the closing delimiter. I really wish I used them more in the past, and also wish more folks used them now.</p> <p>Please note that:</p> <ul class="wp-block-list"> <li><strong>only tab characters are ignored</strong> (you can&#8217;t copy/paste that example due to &#8220;WordPress&#8221;); if you use spaces for indentation, they will be preserved in the output.</li> <li>the closing delimiter (<code>EOF</code> in these examples) <em>must be at the start of the line</em> with no leading whitespace.</li> <li>using <code>&lt;&lt;-</code> instead of <code>&lt;&lt;</code> is what enables this tab-stripping behavior.</li> </ul> <h3 class="wp-block-heading" id="readarray--jq--">readarray + jq == <img alt="💙" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f499.png" style="height: 1em;" /></h3> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate"> readarray -t tags &lt; &lt;(jq -c &#039;.tags&#91;]&#039; tags.json) for tag in &quot;${tags&#91;@]}&quot;; do echo &quot;$tag&quot; done </pre></div> <p>If you&#8217;ve got the memory (and Bash 4+),&nbsp;<code>readarray</code>&nbsp;can help save you extra operations by reading in data into an associative array (in the example, the keys are numeric), which lets you use that data again without re-reading it.</p> <p>I had to do a bunch of different ops with our tags (I talk about our work data alot, apologies), and this made the operations way more efficient. Let&#8217;s break this down, too:</p> <ol class="wp-block-list"> <li><code>readarray -t tags &lt; &lt;(jq -c '.tags[]' tags.json)</code> <ul class="wp-block-list"> <li><code>jq -c '.tags[]' tags.json</code>: This command uses <code>jq</code> to process the <code>tags.json</code> file. It extracts all elements from the <code>tags</code> array in the JSON file and outputs them in compact form (the <code>-c</code> flag).</li> <li><code>&lt;(...)</code>: This is process substitution! It makes the output of the <code>jq</code> command appear as a file.</li> <li><code>readarray -t tags &lt; ...</code>: This reads the output from the process substitution into an array named <code>tags</code>. The <code>-t</code> option trims trailing newlines from each line read.</li> </ul> </li> <li><code>for tag in "${tags[@]}"; do</code><br />This starts a loop that iterates over each element in the <code>tags</code> array.</li> <li><code>echo "$tag"</code> (do something more useful than this pls)</li> </ol> <p>The&nbsp;<code>@</code>&nbsp;symbol in&nbsp;<code>"${tags[@]}"</code>&nbsp;is used to reference all elements of the&nbsp;<code>tags</code>&nbsp;array. It expands to all elements of the array, with each element treated as a separate word. When used inside double quotes like&nbsp;<code>"${tags[@]}"</code>, it preserves the integrity of each array element, even if the elements contain spaces or special characters (so, no word splitting). This syntax ensures that each element of the array is processed separately in the loop, regardless of any whitespace or special characters within the elements. You could use&nbsp;<code>*</code>&nbsp;instead, but that has no similar preservation guarantees.</p> <p><code>mapdef</code>&nbsp;/&nbsp;<code>readarray</code>&nbsp;has&nbsp;<a href="https://helpmanual.io/builtin/readarray/">some&nbsp;<em>cool</em>&nbsp;options</a>&nbsp;that you may want to check out.</p> <h3 class="wp-block-heading" id="useview-the-source-dude"><s>Use</s>View The Source, Dude!</h3> <p><code>readarray</code> is one of Bash&#8217;s &#8220;builtins&#8221;. They&#8217;re just magically there, waiting for you to use them. They&#8217;re written in C, and to get a better idea of how they really operate, you can <a href="https://git.savannah.gnu.org/cgit/bash.git/tree/builtins">view their source code</a>. As intuited in the previous section <code>readarray</code> is really <code>mapdef</code>, and you can see what it does <a href="https://git.savannah.gnu.org/cgit/bash.git/tree/builtins/mapfile.def">right here</a>.</p> <p>Checking out the source may help you workaround any issues you run into, or learn a ton more about the functionality of each builtin.</p> <p>And, if you&#8217;re not a C expert,&nbsp;<a href="https://www.perplexity.ai/search/explain-what-this-source-for-b-AzYJlSE4TjG8zoKZzGtiLw.">you can always phone an AI friend</a></p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>¹ <em>Warning: Use of these Bash tricks may result in sudden bursts of productivity, uncontrollable urges to automate everything, and an inexplicable desire to speak in command-line syntax. Side effects may include elevated geek status, spontaneous terminal opening, and the ability to bend computers to your will. Profits not guaranteed, but fun is virtually assured. The author accepts no responsibility for late-night coding sessions, neglected social lives, or the inevitable &#8220;Just one more script&#8221; syndrome. Use at your own risk, and may the force of the command line be with you.</em></p> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 12, 2024 at 5:35 PM
dailydrop.hrbrmstr.dev.web.brid.gy
Toda's Drop features three image processing tools/microservices to consider: trimage for lossless compression of PNG and JPG files, rimage for modern image format support and fine-grained quality control, and imaginary for fast Go-based image […]

[Original post on dailydrop.hrbrmstr.dev]
<p><em>trimage; rimage; imaginary</em></p> <p>In light of the recent Ghostscript vulnerability, which can be exposed if you use ImageMagick in image processing pipelines, I thought it might be useful to cover some different image processing tools/self-hostable microservices. So, that&#8217;s the focus of today&#8217;s Drop.</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139503492" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="tldr">TL;DR</h3> <p><em>(This is an AI-generated summary of today&#8217;s Drop using Sonnet via Perplexity.)</em></p> <ul class="wp-block-list"> <li><code>trimage</code> (<a href="https://trimage.org/">https://trimage.org/</a>): A cross-platform GUI and CLI tool for lossless image compression of PNG and JPG files, utilizing optipng, pngcrush, advpng, and jpegoptim for optimal compression while maintaining quality.</li> <li><code>rimage</code> (<a href="https://github.com/SalOne22/rimage">https://github.com/SalOne22/rimage</a>): A Rust-based CLI tool supporting modern image formats (JPEG, JPEG XL, PNG, AVIF, WebP), offering parallel processing, fine-grained quality control, and advanced features like quantization and dithering.</li> <li><code>imaginary</code> (<a href="https://github.com/h2non/imaginary">https://github.com/h2non/imaginary</a>): A fast, Go-based HTTP microservice for image processing, offering resizing, cropping, and transformation capabilities through a REST API, with support for remote image fetching and robust access controls.</li> </ul> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <h3 class="wp-block-heading" id="trimage">trimage</h3> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139503494" height="1253" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/pexels-photo-1319460.jpeg" tabindex="0" width="1880" /><figcaption class="wp-element-caption">Photo by Nikolaos Dimou on <a href="https://www.pexels.com/photo/several-scissors-1319460/" rel="nofollow">Pexels.com</a></figcaption></figure> <p><a href="https://trimage.org/">Trimage</a>&nbsp;(<a href="https://github.com/Kilian/Trimage">GH</a>)&nbsp;<em>(&#8220;trim the image&#8221;)</em>&nbsp;is a neat, versatile image optimization tool that efficiently compresses image files without sacrificing quality. It&#8217;s a cross-platform application that combines the power of several command-line utilities into a very friendly graphical interface and CLI, streamlining the process of reducing file sizes for both PNG and JPG formats.</p> <p>The tool leans heavily on the capabilities of&nbsp;<code>optipng</code>,&nbsp;<code>pngcrush</code>,&nbsp;<code>advpng</code>, and&nbsp;<code>jpegoptim</code>&nbsp;to ensure we get optimal compression, depending on what file type we send to the tool. A notable feature and focus of trimage is that it aims to achieve lossless compression at the highest available levels. It does a pretty solid job at doing so, and generally produces the smallest, lossless files.</p> <p>The GUI version sports a traditional file dialog, but also supports drag-and-drop (I&#8217;ll never get used to that in any tool). The CLI version works great in pipelines, or for one-off conversion tasks in the terminal.</p> <p>In addition to compressing image files, the application can automagically strip away&nbsp;<a href="https://en.wikipedia.org/wiki/Exif">EXIF</a>&nbsp;and other metadata. The further reduces file sizes without impacting visual quality.</p> <p>It&#8217;s been around for a ~decade, has a cool community, and is a good choice for predictable output.</p> <h3 class="wp-block-heading" id="rimage">rimage</h3> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139503497" height="1300" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/pexels-photo-5477716.jpeg" tabindex="0" width="867" /><figcaption class="wp-element-caption">Photo by Karolina Kaboompics on <a href="https://www.pexels.com/photo/red-rulers-and-set-of-calipers-in-pink-box-5477716/" rel="nofollow">Pexels.com</a></figcaption></figure> <p>If an &#8220;all-in-one-binary&#8221; solution is more to your liking, you may be up for kicking the tyres on&nbsp;<a href="https://docs.rs/rimage/latest/rimage/">rimage</a>&nbsp;(<a href="https://github.com/SalOne22/rimage">GH</a>), a Rust-based CLI tool that&#8217;s been around for just over a year.</p> <p>It supports a wide array of modern image formats, including JPEG, JPEG XL, PNG, AVIF, and WebP. It also leverages parallel processing and can optimizate multiple images at the same time.</p> <p>With it, we have fine-grained control over image quality (including lossless options, like trimage). Folks with more advanced needs will like the tool&#8217;s support for quantization and dithering, two techniques that can dramatically reduce file sizes while maintaining acceptable image quality. rimage also has options for resizing images.</p> <p>The README and Rust docs site are well-crafted, so I&#8217;ll leave you to explore them vs. suffer through more blathering.</p> <h3 class="wp-block-heading" id="imaginary">imaginary</h3> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139503498" height="1253" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/pexels-photo-13631585.jpeg" tabindex="0" width="1880" /><figcaption class="wp-element-caption">Photo by Reagan Ross on <a href="https://www.pexels.com/photo/gopher-on-gray-soil-13631585/" rel="nofollow">Pexels.com</a></figcaption></figure> <p>If a web-service/app is more aligned with your image processing needs, you should def take a look at&nbsp;<a href="https://github.com/h2non/imaginary">imaginary</a>&nbsp;(it also has CLI functionality, dpending on the&nbsp;<code>docker run</code>&nbsp;incantation). It&#8217;s written in Go (using&nbsp;<a href="https://github.com/h2non/bimg"><code>bimg</code></a>/<a href="https://github.com/jcupitt/libvips"><code>libvips</code></a>&nbsp;image processing libraries), and offers blazing-fast image manipulation through a straightforward API. It can resize, crop, rotate, and transform images at speeds up to 10x faster than traditional tools like ImageMagick.</p> <p>The REST API also lets us chain together complex sequences of image operations in a single HTTP request; and this self-described microservice also has built-in support for fetching images from remote URLs, making it easy to integrate into existing workflows.</p> <p>Security-conscious humans will appreciate imaginary&#8217;s robust access controls. The service can be configured to restrict image processing to specific origins, preventing abuse from malicious actors (but, please consider using Tailscale, and avoid putting any service on the internet if at all possible). An optional URL signature system adds an extra layer of protection for sensitive assets.</p> <p>In benchmark tests, a single imaginary instance processed over 20 concurrent image resize operations per second on modest hardware. The Go runtime&#8217;s efficient concurrency model, combined with&nbsp;<code>libvips</code>&#8216; low memory footprint means imaginary can scale super-well.</p> <p>This project, too, has an excellent README and associated documentation, so you should also def check those out.</p> <p>FWIW, it worked very well under Orbstack (the excellent Docker replacement for macOS), with baseline memory usage at around ~50 MiB.</p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 10, 2024 at 1:11 PM
dailydrop.hrbrmstr.dev.web.brid.gy
Today's typography-centric edition of the Drop has a cryptically titled section "Inter Typus Enim GNOME Defectu" that you'll have to read to find out more, plus a deep dive on a super cool font family that's great for prose and coding/terminals, and […]

[Original post on dailydrop.hrbrmstr.dev]
<p><em>Inter Typus Enim GNOME Defectu; Recursive; nanofont3x4</em></p> <p>Two (well, technically, four) fonts featured in today&#8217;s typography-centric edition of the Drop!</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139503485" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="tldr">TL;DR</h3> <p><em>(This is an AI-generated summary of today&#8217;s Drop using Sonnet via Perplexity.)</em></p> <ul class="wp-block-list"> <li>GNOME project is switching its default font from Cantarell to Inter Variable 11, citing Inter&#8217;s modernity, active maintenance, and improved readability. The change is expected to be implemented in GNOME 47 around mid-September 2024. (<a href="https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/-/issues/52">https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/-/issues/52</a>)</li> <li>Recursive, a variable font family designed by Stephen Nixon, offers both sans-serif and monospace versions with five axes for flexibility. It features true cursive italics and code ligatures, making it suitable for both UI design and coding. (<a href="https://www.recursive.design/">https://www.recursive.design/</a>)</li> <li>Nanofont3x4, created by Michael Pohoreski, is the &#8220;world&#8217;s smallest readable 3&#215;4 font with lowercase&#8221; that includes all ASCII symbols. It&#8217;s designed for use in tiny displays and for rendering miniature but legible text in various applications. (<a href="https://github.com/Michaelangel007/nanofont3x4">https://github.com/Michaelangel007/nanofont3x4</a>)</li> </ul> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <h3 class="wp-block-heading" id="inter-typus-enim-gnome-defectu">Inter Typus Enim GNOME Defectu</h3> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139503489" height="1253" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/pexels-photo-1503441.jpeg" tabindex="0" width="1880" /><figcaption class="wp-element-caption">Photo by Susanne Jutzeler, suju-foto on <a href="https://www.pexels.com/photo/gnome-on-swing-chair-1503441/" rel="nofollow">Pexels.com</a></figcaption></figure> <p><em>(My deepest apologies to&nbsp;<a href="https://en.wikipedia.org/wiki/Inter_Arma_Enim_Silent_Leges_%28Star_Trek%3A_Deep_Space_Nine%29">Ronald Moore</a>&nbsp;for the hacky section title; and, to you, Drop consumer, for the horribad Latin.)</em></p> <p>Back in 2010, the&nbsp;<a href="https://gnome.org/">GNOME project</a>&nbsp;chose&nbsp;<a href="https://cantarell.gnome.org/">Cantarell</a>&nbsp;as a new default font. It was&nbsp;<a href="https://en.wikipedia.org/wiki/Cantarell_%28typeface%29">originally designed by Dave Crossland</a>. I&#8217;m not a big fan of the font, and it&nbsp;<a href="https://gitlab.gnome.org/GNOME/cantarell-fonts/-/issues">has a list of issues</a>&nbsp;(many summed up&nbsp;<a href="https://news.itsfoss.com/gnome-default-font/">here</a>) that remain unfixed.</p> <p>~8 months ago, the project&nbsp;<a href="https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/-/issues/52">decided to experiment with switching to Inter</a>. We&#8217;ve covered Inter before. It&#8217;s a&nbsp;<em>great</em>&nbsp;font.</p> <p>I do suggest reading the issue thread for the change, but I can channel my inner Inigo Montoya and sum up the core reasons for the switch:</p> <ul class="wp-block-list"> <li>Inter is viewed as a more modern typeface compared to Cantarell.</li> <li>Inter is actively maintained and developed to support new font features.</li> <li>Some members of the GNOME design team favor Inter for its readability and consistency.</li> <li>Inter features over 2000 glyphs covering 147 languages, which could improve GNOME&#8217;s internationalization.</li> </ul> <p>To refresh your memory, Inter&#8217;s tall&nbsp;<a href="https://en.wikipedia.org/wiki/X-height">x-height</a>&nbsp;does improve readability, especially in mixed-case and lower-case text. It also has a full&nbsp;<a href="https://ironion.com/blog/2015/11/03/css-font-weights-explained/">weight</a>&nbsp;range from 100 to 900, so it&#8217;s also more versatile. And, it has dope support for&nbsp;<a href="https://learn.microsoft.com/en-us/typography/opentype/spec/featurelist">OpenType features</a>, so GNOME apps can take advantage of advanced typography options.</p> <p>The&nbsp;<a href="https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/-/merge_requests/85">change has been merged</a>&nbsp;in the GNOME repository, setting Inter Variable 11 as the default font, and GNOME users can possibly see it as the default as soon as GNOME 47 (~mid-September 2024).</p> <p>It&#8217;ll be interesting to see the reaction from the humans who use GNOME as their desktops (Fedora, Ubuntu, and Debian all use GNOME by default on the desktop).</p> <h3 class="wp-block-heading" id="recursive">Recursive</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503487" height="551" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-16.png?w=1024" tabindex="0" width="1024" /></figure> <p>If you did read through the &#8220;should we switch to Inter?&#8221; issue thread (as advised in the previous section), you may have noticed a question about possibly switching the default monospace font from the aging Source Code Pro to something else. Following that discussion led me to discover&nbsp;<a href="https://www.recursive.design/">Recursive</a>.</p> <p>Recursive (Sans and Mono) is a variable font family designed by&nbsp;<a href="https://stephennixon.com/">Stephen Nixon</a>&nbsp;<em>(the site is neat; def check it out)</em>, a graduate of the&nbsp;<a href="https://typemedia.org/">TypeMedia program</a>&nbsp;at the Royal Academy of Art in The Hague. The project began as Nixon&#8217;s thesis work in 2018 and has since evolved into a flexible typeface system for both code and user interfaces. His experience with typographic clarity and microinteractions in digital products inspired him to create a font family that could address common limitations in animated transitions and variable design.</p> <p>One reason Recursive is visually appealing comes from a novel drawing method that Nixon developed. In it, he treats each contour as a separate &#8220;brush stroke.&#8221; While this approach helped shape Recursive&#8217;s unique design, it also led to rendering issues on non-HD screens that required later refinement.</p> <p>Under the hood of this variable font (<img alt="💙" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f499.png" style="height: 1em;" />), Recursive incorporates five&nbsp;<a href="https://fonts.google.com/knowledge/glossary/axis_in_variable_fonts">axes</a>: Casual, Monospace, Weight, Slant, and Italic. This required 24 total source font files&nbsp;<em>for each character</em>.</p> <p>As noted, there are both sans-serif and monospace versions of the typeface. Recursive Sans was built as a &#8220;superplexed&#8221; family, meaning all 32 instances share the same glyph width, kerning, and overall letterforms. This makes for super smooth, animated transitions between different font styles without affecting line length.</p> <p>Recursive Mono also has some cool features, such as the inclusion of true cursive italics as an option (which is great — IMO —&nbsp;for comments and other contextual information in code). It, too, is &#8220;superplexed&#8221;, and it also sports OpenType features like code ligatures and alternate characters (though not for R&#8217;s spiffy new pipe symbol).</p> <p>Since these families are intricate/complex, Nixon also developed a Chrome extension called&nbsp;<a href="https://github.com/arrowtype/type-x">Type-X</a>&nbsp;to efficiently test Recursive on live websites, allowing him to override fonts and assess performance in real-world context.</p> <p>I subbed it out for&nbsp;<a href="https://monaspace.githubnext.com/#learn-more">Monaspace Radon Var</a>&nbsp;in all my editors and terminals. The experience, so far, has been&nbsp;<em>great</em>! And, it does pair very nicely with Inter.</p> <p>The section header is a&nbsp;screenshot of a portion of this edition with Radon Mono in VS Codium.</p> <h3 class="wp-block-heading" id="nanofont3x4">nanofont3x4</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503486" height="784" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-15.png?w=1024" tabindex="0" width="1024" /></figure> <p>One of my teammades convinced a bunch of us to grab one of these&nbsp;<a href="https://shop.m5stack.com/products/m5stack-coremp135-w-stm32mp135d">M5Stack cubes</a>. One of its&nbsp;<em>many</em>&nbsp;features is that it sports a 2.6 inch 320&#215;240 TFT LCD module with a capacitive touch panel. While that&#8217;s cool, it&#8217;s also&nbsp;<em>tiny</em>!</p> <p>It&#8217;s a good thing we all have access to the &#8220;world&#8217;s smallest readable 3&#215;4 font with lowercase; includes all ASCII symbol&#8221;&nbsp;—&nbsp;<a href="https://github.com/Michaelangel007/nanofont3x4">nanofont3x4</a>!</p> <p><a href="https://github.com/Michaelangel007">Michael Pohoreski</a>&#8216;s motivation for developing this font was to push the boundaries of what&#8217;s possible in typography and to explore the essence of glyph design. Pohoreski wanted to answer questions such as &#8220;What makes a tiny glyph readable?&#8221; and &#8220;What is the minimal leading?&#8221;.</p> <p>Pohoreski has suggested this font would be great for rendering in-game book pages that don&#8217;t look like gibberish and creating accurate print previews with scaled-down but legible text. But, it&#8217;s also perfect for this tiny screen (and other contexts).</p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 9, 2024 at 1:11 PM
dailydrop.hrbrmstr.dev.web.brid.gy
Today's potpourri Drop includes the evolution of text abbreviations (that ends in a weather surprise!), a new-ish color palette for accessible UI design, and the benefits of using low-tech developer tools.
<p><em>&amp; U Thot Ur SMS Abbrevs Were Cool; Color Me UI-seful; In Praise of Low Tech DevEx</em></p> <p>Another potpourri of topics to ease y&#8217;all back in from the long, sweltering weekend (at least in the U.S.)</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139503476" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="-u-thot-ur-sms-abbrevs-were-cool">&amp; U Thot Ur SMS Abbrevs Were Cool</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503478" height="332" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-12.png?w=856" tabindex="0" width="856" /></figure> <p>In the late 1990s and early 2000s, a new form of communication emerged that permanently alter how we interact with each other and start the downfall of modern society: text (SMS) messaging. With this new medium came a unique language all its own, filled to the brim with abbreviations and acronyms designed to convey maximum meaning with minimal characters.</p> <p>This <em>textspeak</em> was, in fact, a necessity. Back in <em>my</em> day, text messages were limited to a scant 160 characters, and greedy telcos often charged customers per message sent. This created a strong incentive for we humans to squeeze as much meaning as possible into each text, leading to the creation of creative abbreviations and shorthand.</p> <p>Would you believe we cool tech cats of that era weren&#8217;t the first humans to bask in the cleverness of our condensed creation? Textspeak shares similarities with earlier forms of abridged communication. This goes all the way back (at least in the pseudo-modern era) to the telegraph (where charges were by the word). There were all forms of condensed languages and even codes used to save a pence or two (and, sometimes, to obfuscate the message from the telegraph operator).</p> <p>The telegraph also — with no exaggeration — revolutionized weather forecasting by enabling rapid transmission of weather data across long distances. The U.S. Army Signal Corps (USASC) established a national weather service in 1870, using telegraph lines to collect and disseminate weather information. Now, the USASC weather folks transmitted&nbsp;<em>many</em>&nbsp;weather-related messages a day, to many diverse telegraph stations. Even at the outset, these messages —&nbsp;if not optimized in some way — would start to add up $-wise. So, these clever folks developed their own coded system that used numbers and letters to represent different weather conditions, allowing for quick and concise communication of complex meteorological information. For example, cloud types were assigned specific numbers, and wind directions were represented by letters.</p> <p>Over time, the coding system evolved and became more sophisticated. The World Meteorological Organization (WMO) eventually standardized these codes globally, ensuring consistency in weather reporting across different countries and regions.</p> <p>If you want to see a full example of this, along with a ~5 minute video that tells a tale of a cryptic telegraph message found in a dress from the 1800&#8217;s that led someone to dig into what they might be, head on over to the&nbsp;<a href="https://www.grandforksherald.com/weather/weather-wednesday-telegraphic-coded-weather-observations">Grand Forks Herald</a>.</p> <p>The&nbsp;<a href="https://www.nytimes.com/2024/01/08/us/antique-dress-weather-code.html?unlocked_article_code=1.5k0.03lf.4XIzwKbfc4b6&amp;smid=url-share">NYTimes also covered this</a>&nbsp;back in January, and&nbsp;<a href="https://www.noaa.gov/heritage/stories/cryptogram-in-silk-dress-tells-weather-story">NOAA did a pretty deep-dive back in December of last year</a>.</p> <p>If this piqued your interest, perhaps take a stab at writing a program that groks <a href="https://books.google.com/books?id=i7YazQEACAAJ&amp;pg=PA3&amp;source=gbs_toc_r&amp;cad=2#v=onepage&amp;q&amp;f=false">The Signal Service Weather Code</a> and lets folks encode/decode conditions with it.</p> <h3 class="wp-block-heading" id="color-me-ui-seful">Color Me UI-seful</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503479" height="426" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-13.png?w=1024" tabindex="0" width="1024" /></figure> <p>Making pretty and accessible UIs for anything —&nbsp;TUI, web, or full-on native apps — is not a trivial task. Drilling down a bit, even picking the right colors can be daunting (as we&#8217;ve covered in many previous Drops). Thankfully, accessibility and design researchers keep coming up with new ways to help us use smart colors in those pretty and accessible components.</p> <p><a href="https://github.com/yeun/open-color">Open Color</a>&nbsp;presents a comprehensive color palette optimized for UI elements such as fonts, backgrounds, and borders. The project&#8217;s core objectives include ensuring all colors have practical applications, providing a general-purpose color scheme for UI design, and maintaining perceptual brightness consistency across hues of similar luminance.</p> <p>This Open Color system offers a range of colors and supports multiple formats and development environments, including CSS, Sass, Less, Stylus, JSON, SVG, and various design tools. Its variable naming convention follows a structured pattern (e.g.,&nbsp;<code>$oc-color-number</code>&nbsp;in Sass) which makes it &#8220;easy&#8221; to integrate with existing projects.</p> <p>Now, even with this system, one may still find it difficult to achieve both &#8220;pretty&#8221; and &#8220;accessible&#8221;. Enter&nbsp;<a href="https://ferdychristant.com/color-for-the-color-challenged-884c7aa04a56">Colar</a>&nbsp;(with apologies for linking to Medium) (<a href="https://github.com/fchristant/colar">GH</a>). The article is an extensive guide for those of us who struggle with color selection in design. The author — Ferdy Christant — who self-describes themselves as &#8220;color-challenged,&#8221; shares a very comprehensive color palette based on the aforementioned Open Color system, with some extensions. The palette consists of 234 colors (18 hues x 13 luminosity steps) and is designed for wide applicability in web and app design, illustration, and&nbsp;<em>even print</em>.</p> <p>Christant also discusses various characteristics of a good color palette, including attractiveness, completeness, &#8220;themability&#8221;&nbsp;<em>(my spell checker says that&#8217;s not right but <img alt="🤷🏼" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f937-1f3fc.png" style="height: 1em;" />)</em>, equal distance colors, accessibility, and extensibility. It emphasizes the importance of balancing these characteristics, as they can sometimes conflict with each other. The author praises Open Color for its artistic approach to color selection, which results in visually appealing and consistent color scales.</p> <p>The guide also includes a suite of testing tools and helpers to assist in color selection and combination. These tools allow users to quickly evaluate color combinations, test gradients, and generate variations of atomic UI elements. The author describes these tools as &#8220;game-changers&#8221; for those who struggle with color selection, as they speed up the process and reveal unexpected and attractive color combinations.</p> <p>This resource is definitely a keeper!</p> <h3 class="wp-block-heading" id="in-praise-of-low-tech-devex">In Praise of Low Tech DevEx</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503480" height="698" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-14.png?w=1023" tabindex="0" width="1023" /></figure> <p>Ian Miell (author of a few spiffy &#8220;Learn X The Hard Way&#8221; tomes) has a <a href="https://zwischenzugs.com/2024/07/05/in-praise-of-low-tech-devex/">neat post</a> on relying on developer tools that are:</p> <ul class="wp-block-list"> <li>long-standing and battle-hardened</li> <li>stable with few dependencies</li> <li>text-based and command-line oriented</li> <li>aligned with Unix principles</li> </ul> <p>These tools are generally quite portable, fast, composable, and embeddable. They are also usually accompanied by a steep learning curve, with mega-ROI in the long run.</p> <p>Ian&#8217;s post is very readable, with plenty of examples and tool suggestions. Def check it out!</p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 8, 2024 at 5:20 PM
dailydrop.hrbrmstr.dev.web.brid.gy
Today's Knowledge Drop is a first attempt at helping me avoid doomscrolling by using AI to summarize missed socmed content.

Perplexity successfully generates accurate summaries; and local LLM/GPT Gemma v2 27b also provides fast summarization (on […]

[Original post on dailydrop.hrbrmstr.dev]
<p><em>SummarAIse <s>Proust</s> Socials</em></p> <figure class="wp-block-image size-large is-resized is-style-default"><img alt="" class="wp-image-139503466" height="750" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-8.png?w=1000" style="width: 650px; height: auto;" tabindex="0" width="1000" /></figure> <p>Thanks to (<em>*gestures hands opening wide in front of himself*</em>), the state of modern discourse on both Bluesky and Mastodon is definitely trending &#8220;ugh&#8221;. Most recently, it&#8217;s been the SCOTUS disaster, POTUS race, the climate crisis we&#8217;re in, and a smattering of other topics that all trigger <img alt="🤯" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/1f92f.png" style="height: 1em;" />, with there also being very little practical things an individual can do in the moment to change anything.</p> <p>Rather than try to manage lists, micromanage filters, shunt the &#8220;must reads&#8221; to RSS, or just ignore socmed outright, I decided to do something useful and have our AI overlords give me a summary of that I &#8220;missed&#8221; while not doomscrolling.</p> <p>I&#8217;ll show both how I did this with Perplexity, and show a bit of a start on hacking on a local LLM/GPT —&nbsp;<a href="https://huggingface.co/jartine/gemma-2-27b-it-llamafile">Gemma v2 27b Instruct</a>&nbsp;to do something similar.</p> <h3 class="wp-block-heading" id="summaraise-with-perplexity">SummarAIse With Perplexity</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503468" height="768" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-9.png?w=1024" tabindex="0" width="1024" /></figure> <p>I&#8217;m only covering Mastodon in today&#8217;s post, and to get the home timeline I&#8217;m using&nbsp;<a href="https://dailydrop.hrbrmstr.dev/2024/03/24/bonus-drop-46-2024-03-24-method-to-the-madness/">madonctl</a>. We&#8217;re using &#8220;plain&#8221; mode for the output as there&#8217;s a nasty bug in this semi-old CLI that borks the JSON and YAML output (it messes up post author attribution).</p> <p>This is the CLI command I&#8217;m using:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate"> $ madonctl timeline -k 300 -l 300 -o plain home &gt; status.txt </pre></div> <p>and, this is the prompt:</p> <blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow"> <p>The attached input file is a series of Mastodon posts.</p> <p>Each post from the input has this format:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: plain; title: ; notranslate"> - Status ID: `status_post_id_number` From: `user@instance` (`user_handle`) Timestamp: `status_post_iso_timestamp` Contents: `contents_of_status_post` NOTE that the status post `contents_of_status_post` may continue on multiple lines without spaces in front URL: `url_of_status_post` </pre></div> <p>Create a detailed summary of what I missed, especially &gt;if more than one Status talks about a similar topic.</p> <p>Then, pick at least one or two profound or &gt;well-articulated posts and put them in the output with `user_handle` and `contents_of_status_post`.</p> <p>Then, create a separate summary of posts with `#RStats`, `#rstats`, &#8220;#Golang&#8221;, &#8220;#golang&#8221;, &#8220;#DataBreach&#8221;, or &#8220;#databreach&#8221; hashtags.</p> <p>Finally, include a sorted bullet list of all the hashtags used across all the posts in the attached input file.</p> </blockquote> <p>After pasting the prompt into Perplexity, I attach&nbsp;<code>status.txt</code>&nbsp;to the question and have it do some magic. You can see one incantation of that&nbsp;<a href="https://www.perplexity.ai/search/the-attached-input-file-is-a-s-Jv5QyZ_UQlCRSM06f0WWCw">here</a>.</p> <p>Perplexity automates the painful work of chunking the input to fit the context window and has — very obviously — been configured quite well to deal with topic summarisation.</p> <p>Note that it also generated a couple Python scripts to deal with some of my &#8220;<code>then</code>&#8220;s.</p> <p>It definitely impressed me, and I&#8217;ve run it quite a bit and verified that the summaries are accurate. While it does miss some things I might have wanted to look at, summarising is — essentially — editing, and some things will remain on the cutting room floor.</p> <p>If you&#8217;re curious how ChatGPT fared:&nbsp;<a href="https://rud.is/ex/briefing-chatgpt.html">here&#8217;s the output for the same input file/prompt</a>.</p> <h3 class="wp-block-heading" id="local-summaraising-with-llamafile">Local SummarAIsing With llamafile</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503471" height="480" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-11.png?w=620" tabindex="0" width="620" /></figure> <p>Google recently released their Gemma v2 models, and Justine was kind enough to <a href="https://huggingface.co/jartine/gemma-2-27b-it-llamafile">make <code>llamafile</code>s for them</a>. Both versions of the model run stupid fast on macOS with Apple Silicon (I did all my hacking on an Apple M1 Max laptop). I usually cannot tolerate using local LLMs due to the speed issue.</p> <p>I&#8217;m not about to try to replicate all of what Perplexity did for this post (I&#8217;m hacking on the local version over time), so I settled on a per-user summary.</p> <p>I started up the Gemma llamafile in server mode with:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: bash; title: ; notranslate"> $ ./gemma --temp 0 -c 0 --threads 10 --host 127.0.0.1 --port 4141 </pre></div> <p>The, use the following R code to pull the last ~300 posts from my home feed, filter out ones with little or no content, grouped the content text by user, and fed it to the&nbsp;<code>completion</code>&nbsp;API endpoint:</p> <div class="wp-block-syntaxhighlighter-code "><pre class="brush: r; title: ; notranslate"> library(httr) library(stringi) library(tidyverse) home &lt;- vector(&quot;list&quot;, 8) next_url &lt;- &quot;https://mastodon.social/api/v1/timelines/home?limit=40&quot; for (i in 1:8) { writeLines(as.character(i)) httr::GET( url = next_url, httr::add_headers( `Authorization` = sprintf(&quot;Bearer %s&quot;, Sys.getenv(&quot;MASTODON_TOKEN&quot;)) ) ) -&gt; res res |&gt; httr::content(as = &quot;text&quot;) |&gt; jsonlite::fromJSON() -&gt; posts home&#91;i] &lt;- list(posts) res$headers$link |&gt; stri_split_fixed(&quot;&gt;&quot;) |&gt; unlist() |&gt; stri_replace_first_fixed(&quot;&lt;&quot;, &quot;&quot;) |&gt; getElement(1) -&gt; next_url } home |&gt; bind_rows() |&gt; filter( nchar(content) &gt;= 30 ) -&gt; xdf data.frame( username = xdf$account$username, content = map_chr(xdf$content, \(.x) { read_html(.x) |&gt; html_text() |&gt; paste0(collapse = &quot; &quot;) }) ) |&gt; arrange(username) -&gt; user_posts prompt_tmpl &lt;- r&quot;(&lt;start_of_turn&gt;User The input at the end is a series of Mastodon posts from user %s, with each post on a separate line. Give me a very high level overview of topics they posted about while I was away. Use the following format — it is very important to include the `###` and username — and do not provide extra commentary: ### %s - summary bullet (if more than one topic, add more bullets) %s&lt;end_of_turn&gt; &lt;start_of_turn&gt;Llama )&quot; users &lt;- unique(user_posts$username) users |&gt; map(\(.user) { writeLines(.user) httr::POST( url = &quot;http://127.0.0.1:4141/completion&quot;, encode = &quot;json&quot;, body = list( cache_prompt = TRUE, stop = c(&quot;&lt;/s&gt;&quot;, &quot;Llama&quot;, &quot;User&quot;), temperature = 0.7, prompt = sprintf( fmt = prompt_tmpl, .user, .user, user_posts |&gt; filter( username == .user ) |&gt; pull(content) |&gt; paste0(collapse = &quot;\n&quot;) ) ) ) -&gt; res httr::content(res) |&gt; getElement(&quot;content&quot;) }, .progress = TRUE) -&gt; out unlist(out) |&gt; stri_replace_all_fixed(&quot;&lt;end_of_turn&gt;&quot;, &quot;&quot;) |&gt; cat( sep = &quot;\n------\n\n&quot;, file = &quot;/tmp/briefing.md&quot; ) </pre></div> <p>While it is noticeably slower than Perplexity, it&#8217;s just fast enough for this task to make using it tolerable.</p> <p>The prompt definitely needs some tweaking, but you can see markdown output rendered to HTML&nbsp;<a href="https://rud.is/ex/briefing.html">over here</a>.</p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>I&#8217;ll be migrating the local version to behave more like the Perplexity version, and will post and update when I do.</p> <p>If you made it this far (thanks!), why did a Very British TV Show spell summarise with a Zed?</p> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 7, 2024 at 4:58 PM
dailydrop.hrbrmstr.dev.web.brid.gy
Today's Drop takes a quick look at the recent POSIX.1-2024 Standard, showcases the new digs for Mozilla's HTTP Observatory, and covers Portier — a self-hostable, password-less login service that intermediates between websites and authentication providers.
<p><em>POSIX.1-2024; HTTP Observatory&#8217;s New Digs; Portier</em></p> <p>Two updated and one new (to me) resource on this Friday edition of the Drop.</p> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <div class="wp-block-jetpack-subscriptions__supports-newline wp-block-jetpack-subscriptions"> <div> <div> <div> <p> <a href="https://dailydrop.hrbrmstr.dev/?post_type=post&#038;p=139503456" style="font-size: 16px; padding: 15px 23px 15px 23px; margin: 0; margin-left: 10px; border-radius: 0px; border-width: 1px; background-color: #113AF5; color: #FFFFFF; text-decoration: none; white-space: nowrap; margin-left: 0;">Subscribe</a> </p> </div> </div> </div> </div> <h3 class="wp-block-heading" id="tldr">TL;DR</h3> <p><em>(This is an AI-generated summary of today&#8217;s Drop using Sonnet via Perplexity.)</em></p> <ul class="wp-block-list"> <li>POSIX.1-2024 Standard was published in June, featuring updates like 64-bit <code>time_t</code>, enhanced locking mechanisms, and improved memory allocation. The 4,000+ page document covers various aspects of operating system interfaces [<a href="https://ieeexplore.ieee.org/document/10555529">https://ieeexplore.ieee.org/document/10555529</a>].</li> <li>Mozilla&#8217;s HTTP Observatory, a tool for analyzing web server security configurations, has been relocated and redesigned. It evaluates websites based on security features and assigns scores [<a href="https://developer.mozilla.org/en-US/observatory">https://developer.mozilla.org/en-US/observatory</a>].</li> <li>Portier is a self-hostable, password-less login service that acts as an intermediary between websites and third-party authentication providers, offering a streamlined login experience using email addresses [<a href="https://portier.github.io/">https://portier.github.io/</a>].</li> </ul> <hr class="wp-block-separator has-alpha-channel-opacity is-style-dots" /> <h3 class="wp-block-heading" id="posix1-2024">POSIX.1-2024</h3> <figure class="wp-block-image size-large"><img alt="" class="wp-image-139503458" height="755" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/image-7.png?w=529" tabindex="0" width="529" /></figure> <p>The&nbsp;<a href="https://ieeexplore.ieee.org/document/10555529">POSIX.1-2024 Standard</a>&nbsp;was published in late June. In previous Drops, we&#8217;ve talked about POSIX (Portable Operating System Interface) from a few different angles, and now folks have a new opportunity to look at entire, fresh, new (well,&nbsp;<em>updated</em>) 4,000+ page (O_O) standard. Well, most folks will be able to do that&nbsp;<em>soon</em>, but it&#8217;s stuck behind an IEEE paywall, for now.</p> <p>I do recommend folks poke at it a bit when it does come out, and look for a plethora of posts across the blogosphere that usually swell up after release time. Authors with various POSIX special interest areas will go into deep dives on what has been added/changed.</p> <p>Today, y&#8217;all get two things (below). One is (hopefully) a less daunting breakdown of the sections covered in the standard. The other is a list of some things I found notable after a very quick series of&nbsp;<code>diff</code>&#8216;s and&nbsp;<code>rg</code>&#8216;s.</p> <ol class="wp-block-list"> <li>Introduction and Scope: (kind of self-explanatory)</li> <li>Conformance: This section details the requirements for implementation conformance, including POSIX conformance and XSI conformance. It also covers application conformance and language-dependent services for the C programming language.</li> <li>Definitions and Concepts: A large portion of the document is dedicated to defining terms and concepts used throughout the standard. This includes everything from basic computing terms to specific POSIX-related concepts.</li> <li>General Concepts: This section covers broader topics such as file system organization, process management, memory management, and internationalization.</li> <li>File Format Notation and Character Sets: The standard defines file format notations and specifies character sets, including the portable character set and character encoding rules.</li> <li>Locale: Detailed information on locale definition and internationalization, including specifics on various locale categories (<code>LC_CTYPE</code>, <code>LC_COLLATE</code>, <code>LC_MONETARY</code>, etc.).</li> <li>Environment Variables: Definitions and usage of environment variables in POSIX-compliant systems.</li> <li>Regular Expressions: Comprehensive coverage of regular expression syntax and semantics, including both basic (BRE) and extended (ERE) regular expressions.</li> <li>Directory Structure and Devices: Information on standard directory structures and device interfaces.</li> <li>General Terminal Interface: Specifications for terminal I/O interfaces and control.</li> <li>Utility Conventions: Guidelines for the syntax and behavior of utility programs.</li> <li>Shell Command Language: Detailed specifications for the shell command language, including syntax, built-in utilities, and job control.</li> <li>System Interfaces: This extensive section covers a wide range of system programming interfaces, including process control, file I/O, memory management, inter-process communication, networking, and more.</li> <li>Headers: Definitions of standard C language headers and their contents.</li> <li>Utilities: Specifications for standard utility programs and commands.</li> <li>Rationale: Explanations and background information on the design decisions made in the standard.</li> </ol> <p>If you&#8217;ve never read through any decent portion of a POSIX standard, I&#8217;d suggest starting the sections identified in bullets 4, 8, 11, 12, and 15 (unless you&#8217;re doing very low-level work on POSIX systems). We&#8217;ll likely cover some of those (that we have not already covered in various Drops) in-depth in the fall.</p> <p>When it comes to new or changed items, these seemed to be significant to my biased (focused on safety and resilience) eyes:</p> <ol class="wp-block-list"> <li>64-bit <code>time_t</code>: The 2038 32-bit <code>time_t</code> issue has been addressed, effectively canceling the &#8220;Y2K38&#8221; problem.</li> <li>Enhanced locking mechanisms: <ul class="wp-block-list"> <li><code>F_OFD_GETLK</code>, <code>F_OFD_SETLK</code>, and <code>F_OFD_SETLKW</code> are added for per-file-descriptor advisory locks, addressing issues with process-scope locks.</li> </ul> </li> <li>Improved memory allocation: <ul class="wp-block-list"> <li><code>reallocarray()</code> is introduced to reallocate arrays with a multiplication check to catch overflows.</li> <li><code>aligned_alloc()</code> from C11 is added for allocating memory with specified alignment.</li> </ul> </li> <li>Enhanced randomness generation: <ul class="wp-block-list"> <li><code>getentropy()</code> function is added to obtain cryptographically secure random data.</li> </ul> </li> <li>Internationalization improvements: <ul class="wp-block-list"> <li><code>&lt;libintl.h&gt;</code> header is standardized, including key interfaces from <code>gettext</code>.</li> </ul> </li> </ol> <p>There are, of course, many more changes/additions to the internals.</p> <p>When it comes to things you and I may use more frequently:</p> <ul class="wp-block-list"> <li><code>set -o pipefail</code> is now a POSIX standard (meaning it will eventually work across all shells that claim to be POSIX compliant).</li> <li>The section on Awk has been updated to account for the &#8220;recent&#8221; update to the venerable data wrangler.</li> <li><code>cd -e</code> is safer as it checks for existence before changing directories and fails with a status and message.</li> <li><code>head -c</code> is standardized across POSIX systems. It lets us output a specific number of <em>bytes</em> from the beginning of a file or input stream, rather than a number of lines.</li> </ul> <p>I&#8217;ll be watching for more changes that others notice.</p> <p>Let me re-encourage y&#8217;all to at least poke at some of the aforementioned sections, and keep the standard around (when it&#8217;s free) to reference when doing shell ops.</p> <h3 class="wp-block-heading" id="http-observatorys-new-digs">HTTP Observatory&#8217;s New Digs</h3> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139503460" height="1300" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/pexels-photo-994164.jpeg" tabindex="0" width="1300" /><figcaption class="wp-element-caption">Photo by Stephanie Ho on <a href="https://www.pexels.com/photo/person-holding-paint-roller-while-painting-the-wall-994164/" rel="nofollow">Pexels.com</a></figcaption></figure> <p>Mozilla&#8217;s&nbsp;<a href="https://developer.mozilla.org/en-US/observatory">HTTP Observatory</a>&nbsp;got a new home and a redecoration!</p> <p>If you&#8217;re not familiar with it, it&#8217;s an open-source project that conducts comprehensive analyses of web server configurations, focusing on critical security headers and protocols.</p> <p>The Observatory&#8217;s methodology involves assigning an initial score of 100 to each website, which is then adjusted based on the implementation of various security features. Penalties are applied for missing or misconfigured protections, while bonus points are awarded for exemplary security practices, allowing scores to potentially exceed 100. This scoring system,&nbsp;<em>while inherently subjective</em>, is grounded in industry expertise and aims to incentivize the adoption of cutting-edge web security technologies.</p> <p>Key areas of evaluation include Content Security Policy (CSP) implementation, HTTP Strict Transport Security (HSTS) configuration, and protection against cross-site scripting (XSS) attacks. The Observatory also assesses the proper use of secure cookies and the deployment of advanced features like subresource integrity.</p> <p>If you&#8217;ve never tried it out before, I do suggest carving out five minutes to do so. And, if you want to know the &#8220;how&#8221;, check out&nbsp;<a href="https://github.com/mozilla/http-observatory">the repo</a>.</p> <h3 class="wp-block-heading" id="portier">Portier</h3> <figure class="wp-block-image size-full"><img alt="" class="wp-image-139503461" height="1250" src="https://dailydrop.hrbrmstr.dev/wp-content/uploads/2024/07/pexels-photo-1591062.jpeg" tabindex="0" width="1880" /><figcaption class="wp-element-caption">Photo by Miguel u00c1. Padriu00f1u00e1n on <a href="https://www.pexels.com/photo/email-blocks-on-gray-surface-1591062/" rel="nofollow">Pexels.com</a></figcaption></figure> <p><a href="https://portier.github.io/">Portier</a>&nbsp;is a new (to me) self-hostable login service that is&nbsp;<em>not</em>&nbsp;password-based. It sits between a website/webapp and 3rd-party services like Google Sign-In to provide humans who use your kit a pretty nice login experience, without ever needing a new password.</p> <p>This is how it works:</p> <ol class="wp-block-list"> <li>Someone Initiates Login: A visitor taps a website and selects &#8220;Log in&#8221;. Instead of entering a password, they enter their email address.</li> <li>Website Requests Authentication: The website sends an authentication request to its Portier broker (a self-hosted or cloud-hosted service).</li> <li>Broker Verifies Email: The Portier broker checks if the email domain supports OpenID Connect (e.g., Google accounts). If so, it redirects to that provider. If not, it falls back to email verification.</li> <li>Email Verification (if needed): For email verification, the broker sends a one-time link to the individual&#8217;s email address.</li> <li>Visitor Confirms: The visitor taps the link in their email, which opens in their browser and redirects back to the Portier broker.</li> <li>Broker Issues Token: Upon successful verification, the broker issues a signed JSON Web Token (JWT) containing the visitor&#8217;s verified email address.</li> <li>Website Validates Token: The website receives and cryptographically verifies the JWT from the broker.</li> <li>User Logged In: If the JWT is valid, the website considers the visitor authenticated and logs them in.</li> </ol> <p>The&nbsp;<a href="https://demo.portier.io/">demo</a>&nbsp;is a great showcase of how it works.</p> <p>NOTE: this service only covers the&nbsp;<em>authentication</em>&nbsp;part. You need to handle the&nbsp;<em>authorization</em>&nbsp;bits.</p> <h3 class="wp-block-heading" id="fin">FIN</h3> <p>Remember, you can follow and interact with the full text of The Daily Drop’s free posts on Mastodon via&nbsp;<code>@dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev</code>&nbsp;<img alt="☮" class="wp-smiley" src="https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/72x72/262e.png" style="height: 1em;" /></p>
July 5, 2024 at 2:28 PM