So after a day of work I had my 3 line fix :) Add 'SHF_GNU_RETAIN' flag to the inline asm for the main flag list. And a dummy variable in the macro that uses the view sections to force a GC reference:
`__attribute__((retain,used,section("view-section")))
static const char dummy[0];`

Of course this is where I ended up at maskray’s blog doing deep reads of linker GC (maskray.me/blog/2021-02... ) and metadata sections (maskray.me/blog/2021-01...). I realized the main flag list should always exist and the view should only be kept if in use.

There has actually been a ton of consideration to handling these sort of “meta” sections that have no direct access and thus do not participate in general GC references. But upon further looking the dynamic flag didn’t use any of the special flags. So I found my bug, the linker was GCing it

Luckily I had two build flavors with one passing and the other failing. I manually went through every linker flag difference to guess which one was causing issues. And '—gc-sections' looked suspicious. The linker has some crazy complexity to how it decides to GC sections.

So dynamic flag automatically generates sections using inline assembly. One section has the list of all flags. Then there are arbitrarily many other sections that are a “view” into that list of flags for more efficient lookups. There is no access of any sections beyond using '__start' and '__stop'

But the sections existed in the object files leaving me flummoxed. Why wasn’t the linker using the sections? So I went down one of the most interesting systems rabbit holes I’ve had recently.

Google blacklisted the iCloud private relay ip addresses. Been broken for ages while now. On iOS safari you need to press the “show ip address” option and then try searching again and it works.

Everyone complains about fighting the compiler, but the true boss battle is the linker. I’d rather have a thousand line c++ template error than an utterly unhelpful 5 line linker error.

The motivation they cite in the kernel docs is gpu buffer locking www.kernel.org/doc/html/v6....

Yep agreed. I initially mistook synchronous_commit it for a general async durability flag.

Should have read it closer the first time :) Since it isn’t a system wide setting, my personal way to think about it would be readers aren’t defining durability, they are defining if they allow dirty reads. Since you aren’t wholesale changing the definition of commit (that async dura does)

I read deeper into synchronous_commit and it seems like a flavor of early lock release that can be turned on per-txn. But b/c they are encoding commit dependencies by using the wal log, there is no option for non-dirty reads. (If you consider committed to also mean durable).

So serializability also wouldn’t care about durability.

Synchronous commmit means txns are committed after being durable. Isolation levels give guarantees around committed data. So disabling synchronous commit means durability is now separate from the meaning of “commit”. This behavior general to all databases that are ACID and can do async durability.

Since you have the opportunity to build your transaction subsystem from scratch I highly recommend graefe’s on transactional concurrency control. Its got novel ideas, citations going back to the 80s, and some spicy opinions. Pessimistic concurrency control >>> link.springer.com/book/10.1007...

At singlestore we embed a wasmtime engine and compile/run wasm UDF functions docs.singlestore.com/cloud/refere...
snowflake/databricks support Python udfs. To go along with the notebook workflow