When you let LLMs build your app without oversight, you don’t get innovation.

You get 59.3GB of human collateral.

→ AI tools are shipping insecure code at scale
→ ID verification is becoming normalized (and dangerous)
→ Fast growth ≠ secure growth
→ Trust in “safe space” apps is collapsing

The era of “vibe coding” just met its reckoning.

Women uploaded their most sensitive information to feel safer.
Instead, they were exposed.

Because the developers cut corners - possibly with #AI, definitely without care.

More importantly, this is bigger than this one app.

↳ #AI-built code with no guardrails
↳ Misconfigured Firebase (again)
↳ Mandatory ID uploads never deleted
↳ Viral growth with zero time for audits
↳ And a growing trend of devs shipping based on "vibes," not security

Let’s be clear: this isn’t just a data breach. It’s betrayal.

→ Now that data is circulating on 4chan and Reddit.
→ Complete with searchable maps.
→ Doxxing. Harassment. Identity theft.
→ All from an app that promised protection.

What went wrong? Everything:

Prompt: “Build a dating app with selfie verification.”
Output: Firebase bucket wide open to the internet 🫠

Tea marketed itself as a “whisper network” to warn women about dangerous men.

Because someone forgot to secure the storage bucket.

Literally: no password, no encryption, no nothing 🤯

The crazy part?

This wasn't some elite hacker attack.
It was basic negligence - amateur AI-generated code deployed at scale without security reviews.

Unsurprisingly, the workforce didn't shrink - it actually grew 🤷‍♂️

The lesson?

Companies chasing AI blindly risk costly U-turns.

IBM learned this the hard way.

The AI fell short, unable to fully replicate human nuance and critical judgment.

Forced into a strategic pivot, IBM rehired - not the same roles - but specialists whose skills AI couldn't touch: software engineers, strategic thinkers, and client-focused roles.

Augmentation, Not Automation

Embrace GenAI as a tool to enhance and elevate human capabilities, not replace them.

The Path Forward:

First Principles, Not Metaphors

Think about the core capabilities of GenAI and how it fits into your business.

Clear Thinking, Not Brand Buzzwords

Use clear, accurate language to understand GenAI’s real potential, and avoid marketing speak that clouds judgment.

Instead of chasing efficiency at all costs, we need to ask the right questions:

How can GenAI elevate human judgment?

How can it augment creativity, not automate routine tasks?

How can it help us see patterns we couldn’t see before?

It’s a mirror, not a mind.

There is no Hype-as-a-Service (HaaS) in science.

In science, GenAI is used with clear, grounded expectations, augmentation, not magic.

In business, HaaS rules, where the focus is more on selling the promise of AI, not the reality.

When treated like a “Chief Productivity Officer,” GenAI is tested with human-like productivity assumptions, resulting in failure rates as high as 70%.⁵

Inflated Expectations

The hype about “automating everything” leads to disillusionment when GenAI doesn’t deliver on promises.

The confusion in business surrounding GenAI leads to:

Misguided Investments and Lazy Analogies

Companies imagining GenAI as a “smart intern” waste money on unrealistic expectations.

Failed Pilots

It doesn’t reason. It predicts.

It can't replace anyone, it's not qualified.

It doesn’t understand. It correlates.

It needs to be managed as it can’t be trusted

It doesn’t “know” truth. It reflects training data.

It doesn’t have intent, memory, or values.

They are solving problems and using GenAI as the ultimate augmentation partner. It's breathtaking to see what they've accomplished.

But business leaders continue to put on a theatrical performance with their management consulting producers

Here’s what science has known for a while:

But hey, at least it’s entertaining