fenic's Local Data Caching & Persistence keeps expensive AI steps from rerunning and your pipelines resilient.

fenic's Multiple Model Configuration & Selection lets you pick the right model for each step, cheap where you can, powerful where you must.

Think of it as a per-operator model dial across your pipeline.

fenic's Structured Output Extraction turns LLM text into validated tables, directly in your DataFrame.

Think of it as schema-first parsing: you define a Pydantic model; Fenic enforces it and returns structured columns.

fenic's First-Class AI Data Types make embeddings, markdown, and JSON real, typed columns, with the right operations built in.

Think of it as strong types for meaning and structure: safer pipelines, richer queries.

fenic's Semantic Classification turns free-text into clean enums right inside your DataFrame.

fenic's Semantic Similarity Join (Vector Join) finds nearest neighbors across tables using embeddings, right inside your DataFrame.

fenic 0.4.0 is live: declarative tools for agents, a production-ready MCP server, and direct reads from HuggingFace plus big DX & reliability gains. 

Highlights:

Declarative tools: define function-calling tools as data (type-safe, reviewable, reusable).


fenic ensures LLM outputs conform to schemas using Pydantic models.

fenic offers standard DataFrame operations with a familiar Spark/Pandas-like API.

fenic UDFs allow you to inject arbitrary Python (including external libraries) directly into the fenic execution plan while preserving lazy planning, metrics and reproducibility.

fenic offers Flexible Session Configuration. Define AI providers/models (OpenAI GPT‑4, Claude, etc.) and RPM/TPM rate limits at session start so the framework centrally manages throttling, costs and consistency across all AI calls.

Example: group support tickets by account_id, sort by time, produce one weekly summary per account. Think of it as the reduce() for unstructured text.

Semantic Reduce = apply LLM aggregation at group-by scale. Many rows in → one summary per group. You write a prompt and fenic handles packing, ordering, and multi-pass reduction.

This example turns name, details into a one-line job blurb. Think of it as the map() for LLM reasoning inside your DataFrame.

Today, teams hand-roll loops and JSON parsing, fight rate limits, and wire vector lookups. Glue grows. Throughput drops and pipelines become increasingly brittle.

Semantic Map is how you apply inference at scale with fenic. Whether it’s 1 call or 1M, it’s ~5 lines.

You iterate on data and prompts; Fenic handles the rest. Write a Jinja template with column placeholders; Fenic renders it per row and calls the model.

AI-native data types in Fenic. in this demo: the Markdown type.

Fenic adds AI-native scalar functions to DataFrames. They run LLM inference over columns/rows, so reasoning lives inside your pipeline.

Semantic join is one of Fenic’s AI-native DataFrame functions. They operate over whole tables and relationships—not just individual rows.

What an amazing night hosting some of the Peninsula’s brightest minds in data. Delicious tacos, amazing tech talks, and the 2025 view of data by Tomasz from TheoryVC!