The OpenAI apps are a joy to use and miles ahead of the competition. Maybe the price of a polished UX is uneven platform support in the short term.

You just always specify your custom adapter when you build the Req request.

Gotcha. I think thats doable?
Req delegates the request to Finch. Roll your own adapter:

adapter = fn request ->
if System.get_env("MIX_ENV") == "test" do
response = %Req.Response{status: 200, body: "it works!"}
{request, response}
else
Req.Steps.run_finch(request)
end
end

Interesting! I've found Req.Test works well for high-level testing.

Req.Test.stub(GithubRequest, fn conn ->
Req.Test.json(conn, %{ status: "OK"})
end)

assert :ok = GithubAPI.close_issue(id)

Guessing you're testing lower-level? Building the request?

This isn't to say 2.0 is no good, but clearly I can't treat an updated LLM like I would an updated dependency in my application. It's more like a junior team member has moved on and someone new started. They are also pretty good but slightly different. Also, prompt evals are going to be important.

I added an example to the prompt and the diarisation is back, but now in JSON format. Which is fine and quite useful, but obviously problematic if you're relying on the output to be the same as Flash 1.5.

I believe all the cities listed in that screenshot would be similar: Los Angeles is PDT in the summer, while Sydney is AEST in the winter. In June I think you'd see BST for London.

Also: IEx.configure(auto_reload: true) 😍

Here's an example of a much better timezone picker from the Fastmail calendar: the timezone abbreviation is easily identified, and you can search by city or timezone.

I think it’s even hard for software developers who have come to expect their tools to be deterministic.

We are still in the process of early adopters experimenting and setting norms for how to use them, but at the same time, everyone has a chance to come onboard. I’m excited about experimenting but it’s not surprising that people who just want their tools to work are coming away frustrated.