As I'm writing this the hottest topic these past few days has been Jev. It's a whole new model from this company called Typesafe AI Which is founded by one of the initial GPT makers: Diogo Almeida.

Now what Jev is, is a relatively small model with about a 32k context window, where you give it some sort of input and it produces a structured typed output. It cannot produce free-form text like an LLM does. So you can use it for things like yes-or-no questions, classification with predefined answers, and so on.

The magic is exactly in that: being such a small model and only taking input tokens and producing virtually no output, because the typed structured responses are very small in size, it turns out to be a very low-latency, very cheap model.

If you follow me you know that I have a few agentic open source projects. One of them is the job finder. It scrapes ATS websites, looks through jobs based on your predefined profile, evaluates if they're a fit, and then builds a queue for you to review and eventually apply to these jobs. Looks like this now:

I also have a news aggregator agent that scrapes the web and YouTube as well for Romanian news and then filters them for national economic relevance, groups them by theme, and summarizes them to produce a daily report. This one is not open source yet but it's coming soon.

What I'm trying to impress upon you is that I genuinely have various jobs within my agentic systems, which are precisely yes-or-no questions or sometimes simple predefined classification jobs.

So naturally, with everybody going crazy over Jeff, I just had to try it and run it through my eval datasets.

As you can see in these images here, compared to the models which I mostly use right now in production for my two agents, Jev has been doing very strongly qualitatively speaking for yes/no jobs.

From the perspective of latency it's not been as crazy as other people have reported on X but I suspect this is because I live in Romania and most likely the Jev servers are within the USA.

What you'll also notice is that it is basically 6 times cheaper than even Gemini 2.5 and, of course, many times cheaper than Gemini 3.8.

That cost translates into meaningful savings. My news aggregator checks around an avg of ~500 articles per day for relevance so with Jev I'll be spending $0.13 instead of $0.77 with Gemini 2.5 Flash. That's something.

When it came to more complex non-yes/no jobs, the waters became a bit more murky.

For news grouping what I do in production is to just throw a bunch of articles at Gemini 3.8 Flash and let it produce topic-based groups of articles. I call these groups themes. They have their own summary and point to their respective articles.

On ranking and tiering I take these themes and once more send them to Gemini 3.8 Flash to split into rank tiers of main_news, side_news, and left_out.

This flow has been producing very good results for me. But obviously I can't do this with Jev because, for one, it cannot produce the team summaries. Even more important than that, Jev just doesn't have the context window to take a few dozen articles and do this work for me.

In an effort to compare them I split the work and had the 3 models answer yes-or-no questions on pairs:

As you will be able to see, when you structure the prompts this way, it does turn out that Jev is obviously significantly cheaper, with lower latency, because Gemini 3.8 Flash does some reasoning and it seems like everything is beautiful.

But this is not a fair test because, as I have explained, that is not what happens in production. We are not comparing pairs and if you were to apply something like a merge sort with Jev calls to rank articles, I’ve no idea where that would take you (might try tho, lol).

My point and conclusion, though, is that based on the testing I've been doing so far with Jev you should use it if you have a simple job (yes/no, predefined option classification) and want it done fast, very cheap and well enough.

If you have to split a big prompt into a # of prompts/calls, it may be a questionable idea.

Recently I have demoed a new version of my job finder on my YouTube channel, as you can see here. I have made a lot of improvements to it so if you want to try it, please do. It might genuinely help you find a job if you're looking.

But what I want to talk about today is not that. I want to talk about lessons that I've learned while improving the job finder and while building my news aggregator.

As you probably already know, if you're building apps with AI, if you're building agents, a big thing you should be aware of and do is set up evals. What evals are is mainly just synthetic or manually labeled datasets which identify what is a good output and what is a bad output.

In the context of a job finder you might have 50 examples of jobs that should pass and 50 examples of jobs that should not pass the relevancy check for a given profile.

Since AI is non-deterministic you will never get 100% accuracy in identifying these jobs correctly. But you want to have this eval dataset that you use to test your system against to see if the change that you make to a prompt, or if you change a model, or if you fine-tune the model actually improves the accuracy of the system.

Well building these eval datasets often requires tedious boring manual labeling work unless you want to have a frontier LLM do it but then you have to trust that LLM.

I have personally done this tedious work in a number of ways up until now. Initially for the job finder I was building these datasets in the actual terminal with a Claude skill. That kinda worked. I could have Claude help me remove some of that tedium and I would also apply to the jobs as I went because it would show me a link to them, but I had to always go back and forth between terminal and my then Notion CRM.

After that I tried LangSmith with its annotation queues but the problem here is that the annotation queue wasn't really tied to day-to-day usage so I would have to go out of my way (after having applied to the jobs in the job finder) to go to that annotation queue manually. And the fact that I had to go out of my way to do it meant I only got to it every once in a while and so by the time I got there, the queue grew ginormous. And so now I would create a self-defeating spiral of not doing it because it's too big, it's growing bigger, and so on.

Eventually and practically after talking to Matt Ferrante I realized that you can just build them while using. So now all of my AI apps have a real frontend (not Notion anymore) and have various feedback forms within them like this below.

This is genuinely really good because it's in the same spot as your actual app usage. It removes all friction. You just use the app and whenever you see an output that you really loved or really hated, you can very easily just say that.

In my case this feedback gets saved in a normal Pg database, and then every once in a while I use an LLM to pull all the feedback since last time, try to find patterns, and move it into the eval set within LangFuse. I run my experiments, improve my prompt, or change my models or improve the system in whatever way I have to. And we're back to the races.

If you paid a lot of attention as you were reading, you will notice that initially I said I tried LangSmith and its annotation cues but I ended up using LangFuse to store my eval sets (and traces btw).

And it's not because I'm some self-hosted open-source nutcase. I'm perfectly fine with using proprietary software. I don't have any philosophical issues here.

It's more so because of the pricing mechanism that LangSmith uses. They charge you per the root trace and then allow up to 10,000 spans within that trace. If we are being honest this scenario of you having 10,000 spans within one root trace within one LLM call is not realistic. So what tends to happen is that you underutilize the amount of spans within the root trace and you have to use a lot of root traces so your credits get eaten alive.

With LangFuse it's entirely different. They charge you for one unit and one unit can be the trace, it can be the span, it can be a span under the span, it can be whatever you want it to be. This allows you to use the observability in natural normal ways without having to model how you make LLM calls to fit the pricing model of LangSmith.

I completely migrated my stuff to LangFuse. I used the managed service but they also offer a self-hosted open-source version, which you can use, similar to how you might use Grafana which is pretty cool. It's also made by the guys that use ClickHouse and I don't know about you but they make pretty cool stuff in general so I like them.

I will leave you with one last thing to try. Recently I've heard about this thing called Beads.

At first sight it seemed kind of stupid to me. I was like, "What is this? Just GitHub issues, but without GitHub issues, or what exactly?". But a friend told me to try it and I was like, "Fine." Upon trying it, it genuinely makes a lot of sense.

It basically, yes, really is like “GitHub issues but without GitHub issues” and that's exactly the selling point. It's lighter weight. It's faster for the agent to work with. You as the engineer don’t have to see it. It’s git-based, so relatively safe from race conditions and whatnot.

You might still use GitHub issues or Linear issues for larger or human-facing initiatives and your agents use beads under the hood.

It does a really good job of helping you keep the agent on track over long sessions where compaction might happen multiple times or in multi-agent sessions.

It's also saved me more than one time when my cloud agent sandboxes shit the bed and lose data.

That's it for today. See you next time!