I'd say "a verifier" here is a loose term. A great testsuite is a verifier. I've done reverse-engineering projects that involved generating trace logs from the object under test, having a reimplementation emit the same logs, and running strict comparisons.
OP's post is basically pointing out what certainly many others have independently discovered: Your agent-based dev operation is as good as the test rituals and guard rails you give the agents.
I used it (well, a skill based on the same idea) to optimise a prompt that does data extraction from UGC.
However there isn't really a "correct" answer that's easy to define in code (I could manually label a training set, but wanted to avoid that) so I had the LLM just analyse the results itself and decide if they are better or not. It wrote deterministic rules for a few things, but overall it just reviewed the results of each round and decided if the are better or not.
Reviewing the before and after results, I would say yes, it's a big improvement in quality. It also optimised the prompt size to reduce input tokens by 25% and switched to a smaller/cheaper model.
Can you explain your question a little more? The recursive agents will find the minimum to satisfy the deterministic termination condition including cheating. In other words, it will be literally correct yet wrong. I would go so far to say malicious compliance.
I have recursive agent that finds trading strategies after recreating academic research and probing the model using its training on everything. It works really well but I have to force it to write out every line and write a proof that data in the future from the time of the wall clock didn't enter the system. Even then some stupid thing like not converting the timezone with daylight savings will allow it to peek into the future 1 hour. These types of bugs are almost impossible to find. Now there needs to be another agent whose only purpose to write out every line explaining that the timezone for that line of code was correct.
Its tangential, but: I’m currently doing a rewrite of the backend of a project, and the verifier is basically the instruction of “maintain v1 functionality if observed from the api side externally”. This allows making a lot of tests based on existing data in the system and how the frontend expects data.
The thing they are really wildly behind on is a business model. They are losing wild amounts of money per customer and it is hard to see how the competitive situation is going to allow them to fix that.
Thing is, if you're using Codex, you're supporting Sam Altman and the idea of Sam Altmans, in the same way that if you use X or buy a Tesla, you're supporting Elon Musk and the idea of Elon Musks. That's a pretty big tax to factor into the usage of such products. If you even got 5% better coding results, would that make up for the future they're trying to build?
Dario wants to replace you with AI as well. Don't be fooled into thinking he's your friend because he said no to Trump that one time. I'll remind you that Musk used to be the left's hero not too long ago.
I'm in the "AI could be good for humanity" camp, and in this camp, we believe that Dario/Anthropic is a radically better choice going forward than the alternatives at this moment. In this camp we are not 'fooled into thinking he's our friend because he said no to Trump that one time', we are evaluating the entire set of available information and figuring that Anthropic's the best bet.
As for Musk ever being "the left"'s "hero" -- that's amazing, that's what Pauli would call 'not even wrong'.
Funny of you to bring up "humanity" while singing praise of the guy who's on record A-OK with aiding mass surveillance of anywhere not the U.S., and who's happy to help kill people but just wouldn't do it fully automatically, merely because he doesn't believe the tech is there yet. All while collaborating with Palantir.
If by "the best bet" you mean slightly less shitty bet then maybe.
I'm getting pretty close too, but I wouldn't switch to Codex I'd switch to one of the open agents that can use any backing LLM. My reasoning is that if I'm willing to pay the cost of the small changes in usage, I might as well switch to an open source agent that I can add my own convenience features to, like remote sessions and phone-based operation.
Why Codex when you can use something that hasn't been touched by Sam Altman? Surely, your drive to get the very best model isn't stronger than your sense of ethics?
What would be subscription customers, no? Rather than Bedrock or per-api customers? Many of the companies running on Bedrock or by-use have per day limits above the max monthly subscription costs.
They are going to lose, the question is how much. I don't see Russians losing the territory they occupy right now, unless some black swan event happens. It remains to be seen how much more territory Russia is able to conquer until they lose their appetite.
This is a common story, that Ukraine has lost control over a vital part of it's country, and that is true.
But so has Russia, they also have lost control over their stuff, stuff that Ukraine are constantly blowing up.
Ukraine now have a capable long range offensive capability which allows them to strike deep into Russia.
Russias vital strategic assets are slowly depleting due to Ukrainian strikes, while Ukraine has better support from the rest of the world and are now in fact cooperating with some powerful nations across the world.
Russia are managing to advance a few square kilometer of territory per month? while losing expensive, hard to replace assets continuously.
Have you seen how the Russians have treated conquered territories with torture chambers and having to submit to the dictatorship for life? And life is often only a few months as they round the males up and force them into the Russian army to get killed.
And to replenish their losses Ukrainian regime snatches men on the street. There are thousands of videos made by bystanders.[0] Come and see.
P.S. I love the ad I see on that site: "Over 10 million Ukrainians suffer from anxiety due to the war. Free exercises with scientifically proven effectiveness." The most important exercise now is running - it saves your life.
The Ukrainians seem to be dealing with it by switching to bots, drones and the like as much as possible and moving human resources back.
I'm not sure about the Russians. That would kind of make sense for them too but things seem a bit gummed up by bureaucracy - I just read a thead about them having to use firecrackers in dones due to such restrictions https://x.com/ChrisO_wiki/status/2049026651544023271
I hope Russia manages to get some more sensible leadership or policies. The Ilya Remeslo guy being let out and able to criticize Putin seems somewhat promising.
> I just read a thead about them having to use firecrackers in dones due to such restrictions
Haven't read the thread but the post omits crucial detail. This is about using interceptor drones inside Russia, not on the frontline. Apparently, the thinking is that failed interceptor drones present hazard of their own, but it might be outdated now.
>I hope Russia manages to get some more sensible leadership or policies.
Like what?
More and more people in Russia are unhappy with Putin dragging his feet with so called special military operation. They think that it's long overdue to turn to total war and forget about minimizing civilian losses in the Ukraine.
Well an obvious solution would be to back to Russia and do something else. You don't have to invade other countries and have an empire. I'm a Brit and we gave up on that about a century ago and it hasn't been so bad. The whole thing seems anachronistic, I think based on Putin reading too many history books and avoiding modern info on the internet.
>I'm a Brit and we gave up on that about a century ago and it hasn't been so bad.
Wait until Scotland or Northern Ireland gets independent and then China or some other powerful country "midwifes" an anti-British coup there and then we'll talk.
6. Please don't look at our financials. They are horrible and we are hoping to sucker people into an IPO before all of this implodes. The least your Grandma can do for us is give us 2% of her S&P 500 portfolio so we can exit before it goes to zero. This is AGI after all.
I don't think their social networks are allowed in China.
From your link it looks like they might do R&D for Oculus in China (but may not even be able to sell it there due to the data-collection tie in required).
Not sure what you mean by catering to the export market. b2b sales would be just as restricted as sales to consumers.
>Nobody at Anthropic is taking a 5 week summer break or working a 35 hour week
The people working at Mistral, Expedition 33, or other top successful software coming out of the EU, most likely also aren't working only 35 hours/week either. In fact some probably squeeze some work on weekends too out of dedication and pressure to meet deadlines.
In a lot of Austrian SW companies for example, have "all-in" contracts where you waive your rights to the scrutiny of the standard 38,5h/week in exchange for a "higher" salary with longer work hours and less time tracking. Similar cases in France I believe.
The 35h/week European meme people here parrot, you mostly see only in civil servants, old established monopolistic companies with moats and strong unions, not in scrappy start-up trying to make it and fix a bug before release, or semiconductor companies fighting a tape-out.
So no, work hours aren't what's limiting EU startups.
I’ve worked most of my career in US tech satellite offices and I have not experienced EU team members to be less productive than US team members, nor spend less time on work (if anything, more really since they also need to be available for US time zone overlap).
It’s true there are chill jobs here, as there are in the US.
But ambitious people tend to work as much as ambitious US people (and it’s really more like 40 hours work weeks - 39,5 where I live since lunch is not work time). But again, many are not really counting, it’s just a full time job.
Vacations (typically 3 weeks summer holiday and additional weeks to distribute over the year) does create longer time on skeleton crew. Skilled tech labour is also cheaper so you can just hire more to make up for it.
It is interesting to think how AI will potentially change the dynamics back to this from general purpose software.
In a world where implementation is free, will we see a return to built for purpose systems like this where we define the inputs and outputs desired and AI builds it from the ground up, completely for purpose?
Probably. It’s already happening with SaaS as an example. I’ve mentioned this on HN a lot in past but my (established) company has been rolling its own CRM and some other tools with AI.
It seems we can build a product ourselves in the same time it would take us to talk to saas vendors and draft the RFP/requirements. We can build it and iterate as the requirements are being forged, so can essentially have completed software with just the features we care about, with full ability to add features in future (something saas doesn’t promise) often before an implementation would even kick off. We’re searching through all our SaaS products and i expect we’ll cut 50% of them in 1-2 years. The ones that are sufficiently complex or regulated have some protection (like accounting systems).
reply