Open models still haven't caught up to ChatGPT's initial release in 2022. Now that the training data is so contaminated (internet is now mostly LLM slop), they may never.
Also, OpenAI's only real moat used to be the quality of their training data from scraping the pre-GPT-3.5 Internet, but it looks like even they've scratched that too.
Er, what? We've had open models that can outperform ChatGPT 3.5 for several years now, and they can run entirely on your phone these days. There is no metric by which 3.5 has not been exceeded.
Not in the creative writing I care about. I've been looking for years and trying new models practically every month, including closed, hosted models. None of them approach the quality of the logs I have from that original release.
This. ChatGPT Pro personal at $20/month and using GPT 5.4 xhigh is the best deal currently. I don't know if they are actually losing money or betting on people staying well under limits. Clearly they charge extra to businesses on the API plans to make up for it.
In the future, open models and cheaper inference could cover the loss-leading strategies we see today.
> I wonder how much adding a profiler to development flows would help modern apps.
Very much, but ideally you want telemetry on the user's device (assuming desktop app). Or your "optimization" might come back as a regression on the Snapdragons you didn't test on.
> I work on 3D/4D math in F#. As part of the testing strategy for algorithms, I've set up a custom agent with an F# script that instruments Roslyn to find FP and FP-in-loop hotspots across the codebase.
I don't know if there is an equivalent in Roslyn, but in Julia you can have the agent inspect the LLVM output to surface problems in hot loops.
JAX is designed from the start to fit well with systolic arrays (TPUs, Nvidia's tensor cores, etc), which are extremely energy-efficient. WebGL won't be the tool that connects it on the web, but the generation after WebGPU will.
reply