Hey 'software engineer', how much of the output of an LLM it's actually reproducible vs the one from a calculator or any programming language with the same input in different sessions?
A lot vs a human? I bet the LLM with the same prompt will write same code as before more often than I would (given I don’t remember what I wrote in the past).
Not really related to this 'discussion' but this is an interesting problem in the AI space. It's essentially a well understood problem in unreliable distributed systems - if you have a series of steps that might not respond with the same answer every time (because one might fail usually) then how do you get to a useful and reliable outcome? I've been experimenting with running a prompt multiple times and having an agent diff the output to find parts that some runs missed, or having it vote on which run resulted in the best response, with a modicum of success. If you're concerned about having another layer of AI in there then getting the agents to return some structured output that you can just run through a deterministic function is an alternative.
Non-determinism is a problem that you can mitigate to some extent with a bit of effort, and is important if your AI is running without a human-in-the-loop step. If you're there prompting it though then it doesn't actually matter. If you don't get a good result just try again.
I could do, but the end goal is to scale this to 100x what I can do myself, and there isn't time to review all those changes. By attempting to answer the problem when it's tiny and I can still keep it in my head then I'll end up building something that works at scale.
Maybe. The point is that this is all new, and looking forwards I think it's worth figuring out this stuff early.
Why are you so concerned about the LLM producing the exact same code across different sessions? Seems like a really weird thing to focus on. Why aren't you focused on things like security, maintainability, UI/UX, performance?
Agreed. It's not like humans can produce the same output given the same input for anything more than trivial inputs.
I'd argue that it's actually a benefit; I like that I can do several generations and compare them and pick the best result. HP, for example, used to do this with software teams, and that's how we got Rocky Mountain BASIC (AIUI the competing team was East Coast BASIC).
What exactly did the request for information say from DHS? What exactly was the reason for them to look for you specifically (certainly there are many others protesting)? Following up on that, how do others avoid something like this? What red flags should be avoided and how?
There may or may not be a solid answer for any of this. But this article feels like it's made for awareness, when it could also be made for action, with the right details included.
So it's the age of AI. And this seems like a great new benchmark! Lots of text, structured but each item a separate "task". Each thing requiring its own new image + textual representation.
I copy + pasted the whole article (minus the few included images) and added this prompt in Gemini 3 Pro:
> Take each of the following and add an image representing the act being described. The image should be very basic. Think of signs in buildings - exit signs, bathroom door signs, no smoking signs, etc. That style of simplicity. Just simple, flat, elegant vector graphic lines for the chopsticks, hands, bowls, etc.
I think this is pretty dang good for a one-shot run. I also ran this through Claude Opus 4.6 Extended (doesn't generate images directly, so it made an HTML page and some vector icons). Not as good as Gemini IMO. See here if curious: https://claude.ai/public/artifacts/8b6589b3-4da4-4fd5-b862-c...
Anyone able to do this better with a different prompt or model (or both)?
You can buy them from various manufacturers that make them; you often get unsolicited mail from them as your name and address is on the patent filings.
Using Claude for code you use yourself or at your own company internally is one thing, but when you start injecting it into widely-shared projects like this (or, the linux kernel, or Debian, etc) there will always be a lingering feeling of the project being tainted.
Just my opinion, probably not a popular one. But I will be avoiding an upgrade to Node.js after 24.14 for a while if this is becoming an acceptable precedent.
I still think everyone is trying to run away from the copyright problems with AI, and suspect it's going to come back to bite them. Eventually. (No I'm not willing to bet on exactly when because I'm sure it'll be a lot longer than I'd like).
This page is a good example of when Typescript is both unnecessary and off-putting, based on the content's purpose.
The author is trying to demonstrate a problem and the proposed solution, using example code that the reader then needs to read, formulate a visual "structure" in their mind, and then apply that structure to the subsequent code.
Typescript is not in any way invalid here, and as a tool when programming something that will run in the real-world, can be invaluable.
However, when writing anything you want someone to deeply comprehend, you want to use the least amount of text needed to get your point across. Adding types don't serve any purpose here. The types used are generics, which tell you nothing specific, hence the name, and the names given to the functions and object properties are enough to convey what this code is doing.
Good luck finding a phone with a headphone jack anymore though :(
I love my wired headphones though. They support BT but I've used that maybe twice. Ever. Obviously was only because I was using my phone with them, which again don't have a port for the cord.
reply