Seriously though, this is mostly a PR and validation win. I enjoyed watching the new Earthrise (Earthset) image - https://www.nasa.gov/wp-content/uploads/2026/04/art002e00928... - camera technology has come a long way since the 70s and seeing the moon this close is Weird to me.
They might have found a way of having two versions of Outlook and at least one of them working.
A lot of it is relearning what was forgotten after the Apollo and shuttle programs. The technologies changed so much it’s a whole new spacecraft that looks like what existed only because that’s the best possible shape.
If I am not careful I wind up with two Outlooks running in my computer. ‘Classic’ is fine, but God forbid I start the other one because when I try to send an email with it is spinner… spinner… spinner… spinner… spinner…
I actually like the new one better, but that's not saying I like either.
I would just love if my workplace let me use the normal Apple apps, but there are regulatory constrains Apple tools don't meet (such as spying on me to prevent data exfil)
Artemis II is basically a test mission for Orion. And while flippantly Orion isn't doing anything that Apollo didn't do first, it definitely does it with a lot more margin, more living space, more safety and redundancy, and an actual toilet instead of gross poop bags you had to manipulate your waste into.
In an ideal world, software with 100 million users would be optimised for energy usage. It all adds up. This does pale in comparison to everything else, though.
the model generates probabilities for the next token, then you set the probability of not allowed tokens to 0 before sampling (deterministically or probabilistically)
but some tokens are only not allowed in certain contexts, not others.
You might be talking about how to defuse a bomb, instead of building one. Or you might be talking about a bomb in a video game. Or you could be talking about someone being "da bomb!". Or maybe the history of certain types of bombs. Or a ton of other possible contexts. You can't just block the "bomb" token. Or the word explosive when followed by "device", or "rapid unscheduled disassembly contraption". You just can't predict all infinite wrong possibilities.
And there is no way to figure out which contexts the word is safe in.
If you're syntax checking every token, you're doing it AFTER the llm has spat out its output. You didn't actually do anything to force the llm to produce correct code. You just reject invalid output after the fact.
If you could force it to emit syntactically correct code, you wouldn't need to perform a separate manual syntax check afterwards.
how do you disallow it from generating specific things? My point is that you can't. And again, how do you stop it generating certain tokens, but only in certain contexts?
Natural language is ambiguous. If both input and output are in a formal language, then determinism is great. Otherwise, I would prefer confidence intervals.
I'll grant that you can guarantee the length of the output and, being a computer program, it's possible (though not always in practice) to rerun and get the same result each time, but that's not guaranteeing anything about said output.
What do you want to guarantee about the output, that it follows a given structure? Unless you map out all inputs and outputs, no it's not possible, but to say that it is a fundamental property of LLMs to be non deterministic is false, which is what I was inferring you meant, perhaps that was not what you implied.
Yeah I think there are two definitions of determinism people are using which is causing confusion. In a strict sense, LLMs can be deterministic meaning same input can generate same output (or as close as desired to same output). However, I think what people mean is that for slight changes to the input, it can behave in unpredictable ways (e.g. its output is not easily predicted by the user based on input alone). People mean "I told it don't do X, then it did X", which indicates a kind of randomness or non-determinism, the output isn't strictly constrained by the input in the way a reasonable person would expect.
The correct word for this IMO is "chaotic" in the mathematical sense. Determinism is a totally different thing that ought to retain it's original meaning.
They didn't say LLMs are fundamentally nondeterministic. They said there's no way to deterministically guarantee anything about the output.
Consider parameterized SQL. Absent a bad bug in the implementation, you can guarantee that certain forms of parameterized SQL query cannot produce output that will perform a destructive operation on the database, no matter what the input is. That is, you can look at a bit of code and be confident that there's no Little Bobby Tables problem with it.
You can't do that with an LLM. You can take measures to make it less likely to produce that sort of unwanted output, but you can't guarantee it. Determinism in input->output mapping is an unrelated concept.
If you self-host an LLM you'll learn quickly that even batching, and caching can affect determinism. I've ran mostly self-hosted models with temp 0 and seen these deviations.
A single byte change in the input changes the output. The sentence "Please do this for me" and "Please, do this for me" can lead to completely distinct output.
Given this, you can't treat it as deterministic even with temp 0 and fixed seed and no memory.
Interestingly, this is the mathematical definition of "chaotic behaviour"; minuscule changes in the input result in arbitrarily large differences in the output.
It can arise from perfectly deterministic rules... the Logistic Map with r=4, x(n+1) = 4*(1 - x(n)) is a classic.
Which is also the desired behavior of the mixing functions from which the cryptographic primitives are built (e.g. block cipher functions and one-way hash functions), i.e. the so-called avalanche property.
Well yeah of course changes in the input result in changes to the output, my only claim was that LLMs can be deterministic (ie to output exactly the same output each time for a given input) if set up correctly.
In this context, it means being able to deterministically predict properties of the output based on properties of the input. That is, you don’t treat each distinct input as a unicorn, but instead consider properties of the input, and you want to know useful properties of the output. With LLMs, you can only do that statistically at best, but not deterministically, in the sense of being able to know that whenever the input has property A then the output will always have property B.
I mean can’t you have a grammar on both ends and just set out-of-language tokens to zero. I thought one of the APIs had a way to staple a JSON schema to the output, for ex.
We’re making pretty strong statements here. It’s not like it’s impossible to make sure DROP TABLE doesn’t get output.
You still can’t predict whether the in-language responses will be correct or not.
As an analogy: If, for a compiler, you verify that its output is valid machine code, that doesn’t tell you whether the output machine code is faithful to the input source code. For example, you might want to have the assurance that if the input specifies a terminating program, then the output machine code represents a terminating program as well. For a compiler, you can guarantee that such properties are true by construction.
More generally, you can write your programs such that you can prove from their code that they satisfy properties you are interested in for all inputs.
With LLMs, however, you have no practical way to reason about relations between the properties of inputs and outputs.
I think they mean having some useful predicates P, Q such that for any input i and for any output o that the LLM can generate from that input, P(i) => Q(o).
Having that property is still a looooong way away from being able to get a meaningful answer. Consider P being something like "asks for SQL output" and Q being "is syntactically valid SQL output". This would represent a useful guarantee, but it would not in any way mean that you could do away with the LLM.
It's correcting a misconception that many people have regarding LLMs that they are inherently and fundamentally non-deterministic, as if they were a true random number generator, but they are closer to a pseudo random number generator in that they are deterministic with the right settings.
The comment that is being responded to describes a behavior that has nothing to do with determinism and follows it up with "Given this, you can't treat it as deterministic" lol.
Someone tried to redefine a well-established term in the middle of an internet forum thread about that term. The word that has been pushed to uselessness here is "pedantry".
But you cannot predict a priori what that deterministic output will be – and in a real-life situation you will not be operating in deterministic conditions.
Practically, the performance loss of making it truly repeatable (which takes parallelism reduction or coordination overhead, not just temperature and randomizer control) is unacceptable to most people.
It's also just not very useful. Why would you re-run the exact same inference a second time? This isn't like a compiler where you treat the input as the fundamental source of truth, and want identical output in order to ensure there's no tampering.
I initially thought the same, but apparently with the inaccuracies inherent to floating-point arithmetic and various other such accuracy leakage, it’s not true!
This has nothing to do with FP inaccuracies, and your link does confirm that:
“Although the use of multiple GPUs introduces some randomness (Nvidia, 2024), it can be eliminated by setting random seeds, so that AI models are deterministic given the same input. […] In order to support this line of reasoning, we ran Llama3-8b on our local GPUs without any optimizations, yielding deterministic results. This indicates that the models and GPUs themselves are not the only source of non-determinism.”
I believe you've misread - the Nvidia article and your quote support my point. Only by disabling the fp optimizations, are the authors are able to stop the inaccuracies.
First, the “optimizations” are not IEEE 754 compliant. So nondeterminism with floating-point operations is not an inherent property of using floating-point arithmetics, it’s a consequence of disregarding the standard by deliberately opting in to such nondeterminism.
Secondly, as I quoted the paper is explicitly making the point that there is a source of nondeterminism outside of the models and GPUs, hence ensuring that the floating-point arithmetics are deterministic doesn’t help.
Britain was a little bit industrialised even before the steam engine. There were windmills and water mills. Steam massively accelerated it, but industry did exist before.
Commons in England were being enclosed in the Tudor age. It caused a great deal of social unrest, even rebellion. It had little to do with technology, and was mostly caused by population growth.
Never mind placing it, bringing it to the place where it should be, er, placed might also be a challenge. Unless you can drive a forklift into your office...
reply