The distrust is not even ideological anymore, we just want our services to keep working without some guy deciding to nuke everything the next day for literally no reason.
This post resonates with my situation very much, although I have not made even a tiny fraction of the contributions this person has. I think if a person like this is struggling to find work, but the donkeys who code my government's dysfunctional websites are all fully employed, it suggests the software industry is in very big trouble.
That being said, and speaking from my own experience, one can develop ego problems if they've been undervalued by society. You can start viewing other people as lesser than you for not understanding your situation. I managed to escape those toxic thought patterns by practising empathy as a deliberate activity, and forcing myself to give love and grace to others until it felt natural.
> I managed to escape those toxic thought patterns by practising empathy as a deliberate activity, and forcing myself to give love and grace to others until it felt natural.
> but the donkeys who code my government's dysfunctional websites are all fully employed
The juxtaposition of these two statements is amusing.
I think people often make this mistake, where they think they've killed their ego, and then fail to notice when their ego returns.
I think people generally underestimate how hard it is to permanently kill their ego. What ends up happening, then, is that a lot of people genuinely believe that they have humility when in actuality they don't.
I never said I was better than them, but I can acknowledge that some people are. Even then, only in the domain of software, not in terms of human value.
> I think if a person like this is struggling to find work, but the donkeys who code my government's dysfunctional websites are all fully employed, it suggests the software industry is in very big trouble.
Based solely on this blog post, I would question if this person would be capable of holding a regular job, and it has nothing to do with skill. It also kind of reads like they already know this about themself.
> Based solely on this blog post, I would question if this person would be capable of holding a regular job
The blog post states quite clearly that this person has had regular employment in the past, which suggests they may be just as capable of it in the future. There are some bad optics in the post, but they have more to do with wanting to buy a home in SF and fly private aviation than whether OP can hold down a job.
People have regularly done things in the past that they can no longer do in the present. I don’t know what they were like in the past but a blog post like this would concern me if I was hiring.
> I think if a person like this is struggling to find work, but the donkeys who code my government's dysfunctional websites are all fully employed...
It's effective to apply a filter of eccentric vs predictable on these sort of matters. The issue isn't so much whether someone is capable or not, but whether they fit into one of the standard boxes the bureaucracy has prepared. And to make it even worse, managers don't have the skill or inclination to try and make the best of the specific person who they employ. They generally have an approach that they've worked out based on the average of people who they have managed. If they get an employee who falls too far outside that range they may literally not know what to do. Usually at that point they will decide that they are dealing with a deviant and the problem must inherently be the deviance.
It's an unfortunate situation. Particularly if you're the sort of person who cares about getting to the best outcome and you realise how many geniuses must get shut down when the best way isn't the usual one. In practice incompetent but routine gets a lot more tolerance than competent but eccentric. Someone needs to be very competent indeed to overcome eccentricity.
This isn't an ego problem (though I think she may additionally have an ego problem). The author has espoused techno-fascist views (something she avoids actually mentioning in this post), and that seems to be the real reason why many people are uncomfortable working with her.
> In 2012, Tunney started working for Google as a software engineer.[4] In March 2014, Tunney petitioned the US government on We the People to hold a referendum asking for support to retire all government employees with full pensions, transfer administrative authority to the technology industry, and appoint the executive chairman of Google Eric Schmidt as CEO of America.[5][6] Tunney has been inspired by the political views of Curtis Yarvin.[7]
This person seems to be extremely online and Wikipedia has a petty high bar for inclusion of things like that. I wouldn't be surprised of their social media was littered with even more extreme things (like support of slavery, which the talk page mentions).
I tried pretty hard, but even the Wikipedia refs are just people (wildly) extrapolating from the petition. I'm not saying these aren't her beliefs, I'm just saying we don't have enough to know, definitely not enough to cancel her, which this thread feels like it's about.
I think to get what you asked you're going to have to directly read their social media and other writings for the last N years and form an impression, keeping in mind self-censorship doesn't mean a change in core beliefs. Following wikipedia refs isn't going to cut it. We're not talking about a public figure here, but an extremely online regular person/minor internet celebrity.
No, if people want to walk around calling someone some kind of fascist they need receipts. Anything else is repugnant Internet brigading that has no place on HN.
> I think if a person like this is struggling to find work, but the donkeys who code my government's dysfunctional websites are all fully employed, it suggests the software industry is in very big trouble.
I don’t think so. I think what happens is that people believe in meritocracy or karma or universal justice. Generally, one cannot rely on that. You make one mistake and then another and you are out, no matter how much goodness you have done. It’s a hard pill to swallow. It’s absurd, but we must keep pushing the rock and be happy (or else the alternative is s…)
> This post resonates with my situation very much,
How does it resonate with your situation? What do you have in common?
Being rejected for being racist and neo-fascist? Being frustrated by not being able to afford private aviation? Wanting to raise donations to pay for a house and travel? Lots of comments here substantiating that, BTW.
> I didn’t get that they’re struggling to find work from the article. Did I miss something?
They're just struggling to buy "a home in San Francisco" and "travel around the world and experience the cosmopolitan lifestyle my project is named after, using only private aviation."
Also this person apparently went from being part of Occupy Wall Street to being "inspired by the political views of Curtis Yarvin" (https://en.wikipedia.org/wiki/Justine_Tunney).
> Also this person apparently went from being part of Occupy Wall Street to being "inspired by the political views of Curtis Yarvin"
A lot of the far right are anti-globalisation as Occupy Wall Street was, both are a rejection of consensus politics, and people who tend to go to extremes are more likely to go to a different extreme than become moderate.
There is also a type of brilliant person (and Justine is brilliant) who tends to go off the rails in some way. Look at the number of weird or nasty creative artists and authors.
> "travel around the world and experience the cosmopolitan lifestyle my project is named after, using only private aviation."
I did not read that far and thought the post arrogant and entitled! That is a Musk like level of being out of touch with reality. Again, something people who indulge a sense of their own brilliance are prone to.
I suspect that on some level, many of the people slating Justine feel threatened by his genius, and that is part of the reason why they keep bringing up the same tired old laundry list of perceived sins in an attempt to do so.
Did you make this account just to misgender her a couple times? You haven't posted before or since but are very careful to use the masculine pronoun to describe someone that uses the feminine pronoun in the only posts you've made.
FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid.
I don't think you can use lindy on trends as if trends are static objects, but that's another conversation.
So, this is not quite right: Alexander contributed to the report, but his personal opinion is more like the mid-2030s[1]. Freddie feels like this is him backing down from the original statement, but in fact he said this at the time the report was published, and in fact pointed out a graf below the quote that Freddie claims does tie him to 2027:
> Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would be annoying to change). Other members of the team (including me) have medians later in the 2020s or early 2030s, and also think automation will progress more slowly. So maybe think of this as a vision of what an 80th percentile fast scenario looks like - not our precise median, but also not something we feel safe ruling out. [2]
I don't think this changes your observation that he is "personally invested" (i.e. believes this trendline will continue), but I'm pretty sure when AGI doesn't appear in 2027, many people will believe that this invalidates the arguments being made here (or in the report). The actual report was intended to give a feel for what a near-future "disaster" AGI scenario, and settled on a date to give that some concrete immediacy. The collective review that gave that as a possible, but not inevitable date is still ongoing (they originally pushed their best estimate out a bit further, but now they think, judging by the goals that are being hit, their scenario was a little too conservative). [3]
LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text.
Apparently that is enough to fool many people but it’s nothing close to AGI which would require internal models of the world, reasoning etc.
We are nowhere close to AGI and the fools who predicted we were will unfortunately keep lying about their stated timelines when it inevitably doesn’t arrive. You’re already hedging and trying to caveat previous predictions, as OpenAI did with their AGI predictions which they’re now furiously back-pedalling on.
This is all speculative. We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation", especially once you add feedbacks loops.
> "We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation""
Do you mean "token" as in the LLM sense?
Or are you thinking that thoughts in the human brain are also constructed out of some sort of underlying "token" even though the abstract thought happens and is held before any words are used to try to communicate that thought to an external party?
LLMs also don't run on tokens internally, they're just the inputs and outputs. The reasoning models do operate (partially) in the token space, but then so do I.
> the thought happens and then the words are generated to reasonably describe that thought.
Thoughts don't happen in a vacuum, they are triggered by external or internal stimuli, and these stimuli/thought precursors could very easily be tokens (dense info packets), which then map to latent space vectors, which very well could be thoughts.
Claims like "humans don't operate the same way" has no basis. Not only do we literally not know how humans operate mechanistically, and so we literally don't know the logical structure of human thought, but any system that is Turing complete is so easy to create that many wildly different mechanistic systems are fundamentally equivalent/interconvertible.
> Thoughts don't happen in a vacuum, they are triggered by external or internal stimuli, and these stimuli/thought precursors could very easily be tokens (dense info packets), which then map to latent space vectors, which very well could be thoughts.
Yes, possible, that's why I asked you above if that's what you meant by "token". Someone else responded and I didn't notice it wasn't you.
> Claims like "humans don't operate the same way" has no basis. Not only do we literally not know how humans operate mechanistically, and so we literally don't know the logical structure of human thought, but any system that is Turing complete is so easy to create that many wildly different mechanistic systems are fundamentally equivalent/interconvertible.
I think this position is too extreme, we do have some information.
We know how LLM's work when generating a sequence of words and I know that my brain does not work the same way for word generation because I am fully aware of the complete thought in advance of any words getting generated by me externally or internally.
I know prior to generating words that my thought is X and the words I'm about to produce need to express that thought.
But with LLM's we know that the essence of what they produce is not known in advance, that it must complete the word generation process to fully realize the end result and that multiple different end results are possible.
What I'm saying is that this is incorrect. An "idea" exists within a model before it generates tokens. This property does not distinguish humans from LLMs.
Additionally "from learned stats" doesn't disambiguate between a wider variety of things. I'm not aware of any other way to acquire knowledge from measurements. I'd bet that humans do this differently, based on the fact the humans can get further with less training data and that they learn actively during operation, but not so differently that 'learning stats' would be an inaccurate description.
> What I'm saying is that this is incorrect. An "idea" exists within a model before it generates tokens.
If that were the case, then the systems would generate words based on the fully resolved idea, but that is not how the LLM systems currently work (per vendors descriptions).
They choose words sequentially and both the specifics of the input as well as the chosen output words significantly impacts not just the rest of the output but the very correctness of the output.
> but not so differently that 'learning stats' would be an inaccurate description.
Agreed, humans are generalizing using some mechanism that can be modeled with math.
But the execution of our reasoning and thought processes is not obviously similar to LLM's next word generation based on probabilities.
>that is not how the LLM systems currently work (per vendors descriptions)
Anthropic says of the their model[0]:
"""Claude sometimes thinks in a conceptual space that is shared between languages, suggesting it has a kind of universal “language of thought.”
{...}
Claude will plan what it will say many words ahead, and write to get to that destination. We show this in the realm of poetry, where it thinks of possible rhyming words in advance and writes the next line to get there. This is powerful evidence that even though models are trained to output one word at a time, they may think on much longer horizons to do so."""
Anthropic also created 'golden gate claude'[1] by identifying the region of its architecture that corresponded to the concept of the golden gate bridge and activating it. What would such a region exist for if claude could only think one token at a time?
>the execution of our reasoning and thought processes is not obviously similar to LLM's
"Not obviously similar" I can agree with. I don't think you've identified a way in which they are obviously different, though.
We understand it enough to see the obvious massive deficiencies in LLMs.
They can predict likely sentences but not evaluate truth or logic. They can fairly reliably record facts about the world but not construct internal models of the world.
> They can predict likely sentences but not evaluate truth or logic.
They do probabilistically. So do humans as a matter of fact. The best of us are better at it than LLMs, but that's not persuasive evidence of anything meaningful really.
> They can fairly reliably record facts about the world but not construct internal models of the world.
You don't know that, unless your presuppose a very specific definition of world model that necessarily precludes emergent ones.
Humans do not reason by guessing the next most likely token/word. They use logic, morality and systems of thought they have constructed and shared to help them reason and don’t in any way predict tokens in a sequence - we use words to represent our thoughts and feelings about the world, not to construct them.
You’re constructing a post-hoc fantasy of human thought based on how LLMs work because you are desperate for some reason to believe that they are thinking like humans, but they are not. The process is very different and the results are also different.
> LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text.
Argument?
Are LLMs close to being able to significantly help AGI researchers?
Mind you, he is only personally invested insofar as he's staked his reputation on it. Throughout his writing, he expresses the same point over and over again: desperately wants AI to slow down, advocates for politics that would slow it down, and most likely nothing would bring him greater peace than to see a sigmoid curve appear.
This is incorrect as written. The author contributed writing to AI-2027 but distanced himself from the underlying model. That model had 2027 as the modal year of AGI, not median or mean. The authors of that model revised it to a later date shortly after and (if I recall correctly) have since done so again.
It is broadly true that Scott believes that AGI will come in the near future and from LLMs, although his reputation runs a ways deeper than that.
If I'm not mistaken, he's either affiliated with or otherwise connected to the effective altruist movement, hence he can't be unbiased. I find this article tells an interesting perspective on it: https://www.noemamag.com/the-politics-of-superintelligence/
The whole rationalist movement is super bizarre to the "normies".
It's not at all surprising that they are increasingly getting labeled a cult (they aren't by traditional definition but there are a lot similarities). I'm really surprised it hasn't hit the mainstream yet given the connections to Elon, Thiel, frontier labs, dark crypto funding, FTX/SBF, some suicides and some murders. It's all a little nuts.
Meanwhile you got all the anti-democratic NRx people on the other side of it.
I suspect this new doc coming out on HBO will spark a media frenzy.
He only has 1.5 more months.
If he's wrong he needs to own it. Same for Eliezer Yudkowsky. But these people have too much riding on their brands. No one has the courage to fess up to being wrong. Given how many podcasts he and others have been on professing this belief, it will be hard to just pretend otherwise.
Yudkowsky has never predicted that "AGI" will be here in 1-2 years. He has been saying frequently for years that it is easier to predict how the AI juggernaut will turn out (i.e., very badly for us) than to predict when the very bad things will happen.
(I don't know about the other guy mentioned above.)
Eliezer Yudkowsky, now there’s a name I haven’t heard in a while.
What has he been up to since finishing the finest work of literature ever produced, Harry Potter and the Methods of Rationality? I’ve been patiently awaiting a sequel!
His new book has billboards along the highway to the Bay Bridge. Surprised you haven’t heard of him recently. His fame has skyrocketed. If anyone builds it, everyone dies.
Ok, but you can just look at the METR curve. Mythos saturated the 50% time horizon. The 80% is now at 3 hours. The rate of progress is accelerating not slowing down. There’s no indication yet that this is a sigmoid!
The METR task set contains no tasks with a duration greater than 32 hours (conservatively eyeballed from Figure 3: https://arxiv.org/abs/2503.17354 ), so any prediction that naively forecasts a longer time horizon is trivially incorrect. I guess that won't lead to a sigmoid-looking graph though, since METR will likely switch to a different evaluation methodology at that point and stop updating the old curve.
AGI has become such a meaningless nondescript term, arguing when or how it is here has become pointless. Even OpenAI caved in and removed their AGI clause from their contract with Microsoft because they weren't fully sure that we are not there yet. The original ARC AGI was hailed as proof that AGI is not here yet, but now that ARC 1 and 2 got saturated, noone wanted to consider that perhaps we crossed the point where average humans are getting left behind. Frontier models are primarily limited by context and modality at this point, not by intelligence.
To your point, if we had truly unlimited context to the point where at least that instance of a model could “learn” and have what seems like a continuous “consciousness” I think many of us would think that we’ve attained AGI.
Right now we have an incredibly smart thing with severe short term memory loss, and it’s hard for us to reconcile that as it’s so different from us.
Quite a few people were already led to believe that these models are conscious when we had a fraction of current context lengths. Right now the biggest problem is that the "session" info in form of the current conversation gets lost too quickly, but that has become largely an implementation detail. You could fit an entire life's story into modern context windows. With some clever context management, you could probably build something that feels like what you describe. If we truly had this sort of short-term to long-term memory (i.e. from prompt context to weights) system on a technical foundation, we'd probably be closer to runaway superintelligence than mere AGI that could beat most humans on most tasks.
> FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid.
He co-authored a report, which is something more than an opinion. It may be used to inspire policy. There should be greater reputational consequences for publishing something you spent a few months studying and writing about along with several experts. Just my opinion.
I don't understand what you're trying to imply here. Yes, he co-authored a report. What is supposed to be dangerous or suspicious about this? What does your statement about "reputational consequences" have to do with your original comment, which implies that this some indicates a bias on his part?
It seems to me like you're trying to somehow imply that writing things to convince people of what you believe is somehow nefarious? It isn't! It's what we're all doing here right now! Putting it in a format that certain people will take more seriously doesn't make it nefarious either. I am quite confused by your point of view here.
There was no implication of anything you're suggesting. It's a question of correctness (bias vs facts, predicting the sun will rise vs predicting the end of the world), whether you think it's important to be correct as a matter of reputation, and how correctness should be weighed if it is indeed important to one's reputation (a once-off comment vs a full report).
He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy. He's taken a side. if he's wrong I would hope he owns up to it
Yes, that's called "having an opinion". Typically people writing argumentative pieces are doing so because they have a belief about the matter. I'm not sure what exactly you expect here.
> if he's wrong I would hope he owns up to it
I think Scott Alexander is pretty good about that.
> He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy
I mean.. this is 2026 right? You're not writing that comment from 2024 or something?
We see massive problems already where photos are just not believable anymore, nor is audio, and not even video actually with many people falling for AI fake image clips from the Gaza war for example. And since then these tools are MASSIVELY more powerful. Disinformation is essentially free, and the cost of truth has been static. Meaning the "buying power" of truth has collapsed and is falling faster and faster.
Anyone who dismissed AI risks a few years ago IS ALREADY PROVEN WRONG.
I use AI to look things up and I try to learn. That part is speed up, but once I know how X works I’m faster doing it myself. My assumption is that most people seeing things differently, compare their performance of not knowing how X works with Claude, but not with someone who’s really good at X. Which makes a lot of sense given LLMS are prediction generators. My take is that the best use of AI is to get you to the point where you are really good with X and then naturally your AI usage will go down.
what my experience says is that, when you get "really good" with X, then you can easily write a prompt that says exactly how it needs to be done and you'll be able to do it much faster than writing it all yourself because you know the important parts and the rest is just glue.
I'm having a hard time understanding what you mean here. If something is obscured, by definition it is less visible. Being 'less typical' is a form of security because most attacks rely on some form of pattern recognition, and obscurity literally dissolves patterns into noise.
>If something is obscured, by definition it is less visible.
Obscurity is not the same thing as something being "obscured".
Obscurity means something is either difficult to comprehend, not well known or uncommon.
Obscured means something is hidden or concealed. When something is hidden, that means the thing is still there and there is a way to get to it. You can build automated tools around finding it.
>Being 'less typical' is a form of security because most attacks rely on some form of pattern recognition, and obscurity literally dissolves patterns into noise.
This is making the leap of faith assumption that "obscurity" is equivalent to "impossible to understand". In security you have no control over the attacker and therefore have to assume your attacker has more than enough knowledge and intelligence to perform the attack.
Since computer systems are static and unchanging without frequent patching, you can't assume that there is a cat and mouse game where the mouse is adapting its hiding strategies dynamically and managing to escape every single time.
Depends, some systems are dynamic. There is also a gray area where obscurity can be computationally infeasible to attack, but not bound by traditional polynomial assumptions in cryptography.
As is always the case in these semantic discussions, the answer depends on your initial axioms and assumptions, which does kind of make most of these discussions pointless (but I did learn a lot from this one).
You're overly focusing on the term and not the meaning. The term comes about from people choosing tools like "foxit" or "Opera" and saying that those products are safer than their cohorts Adobe/ Firefox because they are attacked less often.
This notion was termed "security through obscurity" ie: "you use the less popular option, therefor that option is safer". It has nothing to do with "obscuring" in the sense of "hiding", that's a linguistic quirk of a colloquial term. If you were actually taking action to reduce the ability to understand a system in a way that you could meaningfully defend, it would no longer be "security through obscurity".
The argument has persisted because there are two different questions that sound the same (X is less typical than Y):
1. Is "X" safer than "Y"?
2. Is a user of "X" safer than a user of "Y"?
When looking at (1) in isolation, you can say things like "X lacks security features, therefor Y is safer" and "X is less often used, therefor X is safer", etc. This is a question about the posture of the project itself, in isolation.
(2) is about the context for users. The reality is that X, which perhaps is fundamentally less well built software, may actually have users who are attacked far less frequently.
Both are likely to favor "rarity is a poor indicator of safety" as we generally reject mitigation approaches that rely on attackers to behave specific ways, but what's important is that these are completely different questions and neither has to do with being obscured but rather rare.
None of this is about what is "obscured" or not. If something is obscured or obfuscated, that is a technique that can be evaluated separately by its own merits (ie: how hard is deobfuscation, how easy is it to adapt to deobfuscation, etc). All of this is about whether you're evaluating (1) or (2) - and in the case of (1), which is what the criticism always has focused on, the answer is that "rarity" is not a mitigation.
> The term comes about from people choosing tools like "foxit" or "Opera" and saying that those products are safer than their cohorts Adobe/ Firefox because they are attacked less often.
Visiblility is also a mental construct of what we expect to see and what we know already and can map to what we see. "Obscure" is doing a lot of work here. It doesn't necessarily mean hidden, it can mean the object's true purpose or form is hidden from some particular vantage, and only that vantage.
England has a long history producing artwork against some institution, only for that institution to get worse over time. George Orwell wrote about the dangers of authoritarianism and surveillance, and since then the UK government has only ratcheted up their surveillance and authority. They also made a movie called This is England which straightforwardly depicts young English nationalists ruining their lives with nationalism, and 20 years later there are more nationalists in England than at any point after WW2.
England has gotten more liberal over time, not less. I'm not following your logic here. It seems you're wanting to criticize the government of the UK for being authoritarian and ratcheting up the surveillance state, but simultaneously criticize nationalists and link them to this government, but nationalists and right-leaning groups haven't really been in charge of the UK.
No, I didn't. But I wouldn't claim that a referendum that was voted on by the people of the country to be the same thing as right-leaning groups being in control of the government of the UK.
No it doesn't. If the right was in charge of the government of the UK they wouldn't have needed to have a referendum or drum up support for it.
Here's perhaps a concrete example to help piece this together. I live in Ohio. Our state government is right-leaning, and controlled by the Republican Party. The Republican Party has an anti-abortion platform.
A couple of years ago, citizens got together, created, and then passed an amendment to the Ohio Constitution providing abortion access as a legal right.
The right is still in control of the government, and that is true regardless of who paid to support the referendum, or how it was voted.
This is such an odd comment. People in arts and letters warning about some element of society or culture and then that element growing in strength is something that can be found in most countries, and doesn't seem more prevalent in England than elsewhere.
It's not an environmental issue, data centers are overleveraged in the US due to a belief that they need to win the "AI race". The government is putting their hand into the market to try and shift this balance, when they should be creating basic infrastructure and services.
The silent ones have also gone insane though. You don't get to see it as much, but they're the people promoting bad policies and funding delusional projects behind the scenes. Every so often one of them will make a public statement or political contribution which reveals that they have also been spending way too much time online.
If it is that easy for foreign governments to influence the very thoughts people have day-to-day, then something is extremely broken in your system and nearly all the blame is on your government for allowing that to happen.
reply