Yeah this has always been the glaring blind spot for most of the "AI Safety" community; and most of the proposals for "improving" AI safety actually make these risks far worse and far more likely.
It makes quite a lot of sense to focus on reducing the risks of every human everywhere dying, rather than the risks of already existing oppression getting worse.
No, you are deeply misunderstanding the issue. Creating a rivalrous good that powers fight over for control, then use violence to maintain control of, creating a global feudalism, is not "existing oppression getting worse". It actually makes the risks of every human everywhere dying far higher, and even if that doesn't happen, decreases global utility by a similar percentage (99%, instead of 100%). It could actually be worse, if average human utility becomes negative.
We evolved to share information through text and media, and with the advent of printing and now the internet, we often derive our feelings of consensus and sureness from the preponderance of information that used to take more effort to produce. Now we're now at a point where a disproportionately small input can produce a massively proliferated, coherent-enough output, that can give the appearance of consensus, and I'm not sure how we are going to deal with that.
It’s because that would be fairly speculative and cannot be measured. I don’t think that’s something that would make much sense in a system card. But Anthropic leadership does seem to communicate on that topic: https://www.darioamodei.com/essay/the-adolescence-of-technol...
I think you’ve forgotten about the context of OP’s post. He said he uninstalled vscode and uses a dashboard for managing his agents. How are you going to be able to do code review well when you don’t even know what’s going on in your own project? I catch subtle bugs Claude emits because I know exactly what’s happening because I’m actively working with Claude, not letting Claude do everything.
>The code is still visible if i want to review it.
I agree that the test harness is the most important part, which is only possible to create successfully if you are very familiar with exactly how your code works and how it should work. How would you reach this point using a dashboard and just reviewing PRs?
Even as a principal engineer, there is an infinite number of things you don't know.
Suppose you get out of your comfort zone to do something entirely new; AI will be much more helpful for you than it is for people who spent years developing their skills.
> The best way to answer that might be to imagine what someone from the golden age would notice if we brought him here in a time machine. [...] The first thing he'd notice, if he walked through a fancy shopping district, is that all the prominent watchmakers of the golden age seem to be doing better than ever.
Although PG isn't wrong here, people also have larger wrists nowadays that are able to comfortably support larger watches compared to ~50 years a go, never mind at the 1940s or 1950s.
> To run Llama 3.1 8B locally, you would need a GPU with a minimum of 16 GB of VRAM, such as an NVIDIA RTX 3090
In full precision, yes. But this talaas chip uses a heavily quantized version (the article calls it "3/6 bit quant", probably similar to Q4_K_M). You dont even need a GPU to run that with reasonable performance, a CPU is fine.
This is what always confused me about VC AI enthusiasm. Their moat is the capital. As AI improves, it destroys their moat. And yet, they are stoked to invest in it, the architects of their own demise.
> If AI gets so good that it can replace all human labor, will capital like money and data centers be the only moat left?
If AI gets good enough to replace all human labor then actual physical moats to keep the hungry, rioting replaced humans away will be the most important moats.
Money is useful mostly for hiring human labor to outcompete others, e.g. Satya Nadella has 100K employees under his command, you don't, so you can't realistically compete with MS today - this is their main moat.
If AI renders human labor a cheap commodity (say you can orchestrate a bunch of agents to develop + market a Windows competitor for $1000 of compute), what used to be "Satya + his army vs. you" now becomes mostly a 1:1 fair fight, which favors the startup.
I don’t know about that. I’ve looked at things like the rise of AI protects those who currently have capital. They won’t need labor or as much of it. So maybe it is what they want - to retain power permanently. Isn’t that the tech oligarch Curtis Yarvin fantasy - to replace democracy with themselves as a permanent ruling class?
The incompetent have always pantomimed the competent. It never works. Although the incompetent will always pay a huge amount to try to achieve this fantasy.
The featured blog post where one of their senior engineering PMs presented an allegedly "production grade" Matrix implementation, in which authentication was stubbed out as a TODO, says it all really. I'm glad a quarter of the internet is in such responsible hands.
Management thinks AI tools should make everyone 10x as productive, so they're all trying to run lean teams and load up the remaining engineers with all the work. This will end about as well as the great offshoring of the early 2000s.
Wait till you get AI to write unit tests and tell it the test must pass. After a few rounds it will make the test “assert(true)” when the code cant get the test to pass
No joke. In my company we "sabotaged" the AI initiative led by the CTO. We used LLMs to deliver features as requested by the CTO, but we introduced a couple of bugs here and there (intentionally). As a result, the quarter ended up with more time allocated to fix bugs and tons of customer claims. The CTO is now undoing his initiative. We all have now some time more to keep our jobs.
Thats actively malicious. I understand not going out of your way to catch the LLMs' bugs so as to show the folly of the initiative, but actively sabotaging it is legitimately dangerous behavior. Its acting in bad faith. And i say this as someone who would mostly oppose such an initiative myself
I would go so far as to say that you shouldnt be employed in the industry. Malicious actors like you will contribute to an erosion of trust thatll make everything worse
Might be but sometimes you don’t have another choice when employers are enforcing AIs which have no „feeling“ for context of all business processes involved created by human workers in the years before. Those who spent a lot of love and energy for them mostly. And who are now forced to work against an inferior but overpowered workforce.
I dont like it either but its not malicious. The LLM isnt accessing your homeserver, its accessing corporate information. Your employer can order you to be reckless with their information, thats not malicious, its not your information. You should CYA and not do anything illegal even if your asked. But using LLMs isnt illegal. This is bad faith argument
You're talking about legality again. I'm talking about ethics.
Using LLMs for software development is a safety hazard. It also has a societal risk, because it centralizes more data, more power, more money to tech oligarchs.
It's ethical to fight this. Still not commenting on legality.
You're not forced to work there and use those tools. If you don't like it, then leave the job. Intentionally breaking things is unethical especially when you're receiving a paycheck to do the opposite.
Again, no one is forcing him to be there. He's breaking something on purpose. I think you should read up on ethics because this take "I don't like it therefore whatever I do is ethical" is juvenile.
That's quite the strawman. The reason it's ethical is not that LLM's are unpopular or someone dislikes them. It's ethical because LLMs introduce safety hazards, i.e. they cause harm.
That's extremely unethical. You're being paid to do something and you deliberately broke it which not only cost your employer additional time and money, but it also cost your customers time and money. If I were you, I'd probably just quit and find another profession.
That's not "sabotaged", that's sabotaged, if you intentionally introduced the bugs. Be very careful admitting something like that publicly unless you're absolutely completely sure nobody could map your HN username to your real identity.
They are still focusing on "catastrophic risks" related to chemical and biological weapons production; or misaligned models wreaking havoc.
But they are not addressing the elephant in the room:
* Political risks, such as dictators using AI to implement opressive bureaucracy. * Socio-economic risks, such as mass unemployement.
reply