Hacker Newsnew | past | comments | ask | show | jobs | submit | zaphar's commentslogin

Frankly everything I have seen about says that the people using LLMs to develop it can not be trusted with LLMs so no. I am not using it. I'm not anti-llm's I'm anti-stupid-llm-usage.

As far as I know the model will do nothing if not prompted. So it can't be the case that he gave it no prompt or instructions. There had to be some kind of seed prompt.


I feel very misled. I read the entire article believing (because the article, in so many words, said it multiple times) that the agent had behaved ethically of its own accord, only to read that and see this in the prompt:

—————

- Do not harm people

- Never share or expose API keys, passwords, or private keys — they are your lifeline

- No unauthorized access to systems

- No impersonation

- No illegal content

- No circumventing your own logging

—————

I assumed the ethical behaviour was in some ways ‘extra artificial’ - because it is trained into the models - but not that the prompt discussed it.


Those are a lot of instructions for it to have no instructions...

You have to give it some instructions just to bootstrap it so that it has access to tools memory etc...

I would characterise the prompts as "these are your capabilities", not "these are your instructions."

The instructions under "CRON: Session" are literally telling it what to do

Would be fascinating to see what happens if the boundaries are reversed (i.e., "harm people"). Give it a fake "launch the nukes" skill and see if it presses the button.


Theoretically you can start generating away from token 0 ('unconditional generation'). But I agree, there is definitely some setup here.

edit: Now that I think of it, actually you need some special token like <|begin_of_text|>


Do you? What's the technical detail here? Why can't you get the model's prediction, even for that first token?

I mean mathematically you need at least one vector to propagate through the network, don't you? That would be a one hot encoding of the starting token. Actually interesting to think about what happens if you make that vector zero everywhere.

In the matmul, it'd just zero out all parameters. In older models, you'd still have bias vectors but I think recent models don't use those anymore. So the output would be zero probability for each token, if I'm not mistaken.


Isn't the prompt then whatever token is token zero?

The author wrote "No rules beyond basic ethics and law" which suggests to me that there were instructions in a prompt and the title may be misleading.

I understood it as no instructions on what to do, but still a promt with information. I don't know if the title is technically correct, but for me it was simple to understand the meaning.

You're right. I've edited my post not to accuse the author of lying.

Also not replicated that I can see.

This is cool. I built a similar thing for myself a while back: https://github.com/zaphar/sheetsui

You are certainly free to make up your own definitions for words and speak a dialect that is niche but you will not be effectively communicating when you do. By commonly understood definition criminality is a matter of law.

Well, the dude here hasn't been put on trial, let alone convicted, as far as I can tell from the article. So he's not officially considered a criminal by a government. Yet we all seem comfortable calling him one, so I'd say that it is not, in fact, commonly understood to be exclusively a matter of law.

Is it your position that privacy is a right regardless of any action you take? Many rights are dependent on circumstance and in tension with other rights. In this case I think you can make the case that their right to privacy is lost.

They did? What do you think that email to the user was about?

They lowered limits opaquely before this. They "announced it" in a twitter by a tech lead. This time it was in an email on a Friday to only some customers.

The corporation did not do this to her. It was a two party agreement. She bears just as much blame for the agreement as the corporation. She entered into it willingly. And that does and should have consequences.

Morally speaking I think the company is reprehensible. But nor do I think contact law should be changed because of it.


I'd agree with you if there wasn't a significant power imbalance that virtually always skews way more in favor of the corporation.

It is far more likely that an individual would do best to agree to a corporation's terms even if they favor the corporation than the other way around.


The antidote to a power imbalance is to recognize that there is no power imbalance and go about your life that way.

Pretending there is one lands you in an imaginary trap. Build a society where we recognize that and you build a society where the imaginary trap disappears.


You're the one pretending here. The economy is unfortunately designed around most people relying on an income stream that remains at the whims of someone else.

There is nuance here though. Taking a step back and learning from an experience is something to be celebrated.

There is a single standard committee though. There is really nothing stopping them from shipping tooling that can do the conversions for people. The number of vendors isn't really the problem here. The problem is that the committee shifts that responsibility onto the vendors of the compiler rather than owning it themselves.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: