Yes, what's your point? That is literally what it does - it adds relevant knowle...

Tade0 · 2026-04-14T16:22:48 1776183768

My point is that this doesn't scale. You want the LLM to have knowledge embedded in its weights, not prompted in.

EagnaIonat · 2026-04-14T17:08:33 1776186513

It scales fine if done correctly.

Even with the weights the extra context allows it to move to the correct space.

Much the same as humans there are terms that are meaningless without knowing the context.

kang · 2026-04-14T20:57:58 1776200278

Would it be possible to make GPT3 from GPT2 just by prompting? It doesn't work/scale

EagnaIonat · 2026-04-15T05:21:20 1776230480

Bit of a straw-man there.