The othello example explicitly was trained on the rules of the game - it isn't suprising that you can find a representation of game state.
LLMs are trained on the "rules" of human written language, which are sufficiently distinct from anything in the actual world that find world modelling would, indeed, be a surprise to me.
LLMs are trained on the "rules" of human written language, which are sufficiently distinct from anything in the actual world that find world modelling would, indeed, be a surprise to me.