Weird, when I spent lots of time with Philosopher AI all I've seen is semantics in action. How else do you expand on a topic based on a simple question and keep the context? How can it then compress text to summarize it well? How can it obtain high level answers for complex questions?
To answer these questions one would need to begin with rigorous definitions of 'semantics', 'context', 'memory', 'complex' etc. What we know is that attention models are heavily trained associative memories. Anecdotal perceptions are deceiving, as humans we are often prone to see human/animal-like behavior where it doesnt exist (e.g. cgi/games).
https://en.wikipedia.org/wiki/Transformer_(machine_learning_...
Given a large enough training dataset it shows that intelligent behavior arises, which is pretty remarkable. It can also be scaled further.
Read the original paper on the architecture: https://arxiv.org/abs/1706.03762