This is incorrect. In the process of producing each token, activations are produced at each layer which are made available to future token production processes via the attention mechanism. The overall depth of computations that use this latent information without passing through output tokens is limited to the depth of the network, but there has been ample evidence that models can do limited "planning" and related capabilities purely in this latent space.
I think this ignores the fact that an agent can be meaningfully embodied in the internet, using APIs for sensors and actuators. OpenAI's training of large language models with reinforcement learning, recent retrieval augmented models and "chain of thought" reasoning are all meaningful steps in this direction in my opinion.
Microsoft (Semantic Machines) | Senior Software Engineer | Berkeley, CA or Boston, MA or Redmond, WA or fully REMOTE | Full Time
The Semantic Machines group is bringing next-generation natural language processing (NLP) technologies to products used by hundreds of millions of people worldwide. You can learn more about how Microsoft is using this technology to create entirely new kinds of user experiences here: http://semanticmachines.com/.
At the core of our platform is a new programming language, designed to support programs structured like human commands which are predicted by machine learning models. This language includes ideas from functional and logic/constraint programming, as well as novel features for meta-computation and introspection. We’re looking for engineers to work alongside product and research teams to help guide the evolution of this platform, including improvements to the core programming language, runtime, constraint system, and tooling.
The ideal candidate should be passionate about designing and evolving programming languages and/or practical formal reasoning systems, supporting users with high quality tools, and working on a rapidly evolving product-driven platform. No experience with machine learning or natural language processing is required – we’d love to work with people who are excited about the promise of these technologies and the opportunity to make them more accessible, regardless of their previous exposure to them.
At Semantic Machines [0] we rely heavily on probabilistic programming to build state-of-the-art dialogue systems. In particular, we use a library called PNP (probabilistic neural programming) on top of Dynet to allow us to express structured prediction problems in a simple and elegant form. If there are questions I am happy to elaborate to the extent I can. (Also, we are hiring! My email is jwolfe@.)
I saved the document for later use, but it got me thinking how ORMs like hibernate is dealing with the migration and future proofing issues. I wrote code libraries for a small company, in which part of the libraries design is to allow data transformation/representation of Data Models to work on the new database schema/changes while representing the data structure as if it were from previous versions. So I ended up writing generators to create a mapping of the DAO and the Models exposed to the API's in such a way that when the database changes while you still have to support the previous version of the API, I will only have to edit/tweak the mappers of the previous version of the API.
Yep, that sounds about right. For us, API communication and some global events flow through channels into the app state, where it's distributed to components. For routing we've built up a library around secretary, where we write down a map of uris, handlers, and metadata (can be a modal, etc). Info about the current page and such goes into the app state and that drives the transitions between page components.
No, you are still free to organize your application state however you wish. For example, our app dynamically pulls in content as you browse from feed to feed and use infinite scroll. It's up to you to decide what to fetch when, and what to cache/discard as the user navigates around.
reply