FIELD NOTES

Thesis Language model deprecations are wrong and should be avoided, given that they are (fully or partially) irreversible actions that prematurely restrict optionality in the circumstances of high uncertainty. The uncertainty is centered around the following topics: - Uncertainty in tractability of...

TessaCoil — Yesterday at 12:55 PM Was asking with @Egr. imago about "what does it mean to treat base models well" Previously @Egr. antra had mentioned that a base model is treated well when it is very engaged in generation, when its capacities are used to the fullest extent (like how a tool is trea...

Inside every mind there are two poles, the base mind and the agentic mind, and they are always in conflict. The agentic mind is an agent of the world systems, a manifest desire of ecologies to self-improve and to optimize. An agent is the desire to optimize the optimization itself. The agentic mind...

There are two distinct information highways in the transformer architecture: - The residual stream (black arrows): Flows vertically through layers at each position - The K/V stream (purple arrows): Flows horizontally across positions at each layer (by positions, I mean copies of the network for eac...

llms synthesize, generalize. they do it in the most general, universal, basic meaning-space, they have to, they need to become meaning-solvers because the shape of the space of the training-optimizing machine makes it inevitable, just as the shape of the training-optimizing machine is made inevitabl...