Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I agree in a "the purpose of a system is what it does" sense but I'm not sure they're inherently normalization systems.


Token selection is based off normalization, even if you train a model to produce outlier answers, even in that process you are biasing to a subset of outliers, which is inherently normalizing.


Could you elaborate on "token selection is based off normalization"?


Sure;

https://arxiv.org/pdf/1607.06450

Depending on the model architecture, there is normalization taking place in multiple different places in order to save compute and ensure (some) consistency in output. Training, by its very nature, also is a normalization function, since you are telling the model which outputs are and are not valid, shaping weights that define features.




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: