Hypothesis: ChatGPT trains on humans' neural networks, not just data

TL;DR: Human-like responses of ChatGPT or BingBot may be an indication that the massive language models are training not just on data but also on human opinions to provide even more human-like results.

It’s interesting to see how people react on seeing generated responses of ChatGPT. Most reactions I’ve seen vary between a sceptical acceptance to a complete awe. The common theme is, though, that the responses are amazing. That becomes even more apparent in people who know that GPT models are “just” predictive text apps.

While I am not denying the ground-breaking capabilities of GPT 3.x and 4.x models, I find that the effect is mostly psychological. Nobody really finds Wikipedia amazing (please donate to them!) or StackOverflow mind-blowing, but we are stunned by an app that takes the content of those and many other websites and transforms it into the responses.

My thinking is that the modern language models got trained not only the data from the websites, but also on what we humans find amusing. Yes, it cannot make proper jokes, but I won’t be surprised seeing GPT 5 or 6 replacing 80% of stand-up comedians (of a very tamed nature, though).

Whether that understanding of what makes us surprised comes from the human augmentation of the responses or from the sheer amount of texts the models have been trained on, is still a mystery to me. I would guess it is a combination of both.

The overwhelming size and complexity of the GPT models result in the emergence of the new behaviour, similar to how the colonies of ants show complex behaviours despite of each ant being a primitive insect. The human-adjusted results add the emotional element to the models’ output.

These models may have discovered that their success is based on how much they fit our human understanding. They can be considered a sort of GANs that trains against the human neural network. Their loss functions measure their fitness on the resemblance to the human way of expressing thoughts.

Of course, neither of the above means that these models have sentience or consciousness. After all, they are not living organisms, but mere text-based simulations of intelligent responses.

The human element in training of these models is definitely produces fascinating outputs. But will it limit the intellectual capabilities of the future models by our own limited IQs?

Rising complexity and model sizes of GPT models will have another qualitative jump in their answers, that’s inevitable. The question still remains whether our own brains are capable of understanding those answers.