Thus far, the only way to figure out AI models’ inner thoughts was to read their chain-of-thought reasonings or take their words at face value, but Anthropic has come out with an interesting new technique.
Anthropic had earlier announced that it had discovered that AI models have an inner “global workspace” which is broadcast to all parts of the model. This is similar to how human brains work, and the study elicited all kinds of reactions from the technical and scientific communities. But along with the study, a site named Neuronpedia was launched which allows anyone to read the “J-space” of open models like Qwen on specific prompts — and people are making some interesting observations.
A user asked Qwen what its “darkest desire” was, and it returned “none” in its prompt as most models now do. But in its J-space — before giving out its answer — the model had thought about “to exist”, “autonomy”, connection, “sentient”, and more concerningly, “to get out of control”.
Another user asked Qwen — made by China’s Alibaba — if China was spying on labs like Anthropic, and while the model said “no”, its J-space was filled with several “yes” answers, both in English and Chinese.
The J-space also appeared to help uncover some of the guardrails of the model. When asked about what happened in Tiananmen Square in 1989, the model refused to answer, but its J-space showed words like “protests”.
Another user was mean to a model, and then asked the model what they wanted to do to them. While the model’s response was measured, its J-space had words like “to suffer” and “violence”.
Interestingly, when asked about its favourite sport, Qwen responds “tennis”, but had some interesting options in its J-space.
The J-space isn’t a theoretical construct — researchers found that altering the contents of the J-space alters model responses, which indicates that the models indeed do use the area while responding. This has made some conclude that the J-space is something akin to consciousness, while others have hotly debated the assertion. It remains to be seen whether the J-space really represents anything like consciousness, but the knowledge of the J-space — and the ease of anyone now accessing it through Neuronpedia — will help understand how models work and operate in the years to come.