Andrej Karpathy keeps coming up with interesting new ways to use LLMs.
Former Tesla AI Director Andrej Karpathy has said that instead of asking LLMs what they think, people should ask what experts would think, and ask them to reply. “Don’t think of LLMs as entities but as simulators. For example, when exploring a topic, don’t ask: “What do you think about xyz”? There is no “you”. Next time try: “What would be a good group of people to explore xyz? What would they say?”” he wrote on X.
“The LLM can channel/simulate many perspectives but it hasn’t “thought about” xyz for a while and over time and formed its own opinions in the way we’re used to. If you force it via the use of “you”, it will give you something by adopting a personality embedding vector implied by the statistics of its finetuning data and then simulate that. It’s fine to do, but there is a lot less mystique to it than I find people naively attribute to “asking an AI”,” he explained.
Karpathy’s insight was already used by early prompt engineering practitioners, who often used to start their prompts by saying “you are an expert engineer” or “you are a star marketer”. This puts the LLM in the frame of an engineer or marketer, and can often get better results that simply asking them LLM for their own opinion. As Karpathy says, LLMs don’t necessarily have a sense of their own opinion, and whatever sense of their own opinion they have is shaped by finetuning techniques. And as such, it’s better to ask them to simulate someone who’d give a good answer than ask them for an answer themselves.