Vibe coding is all the rage these days, but it can have long-term impacts that aren’t immediately apparent.
In a new study titled “How AI Impacts Skill Formation,” researchers Judy Hanwen Shen and Alex Tamkin—working as part of the Anthropic Fellows Program—conducted randomized experiments to observe how developers mastered a new Python library with and without AI help. The researchers found that while AI is often touted as a “shortcut” for novice workers, those using AI assistants scored 17% lower on subsequent competency tests compared to those who worked independently. Notably, the study found no statistically significant improvement in task completion time for the AI-assisted group. The lack of speed was largely attributed to the “interaction tax”—the time participants spent composing queries and parsing AI responses, which in some cases took up over 30% of their total working time.
The data reveals a stark trade-off: participants who fully delegated tasks to the AI were the fastest to finish but suffered the worst learning outcomes, scoring as low as 39% on conceptual quizzes. Conversely, the control group, forced to resolve errors independently, developed a much deeper mastery of the material. The researchers identified six distinct interaction patterns, noting that only those who remained “cognitively engaged”—such as by asking for explanations rather than just code—managed to preserve their learning outcomes while using AI. The largest skill gap appeared in debugging, as the AI-reliant group failed to develop the “technical intuition” required to diagnose errors once the assistant was removed.
These findings highlight the growing risks of “vibe coding,” where developers rely on the high-level suggestions of AI models without grasping the underlying logic. While this approach may produce functional code in the short term, it creates a “knowledge debt” that leaves programmers ill-equipped to supervise or verify more complex AI systems in the future. As the industry moves toward more agentic and autocomplete-heavy workflows, the researchers warn that these “shortcuts” may ultimately compromise the development of the very skills needed to ensure software safety and reliability in critical domains.