How Artificial Intelligence Supports the Growth of Programming Skills

Research shows that AI can help people complete tasks faster. In an observational study of Claude.ai, some tasks were completed up to 80% faster with AI assistance. However, this boost in productivity may come with a cost: other studies suggest that using AI can reduce engagement and encourage cognitive offloading, where users rely on AI instead of thinking for themselves.

This raises an important question: Does using AI at work, particularly in coding, hinder skill development and a deeper understanding of the systems being built?

Our latest study—a randomized controlled trial with software developers—explores this very issue.

Why It Matters

The findings have wide implications:

• How AI tools should be designed to support learning

• How workplaces should set AI usage policies

• How society can ensure long-term workforce resilience

In coding, AI is increasingly standard. While AI can speed up development, humans still need skills to:

• Detect errors

• Guide AI output

• Oversee AI in high-stakes projects

This study asked: Does AI increase efficiency without sacrificing skill acquisition, or do productivity gains come at the expense of learning?

Read More: How Can ‘Vibe Coding’ Make You Millions Overnight?

Study Design

We recruited 52 mostly junior software engineers with weekly Python experience for over a year. All were somewhat familiar with AI coding assistants but new to the Trio Python library, used in the tasks.

The study had three parts:

1. Warm-up

2. Coding task: Participants coded two features using Trio, a library requiring knowledge of asynchronous programming

3. Quiz: Tested comprehension of concepts just applied

Participants were informed of a quiz but encouraged to work as quickly as possible, mimicking self-guided learning.

The AI assistant had access to participants’ code and could generate solutions if requested.

Assessment Metrics

We focused on four areas critical for coding mastery:

1. Debugging: Identify and fix errors in code

2. Code reading: Comprehend what the code does

3. Code writing: Apply correct coding approaches

4. Conceptual understanding: Grasp the underlying principles of libraries and tools

Debugging, code reading, and conceptual understanding were prioritized, as they are key to overseeing AI-generated code.

Read More: The Programmer Behind “Vibe Coding” Admits He Feels Left Behind

Key Findings

• Participants using AI completed tasks about 2 minutes faster (not statistically significant)

• Quiz scores were significantly lower for the AI group: 50% vs. 67% for the hand-coding group (~two letter grades lower, Cohen’s d=0.738, p=0.01)

• The largest gap was in debugging skills, showing that AI may reduce the ability to detect errors

How Interaction Style Affects Learning

Not all AI use led to lower scores. How participants used AI influenced comprehension:

Low-Scoring Patterns

Heavy reliance on AI led to poor learning:

• AI Delegation: Fully relied on AI to write code (fastest completion, quiz <40%)

• Progressive AI Reliance: Gradually offloaded all tasks to AI

• Iterative AI Debugging: Relied on AI for debugging rather than understanding

High-Scoring Patterns

Effective use of AI boosted understanding:

• Generation-then-Comprehension: Generated code, then used AI to clarify concepts

• Hybrid Code- Explanation: Asked AI to generate code and explain it

• Conceptual Inquiry: Asked only conceptual questions, resolving errors independently

These patterns suggest that active engagement with AI supports learning, while passive reliance may hinder it.

Implications

The study highlights a key trade-off in AI-augmented work:

• Productivity gains may come at the cost of skill development

• Junior engineers could lose critical debugging and comprehension skills

• Managers must design AI deployment strategies that foster learning while improving efficiency

Tools like Claude Code Learning or ChatGPT Study Mode show how AI can be used intentionally to support skill development.

Broader Context

Previous studies showed AI can reduce task completion time by up to 80%. However, those studies focused on tasks where participants already had skills. Our study focuses on learning new skills, where AI may speed up work but reduce mastery.

Read More: Most Popular AI Tools for Coding Every Developer Should Know

Conclusion

AI can transform workplaces, but the benefits depend on how humans interact with it.

To maintain long-term skill development while boosting productivity:

• Encourage active engagement with AI

• Design AI tools to teach, not just automate

• Monitor skill retention and comprehension

In an AI-driven workplace, efficiency matters—but so does developing expertise to oversee AI systems effectively.