This Evidence-Based Process Can Make Coding Based Learning More Effective

There will be 7 million current job openings in the U.S. by 2020. Over 1 million will be empty computer science-based jobs.

A tight labor market will become the “new normal” for decades to come. The solution – more effective workplace onboarding, training, and upskilling. Currently, companies spend millions on learning and development… but are those training programs really that successful? Here’s a closer look.

According to a study by BetterUp, companies across the U.S. invest an estimated $160B each year on various learning and development initiatives. Despite the high levels of investment, there are lots of question marks about the efficacy of those initiatives.

The problem is, learning comprehension is difficult to retain and hard to measure.

In fact, the same research suggests that 75% of learning material is forgotten.

The company recently released a series of evidence-based principles which can help to support transformation and knowledge acquisition within workforces. You might be interested in what this research can do to help you boost the results of your training initiatives within your engineering team.

Here’s a closer look at the research from the perspective of development to help your team continue to learn.

Code, Run, Evaluate

The research suggests that there are three stages that must be followed in order to achieve successful individual development and knowledge retention.

These stages are Learning, Doing, and Being. Focused solely on software developers, we call the stages Code, Run, and Evaluate.

Let’s take a closer look a the learning phase.


In this phase, there are four principles.

  • The research suggests that your team should engage in sustained microlearning.
  • Self-assessment is also a powerful step that can help your team to identify any deficits or opportunities for growth.
  • Hyperpersonalization is also recommended here, given that it helps your learners to understand where they are in terms of readiness.
  • Contextual learning is outlined as a method to reduce cognitive load on your developers. This stops your team from burning out while helping to reserve cognitive effort for other important tasks.
We recommend you use an application where developers can learn new content and change existing code based on solution hypothesis.


There are six principles in this phase:

  • The accurate identification of goals can help the learning to have a real-world impact and a concrete effect on work.
  • Accountability, reinforcement, and feedback helps to reinforce learning.
  • Assessment is a great way to regularly check back on how new learning is being applied and used in a real-world setting.
  • The research also suggests it’s important to calibrate the difficulty of the learning so that your developers are at the point of “proximal development”. Essentially, this means that the material should not be so simple it’s boring, or so difficult its intimidating.
  • It’s also important that the individual is easily understood or tool delivering the training is easily navigated.
  • Longitudinal Repetition” is also important. This means that complex skills have to be developed over time and habit will help to mould those abilities.
We recommend you run the code changes and observe the outcome measuring data points like keystroke dynamics.


In the final phase, there are just three principles.

  • For your team to get the most from the process, they should spend time to reflect on their own thoughts and growth.
  • The learners are encouraged to rewrite their narrative. This essentially means reflecting upon the lessons that have been learned to reorient the way the learner views themselves and the role that they play within the team.
  • Generative Practices” is the final principle. This means taking the learning and solidifying it further by repeating the learning and sharing it with others. This is particularly powerful in the world of development where there is a large volume of content.
On Code Pilot, you can create a project based on a configuration, which describes a stack of technologies/services, and a set of code problems.

The Solution

“In tight labor markets, employers must find ways to keep their people engaged. Development consistently shows up as one of the top items for driving employee engagement and retention,” says 2019 Human Capital Management Trends report, author Ben Eubanks.

We think that this method is certainly worth exploring if you want to boost the individual results you see from training and stop wasting $120B of your learning and development investment.

Code Pilot is a platform that makes it simple to create and deploy production grade development environments to gather coding analytics. Our customers include online education companies, technology companies, and engineering teams who use Code Pilot to help answer questions like, “How capable is this job applicant in coding against our stack?” or “How well did this employee understand the training at a coding level?”.

Email us for more details: