Motivating online learners and driving engagement at scale

Learning online is hard, especially for people who aspire to be lifelong learners. I believe this effort to be similar to exercising since one must set the time to sit down, open a website, and start consuming the content but also engage with the materials, practice, and answer tests, all to make sure that the learning effort is effective. Naturally, sustaining the motivation over long periods of time (Ideally a lifetime) is hard for most people.

Learning Experience Designers and Product Managers creating Learning Technologies must find creative and simple ways to tap into students’ motivations and sustain the efforts.

Platzi Retos (Translated to English as Platzi Challenges) are learning experiences powered by a platform feature where students were challenged to complete specific learning activities before a deadline in an effort to identify what elements in design might help us drive sustained user engagement in a scalable manner.

Inspiration: Strava, a mobile app that helps athletes track their training and use data to improve their performance, has a feature called challenges, where athletes are motivated to achieve an athletic goal before a specific deadline.

The Learning Team at Platzi had been releasing Retos for a while, but these were communicated to students through blog posts. At the time (Late 2020, early 2021) 20 to 40 blogposts were published per week by the learning team, so the communication about Retos would be lost in the enormous sea of content published by the team. Furthermore, students had no way of tracking their performance and the learning team had little to no tools to measure the effectiveness of each Reto and improve the design accordingly.

The Retos, however, were very popular among students, and participation was observed to yield strong learning gains as well as a deeper commitment to the Platzi Learning Community.

We decided to create a feature that improved the visibility of Retos in order to grow the number of students enrolling and also improve the number of students actively learning in Platzi every month. The solution was simple and straightforward: Retos would be visible as cards in the student's home to make sure every student was exposed to them whenever they logged in to Platzi.

Measuring impact was straightforward and I will describe the results later in this post, but Retos opened the door to have cohorts of learners engaged in a common learning experience in a specific time window, which enabled the learning team to conduct learning experiments and research to better understand how people learn, and what learning experience design choices had an impact in learners’ evolution.

Did “Retos” work?

The advantages for the Learning Design Team were evident. Easier management of Retos and improved visibility ensured all students had the opportunity to find out about the available learning experiences, this enabled them to build more challenges, in less time, and manage them with more agility.

From a user engagement perspective, Retos’ impact was astounding. Students who participated in challenges through the Retos feature completed more courses than active students who didn’t participate in challenges but did participate in learning experiences

The increase in completed courses ranged from an average of 3 to sometimes 5 completed courses per month among course participants, compared with 1 to 1.5 average completed courses per month among non-participants.

On a personal note, when first analyzing the impact of the new feature I discovered that 11000 people had doubled their average completed courses per month without any expenses in marketing and very little time spent by the learning team, as some challenges only consisted on inviting students to complete a number of courses in a specific time-frame.

Challenges also had a strong impact on learning and enabled the team to conduct research to evaluate the effectiveness of our learning experiences and products. For example, a Challenge called Data Academy enables us to measure if students who completed Data Science courses developed deeper mathematical understandings. By conducting pre and post-testing as well as tracking students’ progress through a free, self-directed, science journal. We observed that Learners who participated and completed the courses had stronger mathematical learning gains and improved testing scores when compared to participants who did not complete the challenge.

These measurement exercises were possible to conduct because the Retos tool enabled different teams to better track learners and control for factors such as time, season, content, but also identify demographic variables, geographic context, prior experience, and competency level.

An exciting research revolved around analyzing how students’ self-expression evolved during learning. By (Manually) reviewing the images contained in the notes and science journals of students participating in the Data Science Challenge (Summer 2022) we noticed that the use of self-made images increased as the weeks progressed, students also relied less on screenshots from Platz’s courses. This helped us understand that more savvy learners are better suited to represent abstract ideas pictographically than novices, a useful trait when analyzing the level of competency of a learner and their evolution.

Conclusion

The new feature enabled us to increase the number of completed courses among Platzi’s students, this was important because completing courses is a predictor of learner engagement and subscription renewal, so this feature directly impacted our business goals while also enabling Platzi’s learning team to conduct novel research on learning evolution and outcomes. The research helped us confirm and identify online learning design best practices, as well as evaluate the effectiveness of our learning experiences.

Finally, Product Management for online learning technologies and ed-tech products is challenging because learning must be kept as the north star. While reaching business goals and addressing strategic needs is always part of the focus areas for Product Managers, learning outcomes must never be compromised. Moreover, if business goals and student outcomes are well-balanced, strong impact, and sustainable, scalable growth in ed-tech and success as a Product Manager in Ed-Tech are ensured.