Educational research · November 2026

Beyond the Question Bank

Learning science, platform engagement and first-attempt NCE outcomes in nurse anesthesia education.

Download the white paper

Inside the full paper · Free PDF, 15 pages, no sign-up

  • Full methods and data definitions
  • All four NCE analysis charts
  • Research summaries with teaching tips
  • 10 references

Considering Ollivate for your program?

Book a 20-minute call

The educator challenge

Faculty know what works. Building it into every course is the hard part.

Retrieval practice, spaced repetition, interleaving and gamification can all support long-term retention. Each is doable on its own, but keeping all four running across a demanding curriculum takes real faculty time.

First-time NCE pass rates are visible outcomes for nurse anesthesia programs and part of accreditation reporting. That gives program directors good reason to look for tools that support student readiness while reducing the manual work for faculty.

The question behind the paper

“Can combining evidence-based learning strategies in one platform support consistent study engagement, and is that engagement associated with certification performance?”

Early findings

What 41 first-time NCE examinees showed us

The contrast

517vs423

Average NCE score for the five students who answered 6,000+ questions (all passed), compared with the five students who didn’t pass (avg. 769 questions).

First-attempt passing

21 of 21

passed with 2,000+ recorded questions, compared with 12 of 17 below 2,000.

Across all 38 students

r = 0.41

More recorded questions went with higher NCE scores on average, a moderate positive relationship.

How to read these. This is a small, observational sample. The first comparison sets a group defined by usage against a group defined by exam results, so it shows the range of outcomes we saw, not a benefit from answering more questions.

Higher recorded usage went with higher NCE scores

Average first-attempt NCE scaled score, nurse anesthesia examinees, 2025–2026

Average first-attempt NCE scaled score by group
GroupAverage NCE scoreStudentsPassed
Did not pass (avg. 769 questions)423.450 of 5
Under 2,000 questions471.21712 of 17
2,000–3,999 questions496.91212 of 12
4,000–5,999 questions496.044 of 4
6,000+ questions517.055 of 5
*These five students also appear in the Under 2,000 group. One more student who didn’t pass had no recorded question count. The axis starts at 350, not zero. Group averages are descriptive and don’t establish cause.

The learning science

Four methods with strong research behind them

The paper reviews published research on each one, with teaching tips faculty can use with or without Ollivate.

Retrieval practice

Answering questions builds memory more than rereading does.

Roediger & Karpicke, 2006

Spaced repetition

Coming back to material over time beats a single review session.

Cepeda et al., 2006 · 317 experiments

Interleaving

Mixing topics teaches students to choose which concept applies.

Brunmair & Richter, 2019

Gamification

Goals, progress and achievements help students keep practicing.

Sailer & Homner, 2020

For educators

Five practical moves

Engagement data is meant to complement course grades, SEE results and clinical feedback, as a prompt for an earlier, supportive conversation.

  1. Make retrieval routine. Use frequent low-stakes questions with explanations.

  2. Bring older material back. Build cumulative review across the curriculum.

  3. Interleave after initial learning. Mix related concepts once students have their footing.

  4. Reward consistency and improvement. Favor attainable goals and cooperative progress over public rankings.

  5. Respond to low engagement with support. Ask how students are preparing and agree on a realistic study plan.

What this data can’t tell us yet

  • It’s a small sample: 41 first-time examinees from 2025–2026, 38 with usage data.
  • It’s observational, with no adjustment for prior ability, school or exam year.
  • The 2,000-question grouping was chosen after reviewing the data. It isn’t a validated readiness threshold.
  • Question counts may include activity after the exam, and results span the July 2026 NCE passing-standard change.

Research ethics

The ongoing study has IRB approval. These preliminary findings come from a retrospective analysis of de-identified student records.

Prepared by Ollivate. Not independently peer reviewed.

Join the multisite study

Help us find out what holds across programs.

We’re inviting nurse anesthesia programs to take part in the next phase, which looks at whether these early patterns hold in a larger, more varied group of students.

  • Study engagement
  • Course exam and SEE scores
  • NCE outcomes
  • Student experience