Using Thinkific webhooks to track progress by lesson through a course

Another post about Thinkific’s webhooks!

In Thinkific, when a user advances through a course, two webhooks fire more or less simultaneously — enrollment.progress and lesson.completed.

The enrollment.progress webhook contains some nice information about the user’s current progress through the course as a percentage, and also their prior progress. So you can use the information in this webhook to see how far along a user is in the course they’re taking.

Meanwhile the lesson.completed webhook contains some nice information about the specific lesson that the user just finished — the lesson ID and name, and also the ID and name of the chapter that this lesson is housed in.

Unfortunately neither webhook contains both sets of information! So if you want to track something like “[X] user completed [Y] lesson in course [Z] on [TIMESTAMP] thus advancing to [PCT]% progress” you’ll need to use both webhooks.¹

Merging data from both webhooks

Making matters slightly more complicated, there doesn’t appear to be any shared session ID or other key in the payloads that can be used to fully-reliably collate the information from these two webhooks.

That said, the event timestamps in combination with the user ID and course ID allows you to approximate well enough.

Each webhook contains a timestamp in its payload, distinct from when the payload was received. So you can stash your webhook events in a database and then, at some other point, just look for the two closest events of these two types for a given user and course.

Here’s more or less how I’m doing it in D1 (SQLite) starting from a given enrollment.progress event that I’ve just received, and then looking (either forward or backwards) for the closest-matching lesson.completed event associated with the same user & course, within some fixed “they probably both came in within this duration” time interval:

SELECT lesson_id, chapter_id 
  FROM thinkific_webhook_event 
 WHERE user_id = {user_id}
       AND course_id = {course_id}
       AND topic = 'lesson.completed' 

       -- Look for matching events within ±500ms;
       -- in practice the events should match precisely 
       -- because Thinkific sends us the event timestamp
       -- which is independent of the webhook-firing timestamp
       -- but you never know
       AND timestamp between {timestamp - 500} and {timestamp + 500}

 ORDER BY abs({timestamp} - timestamp) ASC LIMIT 1

Leveraging the webhook infrastructure

So, when should you run this query?

One option is to set up a cron job that just checks frequently for stashed webhooks that need to be collated. That’s pretty wasteful!

Another option is to set up a one-time alarm / timeout / delay that then asks your database the question, and if no matches are found, repeats for some fixed number of tries. That’s better, but can be a little awkward to set up & log, and doesn’t feel great.

The approach I like is just making the webhook infrastructure do that for me. Like most webhook-firing platforms, Thinkific will retry failed webhooks repeatedly (up to 14 times) on a gradually increasing backoff schedule. So when listening for both of these webhooks, you can:

• Stash the webhook payload in your database if it’s not already in there

• Query your database immediately for the other event to collate

• If found, great! Proceed with your logic² and return a 2xx status code

• If not found, that’s fine! Terminate your logic and return a 4xx status code.³

Now Thinkific will just take care of the alarm/retry/delay logic for you, and you can keep your listening infrastructure a lot simpler.

1. Alternatively, you could listen only for the lesson.completed webhook, and then make a REST API call to fetch information about the associated enrollment by ID. That response will contain the current percentage_completed. However, this won’t tell you the prior progress percentage, which can be helpful for identifying a user’s first time through the course among other use cases. It’s also going to be more prone to race conditions since webhooks can reach you with some delay and users can move quickly through a course — so by the time you’ve seen the event and made an API call, the user may have completed five more lessons and have an entirely unrelated progress percentage! So if you actually need the percentage as of the lesson.completed event this is not a great idea.

2. But pick only one of the two webhook types to be your actual logic-running channel.

3. Best not to use a generic 400 here because it will make your logs a lot harder to filter for actual problems. I like 409 Conflict for this but there’s a lot of great options in the 4xx range — 422, 424, and 418 are solid choices too. Take your pick!