« Philippe Creux

Handling webhooks in Ruby on Rails

19 Jan 2024

I had to handle webhooks on most web applications that I worked on: email events (deliveries, bounces, etc), payment/subscription notifications, inbound emails, etc. They can all contain large payloads and multiple events, and they expect the app to return promptly a successful response otherwise they eventually retry.

I recommend to:

1. Make the controller as simple as possible and process webhooks asynchronously
2. Process individual events contained in the webhook asynchronously
3. Persist the payload to the database to improve visibility for monitoring and debugging purposes

1. Make the controller as simple as possible and process webhooks asynchronously

I’ve seen apps where the webhook is entirely handled in the controller. That can work for small payloads and simple logic but it doesn’t scale™. Production level payload size, data, and logic are likely to make your app take several seconds to process the webhook – that will block a web process and the service sending the webhook could time out as they tend to have short request timeouts (~5 to 10 seconds).

You also don’t control the schema of the webhook and while it should remain fairly stable nothing prevents the third-party service from updating it. For example , I ran into timestamp format changes multiple times.

So I recommend processing the webhook asynchronously and making the controller as simple as possible.

2. Process individual events contained in the webhook asynchronously

Webhooks often contain a batch of events. To prevent a failure to affect all events, I tend to make the controller parse the payload to enqueue one job per event. Your job will be responsible for processing just one event - nice!

3. Persist the payload to the database to improve visibility for monitoring and debugging purposes

When the logic or the payload are complex you want to have good visibility into the webhooks and to be able to re-process them.

I find persisting the raw payloads in an ActiveRecord model the best way to support this.

The controller persists the webhook payload (or webhooks events) in ActiveRecord models A job processes the webhook record The job marks the webhook record as completed or failed.

Here is a sample code:

class StripeWebhook < ApplicationModel
  # id, type, payload:json, created_at, updated_at, status, details:json

  enum { pending: 10, completed: 20, failed: 30, skipped: 40 }

  after_create :enqueue_job

  def failed!(error)
    update!(status: :failed, details: { error_message: error.message, error_type: error.class.name, error_backtrace: error.backtrace })
  end

  def enqueue_job
    StripeWebhookJob.constantize.perform_async(self)
  end
end

class StripeWebhookJob < ApplicationJob
  attr_reader :webhook

  def perform(webhook)
    @webhook = webhook

    ApplicationRecord.transaction do
      webhook.with_lock do
        break unless webhook.pending?

        execute(webhook)

      rescue => e
        webhook.failed!(e)
        raise e
      end
    end
  end

  def execute(webhook)
    # real code here
  end
end

You could extract the “BackgroundJob model and job” behaviors into concerns if this is something you want to apply to other webhooks or background jobs (ex: generating a report, importing a CSV, etc).

Oh, and it’s also worth mentioning that webhooks are not always sent, received, and processed in order – so watch out for race conditions, especially when it has to do with subscriptions or payments. :)

Feel free to leave comments on GitHub.