Skip to main content

Webhooks

Receive real-time notifications when scrapes complete or fail.

Setup Webhook​

Provide a webhook URL when creating a scrape:

result = client.scrape(
    url="https://example.com",
    options={
        "async": True,
        "webhook_url": "https://your-app.com/webhook",
        "webhook_events": ["completed", "failed"]
    }
)

Webhook Payload​

When a scrape completes, Wryn sends a POST request:

{
  "event": "scrape.completed",
  "scrape_id": "scr_1234567890",
  "status": "success",
  "url": "https://example.com",
  "data": {
    "title": "Example Page",
    "description": "..."
  },
  "metadata": {
    "scraped_at": "2025-12-06T10:30:00Z",
    "response_time": 1.5
  }
}

Handle Webhook​

Flask example:

from flask import Flask, request

app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def handle_webhook():
    payload = request.json

    if payload['event'] == 'scrape.completed':
        scrape_id = payload['scrape_id']
        data = payload['data']
        # Process the data
        process_scraped_data(data)

    elif payload['event'] == 'scrape.failed':
        error = payload['error']
        # Handle the error
        log_error(error)

    return {'status': 'ok'}, 200

if __name__ == '__main__':
    app.run(port=5000)

Verify Webhook Signature​

Verify requests come from Wryn:

import hmac
import hashlib

def verify_signature(payload, signature, secret):
    expected = hmac.new(
        secret.encode(),
        payload.encode(),
        hashlib.sha256
    ).hexdigest()

    return hmac.compare_digest(expected, signature)

@app.route('/webhook', methods=['POST'])
def handle_webhook():
    signature = request.headers.get('X-Wryn-Signature')
    payload = request.get_data(as_text=True)

    if not verify_signature(payload, signature, WEBHOOK_SECRET):
        return {'error': 'Invalid signature'}, 401

    # Process webhook
    return {'status': 'ok'}, 200

Event Types​

EventDescription
scrape.completedScrape finished successfully
scrape.failedScrape encountered an error
scrape.partialScrape completed with warnings

Retry Logic​

Wryn retries failed webhook deliveries:

  • 3 retry attempts
  • Exponential backoff (1s, 5s, 25s)
  • 5-second timeout per attempt

Next Steps​