Enforce a valid access token in Spark Data API requests

[bajtos]

Let's reject requests without a valid access token. The rejection message should include a link or an email address where to get help.

[bajtos]

Potentially tricky problem to solve: how to know if an access token is a valid one.

[pyropy]

Introduction

By observing our API token usage we have came to realisation that none of the requests use API token. This is in part due to the fact that the API token is not required for the current set of APIs. However, we have decided to implement API token authentication for all incoming requests to the API. This will allow us to have a better control over the API usage and also to have a better understanding of the API usage patterns.

Currently our API tokens are created by Tally on form submission and are stored in Google Sheets. I propose that we keep it this way and add an additional step to store the API tokens in Cloudflare Worker KV via custom webhook. This will allow us to authenticate each incoming request using the API token from Cloudflare workers while still keeping our sign-up process simple.

Overview

• Create Cloudflare Worker KV namespace to store API tokens.
• Add a new API endpoint to handle token creation from Tally.
• Store the API tokens in Cloudflare Worker KV.
• Authenticate each incoming request using the API token in case AUTH_ENABLED env variable is set to true.

Implementation plan

Create a new Cloudflare Worker KV namespace to store API tokens.

This can done using wrangler CLI as described here and wranger.toml configuration should be extended with the newly create KV namespace.

[[kv_namespaces]]
binding = "<BINDING_NAME>"
id = "<BINDING_ID>"

Add a new API endpoint to handle token creation from Tally and store created tokens.

We need to create a new route in our Cloudflare Worker to receive webhook notifications from Tally when a new form is submitted. This endpoint will be responsible for extracting the API token and storing it in KV.

// routes/saveToken.js
export async function handleSaveToken(request, env) {
  // Verify the request is coming from Tally using a shared secret
  verifyTallyWebhook(request, env);

  try {
    // Parse the incoming form data from Tally
    const formData = await request.json();
    
    // Extract relevant information
    const email = formData.fields.find(field => field.key === 'email').value;
    const apiToken = formData.fields.find(field => field.key === 'apiToken').value;
    
    if (!email || !apiToken) {
      return new Response('Missing required fields', { status: 400 });
    }
    
    // Store the API token in KV with the email as the key
    await env.API_TOKENS.put(apiToken, JSON.stringify({
      email,
      createdAt: new Date().toISOString()
    }));
    
    // Log token creation
    console.log(`API token created for ${email}`);
    
    return new Response(JSON.stringify({ success: true }), {
      headers: { 'Content-Type': 'application/json' }
    });
  } catch (error) {
    console.error('Error saving token:', error);
    return new Response('Internal Server Error', { status: 500 });
  }
}

Add authentication-related functions

We'll need to create a library file for authentication-related functions:

// lib/auth.js
export const getApiKey = (request) => {
  const url = new URL(request.url)
  return request.headers.get('api-key') ||
    url.searchParams.get('api-key') || (
    request.headers.get('Authorization')?.startsWith('Bearer ')
      ? request.headers.get('Authorization').substring(7)
      : 'unknown'
  )
}

export async function isAuthenticated(request, env) {
  // Get the API token from the request headers
  const apiKey = getApiKey(request);
  return (apiKey, env);
}

async function isValidApiKey(token, env) {
  try {
    // Check if the token exists in KV
    const tokenData = await env.API_TOKENS.get(token, { type: 'json' });
    return !!tokenData;
  } catch (error) {
    console.error('Error validating token:', error);
    return false;
  }
}

Update main worker file to include the new routes

We should modify our main worker file to include the new token creation endpoint:

import { isAuthenticated } from './lib/auth.js';
import { handleSaveToken } from './routes/saveToken.js';
import { reportRequestMetrics as reportRequestMetricsToInflux } from './lib/influx.js';

export default {
  async fetch(request, env, ctx, { reportRequestMetrics = reportRequestMetricsToInflux } = {}) {
    const url = new URL(request.url);
    
    // Handle webhook from Tally for token creation
    if (url.pathname === '/api/webhook/tally-token') {
      return handleSaveToken(request, env);
    }
    
    // Authenticate all other API requests
    if (env.AUTH_ENABLED === 'true' && !await isAuthenticated(request, env)) {
      return new Response('Please include an authentication token. If you don\'t have one, generate it at https://filspark.com/api-docs.', { status: 401 });
    }

    // Continue with the existing request handling
    const response = await fetch(request);
    ctx.waitUntil(reportRequestMetrics(request, env));
    return response;
  }
}

Add additional Tally setup instructions

1. In Tally, create a webhook that triggers when a form is submitted.
2. Set the webhook URL to your worker endpoint: https://yourworker.yourdomain.workers.dev/api/webhook/tally-token
3. Configure a shared secret in Tally and store it as an environment variable in your Cloudflare Worker.

Configure environment variables

Add the following environment variables to your Cloudflare Worker:

AUTH_ENABLED=false  # Set to true to enable authentication
TALLY_WEBHOOK_SECRET=your_shared_secret_here

Testing Plan

1. Test the token creation endpoint by simulating a Tally webhook request
2. Test authentication with a valid token
3. Test authentication with an invalid token
4. Test authentication with no token
5. Verify that the system works correctly when AUTH_ENABLED is set to false

Deployment Strategy

1. Deploy the changes to a staging environment first
2. Test all functionality in staging
3. Initially deploy to production with AUTH_ENABLED=false
4. Monitor for any issues
5. After confirming everything works and we decide we want to force authentication, set AUTH_ENABLED=true

Monitoring and Metrics

Add additional metrics tracking for authentication:

• Number of authenticated requests
• Number of unauthenticated requests
• Number of new API tokens created
• Number of failed token creation attempts

This information can be integrated with our existing Influx metrics system to provide visibility into API usage patterns.