Skip to content

Gahroot/the-forge

Repository files navigation

The Forge

Build powerful asynchronous automations with Claude Code. Drop your N8N, Zapier, or Make workflows—this is a developer-first automation framework with built-in resilience patterns, 21 pre-built integrations, and production-ready infrastructure.

Why The Forge?

For Claude Code Users: Tell Claude to "build an automation like ai-actor-testimonials" and reference this codebase. Claude will understand the patterns and create production-ready async workflows.

For Developers: No-code tools are limited. The Forge gives you full TypeScript control with circuit breakers, rate limiting, job queues, and structured logging—all pre-configured.

For Teams: Replace fragile Zapier chains with version-controlled, testable automation code that scales.

Quick Start

1. Clone and Install

git clone https://github.com/yourusername/the-forge.git
cd the-forge
npm install

2. Start Infrastructure

# Start PostgreSQL and Redis via Docker
npm run docker:start

# Copy environment template
cp .env.example .env.local

3. Configure Environment

Edit .env.local with your credentials:

# Required
DATABASE_URL=postgresql://postgres:postgres@localhost:5435/forge_dev
REDIS_URL=redis://localhost:6381

# Add API keys for integrations you'll use
OPENAI_API_KEY=sk-...
SLACK_BOT_TOKEN=xoxb-...
# See integrations list below for all options

4. Initialize Database

npm run db:push

5. Run Development Server

# Run both Next.js server and background worker
npm run dev:full

Open http://localhost:3456 to see the dashboard.


Using with Claude Code

The Forge is designed to be understood and extended by Claude Code. Here's how to leverage it:

Create New Automations with Claude

Tell Claude:

I want to build asynchronous functions with resilience patterns.
Reference /path/to/the-forge as an example. Understand it completely
because I want to build a new automation similar to how ai-actor-testimonials functions.

Claude will analyze the codebase and create automations following the established patterns:

  • Pipeline-based multi-step workflows
  • Circuit breakers for external API calls
  • Rate limiting to respect API quotas
  • Structured logging throughout
  • Database tracking of all runs

Example: Recreating a Zapier Workflow

Zapier workflow: "When a new Stripe payment comes in, create a Notion page, send a Slack message, and add to Airtable"

Tell Claude:

Create an automation called "payment-processor" that:
1. Receives Stripe webhook for successful payments
2. Creates a Notion page with payment details
3. Sends a Slack notification to #sales
4. Adds a row to our Airtable CRM

Use the pipeline pattern from ai-actor-testimonials. Include circuit
breakers for each external service.

Claude will generate a production-ready automation following The Forge patterns.


Project Structure

src/
├── app/                    # Next.js App Router
│   ├── api/                # API endpoints
│   │   ├── automations/    # Automation CRUD & triggers
│   │   ├── credentials/    # Credential management
│   │   ├── health/         # Health check endpoint
│   │   └── runs/           # Automation run logs
│   └── dashboard/          # Dashboard UI
├── automations/            # Your automation workflows
│   ├── index.ts            # Automation registry
│   └── examples/           # Example automations
├── components/             # Reusable React components
├── integrations/           # 21 pre-built service clients
└── lib/                    # Core utilities
    ├── automation.ts       # Automation orchestration
    ├── pipeline.ts         # Multi-step workflow builder
    ├── queue.ts            # BullMQ job queue
    ├── resilience.ts       # Circuit breaker patterns
    ├── rate-limiter.ts     # API rate limiting
    ├── scheduler.ts        # Cron scheduling
    └── schema.ts           # Database schema

Creating Automations

Anatomy of an Automation

Every automation in The Forge follows this structure:

// src/automations/my-automation.ts
import { defineAutomation } from '@/lib/automation';
import { createPipeline } from '@/lib/pipeline';
import { slack, openai } from '@/integrations';

interface WorkflowContext {
  // Data passed between steps
  inputData: string;
  processedData?: string;
  result?: string;
}

export default defineAutomation({
  name: 'my-automation',
  description: 'What this automation does',

  // Optional: Run on schedule (cron syntax)
  schedule: '0 9 * * *', // Daily at 9am

  // Optional: Expose as webhook endpoint
  webhook: true,

  // Required API keys
  credentials: ['OPENAI_API_KEY', 'SLACK_BOT_TOKEN'],

  async run(ctx) {
    const pipeline = createPipeline<WorkflowContext>();

    const result = await pipeline
      .step('process-input', async (data) => {
        // Step 1: Process input
        return { ...data, processedData: data.inputData.toUpperCase() };
      })
      .step('ai-analysis', async (data) => {
        // Step 2: AI processing with built-in resilience
        const analysis = await openai.prompt(data.processedData);
        return { ...data, result: analysis };
      })
      .step('notify', async (data) => {
        // Step 3: Send notification
        await slack.sendMessage('#general', data.result!);
        return data;
      })
      .execute({ inputData: ctx.input });

    return result.finalData;
  },
});

Register Your Automation

Add to src/automations/index.ts:

import myAutomation from './my-automation';

const automations: AutomationConfig[] = [
  aiActorTestimonials,
  myAutomation, // Add here
];

export default automations;

Trigger Your Automation

Via API:

curl -X POST http://localhost:3456/api/automations/my-automation/trigger \
  -H "Content-Type: application/json" \
  -d '{"inputData": "hello world"}'

Via Dashboard: Navigate to http://localhost:3456/dashboard

Via Cron: Set the schedule property in your automation definition.


Example: AI Actor Testimonials

The included example converts customer testimonials into UGC-style video ads:

src/automations/examples/ai-actor-testimonials.ts

Workflow:

  1. Validate input testimonial
  2. Generate high-converting UGC script (OpenAI)
  3. Submit video generation (Mirage API)
  4. Save operation to database

Features demonstrated:

  • Pipeline pattern for multi-step workflows
  • Circuit breakers on external APIs
  • Rate limiting (OpenAI: 500 req/min, Mirage: 5 req/min)
  • Structured logging to logs/app.log
  • Database tracking of all runs

Trigger it:

curl -X POST http://localhost:3456/api/automations/ai-actor-testimonials/trigger \
  -H "Content-Type: application/json" \
  -d '{
    "testimonialText": "This product changed my life!",
    "creatorName": "sarah_lifestyle",
    "targetDuration": 30
  }'

Pre-Built Integrations

All integrations include circuit breakers, rate limiting, and structured logging:

Integration Description Required Env Var
openai GPT models, embeddings OPENAI_API_KEY
anthropic Claude AI ANTHROPIC_API_KEY
slack Messaging, channels SLACK_BOT_TOKEN
discord Bots, webhooks DISCORD_BOT_TOKEN
twitter Posts, DMs TWITTER_* credentials
telegram Bots, messages TELEGRAM_BOT_TOKEN
github Repos, issues, PRs GITHUB_TOKEN
notion Pages, databases NOTION_API_KEY
airtable Tables, records AIRTABLE_API_KEY
sheets Google Sheets GOOGLE_* credentials
email SMTP email SMTP_* credentials
sendgrid Transactional email SENDGRID_API_KEY
resend Modern email API RESEND_API_KEY
stripe Payments, webhooks STRIPE_SECRET_KEY
twilio SMS, voice TWILIO_* credentials
s3 AWS S3 storage AWS_* credentials
cloudinary Image/video hosting CLOUDINARY_* credentials
mongodb MongoDB database MONGODB_URI
http Generic HTTP client None
webhook Outgoing webhooks None

Using Integrations

import { openai, slack, notion } from '@/integrations';

// OpenAI
const response = await openai.prompt('Summarize this:', 'Be concise');
const embeddings = await openai.embed(['text1', 'text2']);

// Slack
await slack.sendMessage('#channel', 'Hello from automation!');

// Notion
await notion.createPage(databaseId, { title: 'New Page' });

Creating Custom Integrations

To add your own integration, reference existing integrations in src/integrations/ as examples. Tell Claude:

I want to add a new integration for [ServiceName].
Reference /path/to/the-forge/src/integrations/slack.ts as an example.
Include circuit breakers and rate limiting like the existing integrations.

Integration structure:

// src/integrations/my-service.ts
import { createCircuitBreaker } from '@/lib/resilience';
import { createRateLimiter, withRateLimit } from '@/lib/rate-limiter';
import { logger } from '@/lib/logger';

// 1. Initialize client
let client: MyServiceClient | null = null;

function getClient(): MyServiceClient {
  if (!client) {
    const apiKey = process.env.MY_SERVICE_API_KEY;
    if (!apiKey) throw new Error('MY_SERVICE_API_KEY not set');
    client = new MyServiceClient({ apiKey });
  }
  return client;
}

// 2. Create rate limiter
const rateLimiter = createRateLimiter({
  maxConcurrent: 5,
  minTime: 200,
  id: 'my-service',
});

// 3. Internal function (unprotected)
async function doSomethingInternal(param: string): Promise<Result> {
  const client = getClient();
  logger.info({ param }, 'Calling MyService API');
  return client.doSomething(param);
}

// 4. Protected with circuit breaker + rate limiting
const doSomethingWithBreaker = createCircuitBreaker(doSomethingInternal);
export const doSomething = withRateLimit(
  (param: string) => doSomethingWithBreaker.fire(param),
  rateLimiter
);

Register in src/integrations/index.ts:

export * as myService from './my-service';

Resilience Patterns

Circuit Breakers

Prevent cascading failures when external services go down:

import { createCircuitBreaker } from '@/lib/resilience';

const protectedFn = createCircuitBreaker(myApiCall, {
  timeout: 10000,           // 10s timeout
  errorThresholdPercentage: 50,  // Open after 50% failures
  resetTimeout: 30000,      // Try again after 30s
  volumeThreshold: 5,       // Minimum calls before tripping
});

// Use it
await protectedFn.fire(arg1, arg2);

Rate Limiting

Respect API quotas automatically:

import { createRateLimiter, withRateLimit } from '@/lib/rate-limiter';

const limiter = createRateLimiter({
  maxConcurrent: 5,
  minTime: 200, // 5 requests per second
});

const rateLimitedFn = withRateLimit(myFunction, limiter);

Pipelines

Build multi-step workflows with automatic error handling:

import { createPipeline } from '@/lib/pipeline';

const result = await createPipeline<MyContext>()
  .step('step-1', async (ctx) => ({ ...ctx, data: 'processed' }))
  .step('step-2', async (ctx) => ({ ...ctx, result: 'done' }))
  .execute(initialContext);

if (!result.success) {
  console.log('Failed at:', result.results.find(r => !r.success)?.name);
}

Job Queue

Background processing with BullMQ:

import { addJob, QUEUE_NAMES } from '@/lib/queue';

// Add job to queue
await addJob(QUEUE_NAMES.AUTOMATIONS, {
  automationName: 'my-automation',
  trigger: 'manual',
  input: { data: 'value' },
});

Jobs are processed by the worker (npm run worker), with automatic retries and failure handling.


Scheduling

Cron-based scheduling:

// In automation definition
export default defineAutomation({
  name: 'daily-report',
  schedule: '0 9 * * *', // Every day at 9am
  // ...
});

Common cron patterns:

  • * * * * * - Every minute
  • 0 * * * * - Every hour
  • 0 9 * * * - Daily at 9am
  • 0 9 * * 1 - Every Monday at 9am
  • 0 0 1 * * - First of each month

API Reference

Automations

# List all automations
GET /api/automations

# Trigger an automation
POST /api/automations/{name}/trigger
Body: { ...input data }

# Get automation runs
GET /api/runs?automationId={name}&limit=50

Health Check

GET /api/health
# Returns: { status: 'ok', timestamp: '...' }

Commands Reference

Command Description
npm run dev Start Next.js server (port 3456)
npm run worker Start background worker
npm run dev:full Start both server and worker
npm run build Build for production
npm run lint Run ESLint
npm run typecheck Run TypeScript check
npm run db:push Push schema to database
npm run db:studio Open Drizzle Studio
npm run docker:start Start PostgreSQL + Redis
npm run docker:stop Stop Docker containers

Environment Variables

Create .env.local:

# Infrastructure (Required)
DATABASE_URL=postgresql://postgres:postgres@localhost:5435/forge_dev
REDIS_URL=redis://localhost:6381

# AI
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...

# Communication
SLACK_BOT_TOKEN=xoxb-...
DISCORD_BOT_TOKEN=...
TELEGRAM_BOT_TOKEN=...

# Email
SENDGRID_API_KEY=SG....
RESEND_API_KEY=re_...
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=...
SMTP_PASS=...

# Payments
STRIPE_SECRET_KEY=sk_...

# Storage
AWS_ACCESS_KEY_ID=...
AWS_SECRET_ACCESS_KEY=...
AWS_REGION=us-east-1
AWS_S3_BUCKET=...

CLOUDINARY_CLOUD_NAME=...
CLOUDINARY_API_KEY=...
CLOUDINARY_API_SECRET=...

# Databases
MONGODB_URI=mongodb://...

# Social
TWITTER_API_KEY=...
TWITTER_API_SECRET=...
TWITTER_ACCESS_TOKEN=...
TWITTER_ACCESS_SECRET=...

# Productivity
NOTION_API_KEY=secret_...
AIRTABLE_API_KEY=...
GITHUB_TOKEN=ghp_...

# Google
GOOGLE_CLIENT_EMAIL=...
GOOGLE_PRIVATE_KEY=...

Production Deployment

Build

npm run build

Run

# Start server
npm run start

# Start worker (separate process)
npm run worker:prod

Docker (Recommended)

The included docker-compose.yml provides PostgreSQL and Redis. For production, add your app:

services:
  app:
    build: .
    ports:
      - "3456:3456"
    environment:
      - DATABASE_URL=postgresql://postgres:postgres@postgres:5432/forge_prod
      - REDIS_URL=redis://redis:6379
    depends_on:
      - postgres
      - redis

Migrating from No-Code Tools

From Zapier

Zapier Concept The Forge Equivalent
Zap Automation file in src/automations/
Trigger webhook: true or schedule: '...'
Action Pipeline step
Filter Conditional logic in step
Path Multiple pipelines or branching logic

From N8N

N8N Concept The Forge Equivalent
Workflow Automation file
Trigger Node webhook or schedule property
Function Node Pipeline step
HTTP Request http integration
IF Node JavaScript conditionals

From Make (Integromat)

Make Concept The Forge Equivalent
Scenario Automation file
Module Integration or pipeline step
Router Conditional logic in steps
Aggregator Array operations in JavaScript

Contributing

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/amazing-automation)
  3. Write your automation following the patterns in src/automations/examples/
  4. Run quality checks: npm run lint && npm run typecheck
  5. Commit your changes (git commit -m 'Add amazing automation')
  6. Push to the branch (git push origin feature/amazing-automation)
  7. Open a Pull Request

License

MIT


Support

  • Open an issue for bugs or feature requests
  • Check existing automations in src/automations/examples/ for patterns
  • Reference this README when prompting Claude Code to create new automations

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors