@happyvertical/comfyui

A WebSocket-based client for ComfyUI workflow orchestration and video generation in the HAVE SDK.

Overview

The @happyvertical/comfyui package provides a TypeScript client for interacting with ComfyUI servers via WebSocket. It supports workflow execution, progress monitoring, and output downloading for AI-powered image and video generation pipelines.

Features

WebSocket Connection: Real-time communication with ComfyUI servers
Workflow Execution: Queue and execute ComfyUI workflows
Progress Monitoring: Track execution progress with detailed events
Parameter Injection: Dynamically inject values into workflow nodes
Output Download: Retrieve generated images and videos
Queue Management: Monitor and manage the execution queue
System Stats: Query server status and capabilities
File Upload: Upload images and other files for workflow inputs

Installation

# Install with bun (recommended)
bun add @happyvertical/comfyui

# Or with npm
npm install @happyvertical/comfyui

# Or with pnpm
pnpm add @happyvertical/comfyui

Quick Start

Basic Usage

import { ComfyUIClient, injectWorkflowParams } from '@happyvertical/comfyui';

// Create client
const client = new ComfyUIClient({
  url: 'http://localhost:8188',
});

// Connect to server
await client.connect();

// Load and modify workflow
const workflow = JSON.parse(fs.readFileSync('workflow.json', 'utf-8'));
const modifiedWorkflow = injectWorkflowParams(
  workflow,
  {
    seedImage: '3',
    prompt: '6',
  },
  {
    seedImage: 'input/anchor.png',
    prompt: 'professional news anchor speaking',
  }
);

// Queue prompt and wait for completion
const result = await client.queuePrompt(modifiedWorkflow);
const history = await client.waitForCompletion(result.promptId);

// Download output
const output = history.outputs['SaveVideo'];
if (output?.videos?.[0]) {
  const video = await client.downloadOutput(output.videos[0].filename);
  fs.writeFileSync('output.mp4', video);
}

await client.disconnect();

Progress Tracking

const history = await client.waitForCompletion(promptId, {
  onProgress: (event) => {
    switch (event.type) {
      case 'progress':
        console.log(`Progress: ${event.value}/${event.max}`);
        break;
      case 'executing':
        console.log(`Executing node: ${event.node}`);
        break;
      case 'execution_cached':
        console.log(`Cached nodes: ${event.nodes.join(', ')}`);
        break;
    }
  },
});

File Upload

// Upload image for workflow input
const imageBuffer = fs.readFileSync('input.png');
const uploadResult = await client.uploadImage(imageBuffer, 'input.png');

console.log(`Uploaded: ${uploadResult.name}`);

System Status

// Get system stats
const stats = await client.getSystemStats();
console.log(`Queue: ${stats.queue.running}/${stats.queue.pending}`);

// Get queue status
const queue = await client.getQueueStatus();
console.log(`Running: ${queue.running.length}, Pending: ${queue.pending.length}`);

API Reference

ComfyUIClient

The main client class for interacting with ComfyUI servers.

Constructor

new ComfyUIClient(options: ComfyUIClientOptions)

Option	Type	Default	Description
`url`	`string`	-	ComfyUI server URL (e.g., `http://localhost:8188`)
`clientId`	`string`	auto-generated	Unique client identifier
`timeout`	`number`	`600000`	Request timeout in milliseconds

Methods

Method	Description
`connect()`	Establish WebSocket connection
`disconnect()`	Close WebSocket connection
`queuePrompt(workflow)`	Queue a workflow for execution
`waitForCompletion(promptId, options?)`	Wait for prompt completion with progress tracking
`getHistory(promptId)`	Get execution history for a prompt
`getQueueStatus()`	Get current queue status
`getSystemStats()`	Get server system statistics
`uploadImage(buffer, filename, options?)`	Upload an image file
`downloadOutput(filename, subfolder?, type?)`	Download a generated output file
`interruptExecution()`	Interrupt the current execution
`clearQueue()`	Clear all pending queue items

injectWorkflowParams

Utility function to inject dynamic values into workflow nodes.

function injectWorkflowParams(
  workflow: ComfyWorkflow,
  nodeMapping: NodeMapping,
  values: Record<string, unknown>
): ComfyWorkflow

Parameters

workflow - The ComfyUI workflow JSON
nodeMapping - Maps parameter names to node IDs (e.g., { seedImage: '3' })
values - Values to inject (e.g., { seedImage: 'path/to/image.png' })

Types

interface ComfyUIClientOptions {
  url: string;
  clientId?: string;
  timeout?: number;
}

interface PromptResult {
  promptId: string;
  number: number;
  nodeErrors?: Record<string, unknown>;
}

interface HistoryEntry {
  promptId: string;
  outputs: Record<string, NodeOutput>;
  status: { completed: boolean; messages?: unknown[] };
}

interface ProgressEvent {
  type: 'progress' | 'executing' | 'executed' | 'execution_cached' | 'execution_error';
  promptId?: string;
  node?: string;
  value?: number;
  max?: number;
  nodes?: string[];
}

Use Cases

AI Video Generation Pipeline

import { ComfyUIClient, injectWorkflowParams } from '@happyvertical/comfyui';

async function generateVideo(seedImage: string, audioFile: string) {
  const client = new ComfyUIClient({ url: 'http://localhost:8188' });
  await client.connect();

  try {
    // Upload inputs
    await client.uploadImage(fs.readFileSync(seedImage), 'seed.png');
    await client.uploadImage(fs.readFileSync(audioFile), 'audio.wav');

    // Load lip-sync workflow
    const workflow = JSON.parse(fs.readFileSync('lipsync-workflow.json', 'utf-8'));
    const modified = injectWorkflowParams(workflow, {
      seedImage: '3',
      audioFile: '5',
    }, {
      seedImage: 'input/seed.png',
      audioFile: 'input/audio.wav',
    });

    // Execute with progress
    const result = await client.queuePrompt(modified);
    const history = await client.waitForCompletion(result.promptId, {
      onProgress: (e) => {
        if (e.type === 'progress') {
          console.log(`Rendering: ${Math.round((e.value! / e.max!) * 100)}%`);
        }
      },
    });

    // Download output
    const output = history.outputs['SaveVideo'];
    return await client.downloadOutput(output.videos[0].filename);
  } finally {
    await client.disconnect();
  }
}

Batch Image Generation

async function generateBatch(prompts: string[]) {
  const client = new ComfyUIClient({ url: 'http://localhost:8188' });
  await client.connect();

  const results = [];
  for (const prompt of prompts) {
    const workflow = injectWorkflowParams(baseWorkflow, { prompt: '6' }, { prompt });
    const result = await client.queuePrompt(workflow);
    const history = await client.waitForCompletion(result.promptId);
    results.push(history);
  }

  await client.disconnect();
  return results;
}

Requirements

ComfyUI server running and accessible
Node.js 20+ or Bun
WebSocket support in runtime

License

This package is part of the HAVE SDK and is licensed under the MIT License - see the LICENSE file for details.

Overview​

Features​

Installation​

Quick Start​

Basic Usage​

Progress Tracking​

File Upload​

System Status​

API Reference​

ComfyUIClient​

Constructor​

Methods​

injectWorkflowParams​

Parameters​

Types​

Use Cases​

AI Video Generation Pipeline​

Batch Image Generation​

Requirements​

License​