@happyvertical/smrt-video

Video production pipeline for AI-powered video generation in the s-m-r-t ecosystem. Models characters, performers, scenes, shots, sequences, compositions, and ComfyUI workflow templates.

Installation

pnpm add @happyvertical/smrt-video

Usage

import {
  Character,
  Performer,
  Scene,
  VideoShot,
  VideoSequence,
  VideoComposition,
  VideoShotCharacter,
  VideoWorkflow,
} from '@happyvertical/smrt-video';
import { Asset } from '@happyvertical/smrt-assets';

// Character = role being played (outfit, voice, branding)
const anchor = new Character({
  name: 'Bentley News Anchor',
  voiceProfileId: 'voice-123',
});
await anchor.save();

const seedImage = new Asset({
  name: 'Anchor Seed',
  sourceUri: 'file:///tmp/anchor-seed.png',
  mimeType: 'image/png',
});
await seedImage.save();
await anchor.addAsset(seedImage, 'seed-image');

// Performer = physical likeness for IP-Adapter face consistency
const performer = new Performer({ name: 'Alex', ipAdapterWeight: 0.85 });

// Scene = virtual background (image, video, or 360 panorama)
const studio = new Scene({
  name: 'News Studio',
  sourceType: 'image',
  projection: 'flat',
});

// Hierarchy: Composition -> Sequence -> Shot -> ShotCharacter
const composition = new VideoComposition({
  title: 'Evening News - March 2, 2026',
  fps: 30,
  width: 1920,
  height: 1080,
});
await composition.save();

const shot = new VideoShot({
  scriptText: 'Welcome to the evening news broadcast.',
  targetDuration: 30,
});
await shot.save();
// Estimated speech duration: scriptWordCount / 2.7 words per second
shot.estimatedDuration; // ~2.2 seconds
await shot.addAsset(seedImage, 'thumbnail');

// ComfyUI workflow with dynamic parameter injection
const workflow = new VideoWorkflow({
  name: 'Wan 2.6 + EchoMimic',
  workflowType: 'broadcast',
  workflowJson: comfyuiApiJson,
  nodeMapping: { seedImage: '1', audioFile: '5', outputVideo: '12' },
  requiredModels: ['wan_2.6_t2v_14b_fp8', 'echomimic_v2'],
});
await workflow.save();

// Inject runtime parameters into a deep-cloned workflow
const injected = workflow.injectParameters({
  seedImage: '/path/to/anchor.png',
  audioFile: '/path/to/tts.wav',
});

API

Models

Export	Description
Character	Virtual persona: seed image + voice profile + branding kit
Performer	Physical likeness for IP-Adapter FaceID consistency
Scene	Virtual background with viewpoints, lighting, and anchor points
VideoShot	Atomic generation unit (extends Content) with script and frame duration
VideoShotCharacter	Junction: shot-to-character with role (primary/secondary/background)
VideoSequence	Thematic section (extends Content) with transition type and ordering
VideoComposition	Top-level container (extends Content): fps, resolution, render status
VideoWorkflow	ComfyUI template with node mapping for parameter injection
CompositeJob	Tracks a multi-step video generation job

Collections: CharacterCollection, VideoCompositionCollection, VideoSequenceCollection, VideoShotCollection, VideoShotCharacterCollection.

Canonical owned asset joins:

• CharacterOwnedAsset -> character_assets
• PerformerOwnedAsset -> performer_assets
• SceneOwnedAsset -> scene_assets
• VideoShot, VideoSequence, and VideoComposition inherit Content asset helpers backed by content_assets

Legacy STI asset subclasses CharacterAsset, PerformerAsset, SceneAsset, VideoCompositionAsset, VideoSequenceAsset, and VideoShotAsset remain readable during migration, but new writes should go through the model helpers. Provision the new join tables with smrt db:migrate before relying on canonical writes. Reads still fall back to legacy fields / STI rows when the new tables are not present yet.

Key Types

Export	Description
CharacterStatus	pending, ready
BrandingConfig	Logo, colors, fonts, lower-thirds, ticker settings
WorkflowType	prebake, broadcast, lipsync, postprod, custom
NodeMapping	Maps semantic names (seedImage, audioFile, etc.) to ComfyUI node IDs
RenderStatus	draft, rendering, ready, failed
VideoShotStatus	draft, queued, processing, ready, failed, published
TransitionType	none, fade, slide, wipe
VideoMetadata	Duration, resolution, codec, fps, word timings
SceneSourceType	image, video, panorama_360, panorama_180
VideoShotCharacterRole	primary, secondary, background

Design Principle

Store frames, compute seconds. All duration fields use durationInFrames; convert with frames / fps. Speech duration estimated at 2.7 words/second (+/- 15% tolerance).

Asset Ownership

Character, Performer, and Scene expose getAssets(), getAssetByRole(), addAsset(), and removeAsset() on canonical noun-join tables. They continue to read legacy scalar slots and legacy STI asset rows during migration.

VideoShot, VideoSequence, and VideoComposition inherit the same helpers from Content, so their canonical ownership table is content_assets rather than a video-specific join table.

Dependencies

• @happyvertical/smrt-core -- ORM and code generation
• @happyvertical/smrt-assets -- base asset management
• @happyvertical/smrt-config -- configuration loading
• @happyvertical/smrt-content -- content models (VideoShot/Sequence/Composition extend Content)
• @happyvertical/smrt-profiles -- profile references for characters
• @happyvertical/smrt-tenancy -- multi-tenant scoping
• @happyvertical/smrt-voice -- voice profile references