Audio to Video API: Build an Async Music Video Workflow

An audio to video API should be treated as an asynchronous product workflow, not a single blocking render call. Your backend prepares reachable audio and image URLs, creates a music-video task, stores the task id, watches status through polling or webhooks, then saves the hosted MP4 URL when the task succeeds.

• Use POST /v1/files when the user's audio, image, or subtitle file is local.
• Create the render with POST /v1/music-video/tasks.
• Poll GET /v1/tasks/{task_id} every 5-10 seconds with jitter; add /v1/webhooks when your backend needs completion events.
• Read the final video from output.media[].url after status becomes succeeded.
• Keep provider-specific details behind BeatAPI's task contract so your product owns one stable integration.

Quick answer

The reliable implementation path is:

1. Validate or upload the input audio and visual references as public HTTPS URLs.
2. Send those URLs to POST /v1/music-video/tasks with prompt, aspect ratio, resolution, and optional subtitle or lip-sync settings.
3. Persist the returned BeatAPI task id in your database.
4. Poll GET /v1/tasks/{task_id} until the task reaches succeeded or failed.
5. Register webhooks for backend completion events when polling alone is not enough.
6. Store output.media[].url, usage, and request_id for delivery and support.

This shape is safer than holding an HTTP request open while video generation runs. Audio-to-video jobs can spend time in queueing, generation, storyboard, editing, and composition stages. The product should remain observable after the browser tab closes.

Workflow shape

An audio-to-video API has to own more than generation. It needs a contract for files, state, output, and recovery.

Prepare inputs

BeatAPI music-video tasks expect one audio URL and one to seven image URLs. They should be public HTTPS URLs reachable from the public internet. If your user uploads files in your app, send them through POST /v1/files first and pass the returned URLs into the task.

Use these preflight checks before the API call:

• Audio: public HTTPS mp3, wav, aac, or m4a.
• Audio duration: 10-180 seconds.
• File size: 50 MB or smaller for verified uploaded assets.
• Images: public HTTPS png, jpg, jpeg, or webp.
• Image count: 1-7.
• Image aspect ratio: roughly 1:4 through 4:1.
• Prompt: optional, up to 3000 characters.
• srt_url: optional, but it should point to an .srt file when subtitles are enabled.
• duration: a fallback when audio duration cannot be detected, not a way to override detected audio length.

Early validation gives the user a fast fix path and prevents expensive failed jobs. BeatAPI still validates URL shape, file metadata, text length, enum values, and audio duration at task creation.

Create the task

Start with a minimal request, then add controls only when your product needs them.

export BEATAPI_API_KEY="sk_your_key"

curl https://api.beatapi.io/v1/music-video/tasks \
  -H "Authorization: Bearer $BEATAPI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "audio_url": "https://media.beatapi.io/samples/neon-singer-preview.mp3",
    "images": ["https://media.beatapi.io/samples/neon-singer.png"],
    "prompt": "Neon rooftop performance with cinematic light trails, slow dolly movement, energetic chorus cuts.",
    "language": "en",
    "aspect_ratio": "9:16",
    "resolution": "720p",
    "quality": "standard",
    "lip_sync": false,
    "add_subtitle": false,
    "compose_mode": "auto"
  }'

Store the returned task id immediately. Your database should track the BeatAPI task id, user id, internal job id, current status, request id, and final output URL when it exists. Do not make a provider job id the only durable reference in your product.

Poll and webhooks

BeatAPI tasks can move through queued, processing, storyboard_ready, requires_action, editing, composing, succeeded, and failed.

Poll the task endpoint every 5-10 seconds with jitter:

curl https://api.beatapi.io/v1/tasks/task_8K2qA \
  -H "Authorization: Bearer $BEATAPI_API_KEY"

A successful task exposes the final MP4 in output.media:

{
  "data": {
    "id": "task_8K2qA",
    "object": "task",
    "workflow": "music-video",
    "status": "succeeded",
    "output": {
      "media": [
        {
          "type": "video",
          "url": "https://media.beatapi.io/outputs/task_8K2qA/0.mp4",
          "mime_type": "video/mp4"
        }
      ]
    },
    "usage": {
      "credits_reserved": 150,
      "credits_charged": 150,
      "credits_settled": 150,
      "credits_refunded": 0
    },
    "request_id": "req_abc123"
  }
}

When you add webhooks, treat task.succeeded and task.failed as backend update events. Verify signatures with BeatAPI webhook headers, then fetch the task again if your handler needs a fresh source of truth.

Worked example

Imagine a creator app where musicians upload a 30-second preview and one artist image. The product goal is a vertical social clip.

Implementation shape:

1. Browser uploads the MP3 and image to your backend.
2. Backend uploads each local file through POST /v1/files.
3. Backend creates a BeatAPI music-video task with the returned URLs.
4. Backend stores { user_id, internal_job_id, beatapi_task_id, status, request_id }.
5. Frontend polls your backend, not BeatAPI directly.
6. Backend polls BeatAPI or receives a webhook.
7. When the task succeeds, backend stores output.media[0].url and exposes the MP4 to the user.
8. When the task fails, backend shows a useful status with error_code, error_message, and usage or refund evidence.

Reusable task payload:

{
  "audio_url": "https://media.beatapi.io/inputs/file_song_preview.mp3",
  "images": ["https://media.beatapi.io/inputs/file_artist_portrait.png"],
  "prompt": "Create a vertical artist promo video for a synth-pop chorus. Use neon reflections, close-up performance energy, quick cuts on beat, and no visible text.",
  "language": "en",
  "aspect_ratio": "9:16",
  "resolution": "720p",
  "quality": "standard",
  "lip_sync": false,
  "add_subtitle": false,
  "compose_mode": "auto"
}

For a lyric-video product, keep the same audio-to-video API flow and change only the workflow controls:

{
  "audio_url": "https://media.beatapi.io/inputs/file_song_preview.mp3",
  "images": ["https://media.beatapi.io/inputs/file_cover_art.png"],
  "prompt": "Create a clean lyric visualizer with soft camera movement, rhythmic background motion, and safe empty space for subtitles.",
  "language": "en",
  "aspect_ratio": "16:9",
  "resolution": "1080p",
  "quality": "standard",
  "add_subtitle": true,
  "subtitle_color": "#FFFFFF",
  "srt_url": "https://media.beatapi.io/inputs/file_lyrics.srt",
  "compose_mode": "auto"
}

Mistakes and fixes

Production checklist

Before exposing an audio-to-video API workflow to customers, check:

• API keys stay server-side.
• Local audio, images, and SRT files are uploaded before task creation.
• Asset URLs are public HTTPS URLs.
• Audio duration and file size are validated before task creation.
• Your database stores the BeatAPI task id and internal job id together.
• Frontend polling goes through your backend.
• Backend polling has jitter and stops on terminal states.
• Webhook handlers verify signatures and are idempotent.
• Final MP4 URLs, request ids, usage, and errors are stored for support.
• The UI gives a clear retry path for failed input validation and terminal task failures.

FAQ

What is an audio to video API?

An audio to video API accepts an audio asset plus visual direction and returns a generated video through an asynchronous task. In BeatAPI, the music-video workflow uses audio_url, images, prompt controls, task status, optional webhooks, and hosted MP4 output.

Do I need to upload files before creating a task?

Only when the user's files are local or not publicly reachable. BeatAPI tasks need public HTTPS input URLs. Use POST /v1/files to turn local images, audio, or subtitles into task-ready URLs.

Should I poll or use webhooks?

Use polling first because it is simple and recoverable. Add webhooks when your backend needs completion events without waiting for the next poll. Even with webhooks, GET /v1/tasks/{task_id} remains the source of truth.

Can I create lyric videos?

Yes, when your workflow supplies audio, visual references, and subtitle controls. Use add_subtitle, subtitle_color, and srt_url for subtitle-ready outputs.

Can I use lip sync?

Yes, when the job needs a performance clip and you provide the required reference controls. Enable lip_sync and provide lip_ref_url when the product scenario calls for it.

What should I store after task creation?

Store your internal job id, BeatAPI task id, status, request id, usage fields, final output.media URL, and any terminal error fields. That is the minimum support and retry context.

What happens if a task fails?

The task endpoint returns a terminal failed status with error information when no usable output is produced. Your product should keep the task record and show a retry or support path based on the error.

Where should the next integration start?

Start with the Music Video API page if your product needs audio-to-video generation with task state, webhooks, credits, and hosted MP4 output.