AI Reference to Video API

It generates a new video from a text prompt and reference assets such as images, and on some models also videos or audio. The references define subjects, appearance or style, while the prompt describes what happens. The output is a newly generated scene, not an edit of an existing clip.

Image to video animates a specific image, usually as the first frame (and sometimes a last frame), so the clip starts from that picture. Reference to video uses images as guidance for identity, objects or style and lets the model build a new composition. Choose image to video when the opening frame must match exactly.

Video to video takes existing footage as the source to edit, extend or transform. In reference to video, a reference video only guides elements such as motion, camera or style for a newly generated clip. If you need to change an existing clip while keeping its content, compare video to video endpoints instead.

Limits vary by model, from a few images to dozens on multimodal endpoints, and some models lower the image limit when a reference video is included. Check the endpoint documentation for counts, formats, file size and aspect ratio requirements before building your upload flow.

Some models accept multimodal references: images for appearance, videos for motion or camera, and audio for sound or rhythm. Others accept reference images only. Some endpoints require at least one visual reference when audio is supplied, and limit reference video duration and resolution.

Usually not in the same request. On many models, frame image fields and reference fields are mutually exclusive, and the request is rejected if both are sent. Decide whether the task is frame-driven (image to video) or reference-driven before submitting.

Use clear, well-lit references that show the subject without heavy occlusion, and keep the set focused on what must stay the same. Refer to the subjects explicitly in the prompt, avoid conflicting references and test a short clip first. Consistency still varies by model, so compare outputs on your own assets.

They can. Some models restrict the maximum resolution, default to an adaptive aspect ratio or fix the duration when references are used. Read the parameter rules for the chosen endpoint so the request does not fail validation.

Output duration and resolution are the main factors. Depending on the model, reference videos may be billed by their duration, extra reference images may add cost, or reference tokens beyond a free allowance may be charged. Check the current pricing description for the model you choose.

Send the model, prompt and reference URLs to the documented PoYo generation endpoint, save the returned task_id and use status queries or a webhook to fetch the result. Make sure reference URLs are publicly reachable, and keep the reference set with the job record for comparison.