Meta has launched Movie Gen, a new artificial intelligence (AI) tool specially-created for filmmakers and content creators to expand their creativity.
This generative AI platform allows users to create custom videos and audio simply by typing in text prompts, Meta says about Movie Gen.
Users can generate high-definition videos from text prompts using a powerful 30 billion parameter transformer model, producing clips of up to 16 seconds at a rate of 16 frames per second.
Movie Gen, which marks Meta’s third wave of rolling out AI tools, combines Meta’s “Make-A-Scene series of models that enabled the creation of image, audio, video, and 3D animation” with Meta’s second wave of work with “Llama Image foundation models, which enabled higher quality generation of images and video, as well as image editing.”
According to Meta’s research, Movie Gen stands out in the industry, outperforming similar models in generating high-quality video and audio content based on human evaluations. The tool offers a range of capabilities, including video generation, personalized video creation, precise video editing, and audio generation.
What can Movie Gen do?
Here are some of its key features:
1. Users can generate high-definition videos from text prompts using a powerful 30 billion parameter transformer model, producing clips of up to 16 seconds at a rate of 16 frames per second. “We find that these models can reason about object motion, subject-object interactions, and camera motion, and they can learn plausible motions for a wide variety of concepts—making them state-of-the-art models in their category,” Meta says.
2. The platform also enables personalised video creation by combining a user’s image with a text prompt, resulting in content that captures the individual’s identity along with rich visual details.
3. For those looking to edit their videos, Movie Gen’s “editing variant of the same foundation model takes both video and text prompt as input, executing tasks with precision to generate the desired output,” Meta says. “It combines video generation with advanced image editing, performing localized edits like adding, removing, or replacing elements, and global changes such as background or style modifications.”
4. Additionally, the audio generation feature can create high-quality soundscapes, including ambient sounds and music, synchronised with the video.
Meta’s Movie Gen competes with notable players like OpenAI’s DALL-E and Runway’s Gen-2. While DALL-E has gained attention for its image generation, it is now expanding into video. Runway offers editing tools and real-time video generation.
“Generative AI isn’t a replacement for the work of artists and animators,” Meta adds. “We’re sharing this research because we believe in the power of this technology to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them.”
Looking ahead, Meta says that there’s still work to be done to optimise the models, particularly regarding inference time and overall quality. The tech company says it plans to collaborate closely with filmmakers and creators to integrate feedback to refine the tool, ensuring it meets the needs of its users.
Home
