Forward-looking: Advances in AI are attempting to remove many of the prohibitive financial and technical challenges that can make content creation more challenging outside of big cloud data centers. AI research teams are developing and refining tools allowing even the most novice creators to generate content based on nothing more than a description of what they want to see. This week, Meta published a blog post highlighting Movie Gen, the company’s upcoming text-to-content audio and video generator.
Movie Gen was introduced this week by Mark Zuckerberg in an Instagram post reminding us that we shouldn’t skip leg day. The preview comes several months after OpenAI’s introduction of its Sora AI text-to-video solution.
A longer and more informative announcement was posted on Meta’s AI research team blog, providing additional information about Movie Gen’s capabilities. The new tool will support text-based video and audio generation and editing capabilities.
The post includes several videos demonstrating Movie Gen’s ability to create and modify content easily based on a user’s text inputs, producing video content up to 16 seconds in length at a rate of 16 frames per second. The tool can also generate highly personalized videos using a single photo in combination with a descriptive text prompt.
In addition to video editing, Meta’s AI solution can also create ambient sound, background music, and sound effects to accompany generated or pre-created videos. Much like its video generation capabilities, Movie Gen’s audio generation is driven by user-defined text inputs.
The post showcases several examples of its audio capabilities, including an ATV riding through sand dunes, a skateboarder rolling through a skate park, and penguins swimming through an AI-generated body of water filled with large floating spheres.
Despite Zuckerberg’s post claiming the new feature could be available for Instagram users next year, a Threads post by Meta’s Chief Product Officer, Chris Cox, clarified the current status of Movie Gen’s current state. Cox explained that Meta is incorporating feedback from filmmakers and video creators to prioritize features like enhanced editing capabilities and the ability to generate videos using a specific character or image.
While the preview appears to show that Movie Gen already offers these features, Cox mentioned that the team isn’t ready to release them as a product just yet. According to him, “…it’s still expensive, and generation time is too long – but we wanted to share where we are since the results are getting quite impressive.”
