Meta Launches AI Model for Creating Videos with Sound: Revolutionary Tech

Meta’s Movie Gen AI makes amazing movies of animals surfing and swimming.

Meta, the company that owns Facebook, said it had created a new AI model called Movie Gen that can make realistic-looking audio and video clips when asked to by users. The company says its tools can compete with those made by OpenAI and ElevenLabs, two of the best startups in the media creation field.

Meta showed examples of Movie Gen’s work that included videos of animals swimming and surfing, as well as videos that used real photos of people to show them doing things like painting on a board.

Meta said in a blog post that Movie Gen can also make background music and sound effects that work with the movies. The tool can also be used to edit videos that already exist.

Meta showed one of these videos it turned a man skating in a dry parking lot into one with a splashing puddle. Meta said that films made by Movie Gen can be up to 16 seconds long and sounds can be up to 45 seconds long. It shared information from blind tests that show the model does better than products from companies like Runway, OpenAI, ElevenLabs, and Kling.

This year, Hollywood has been trying to figure out how to use generative AI video technology. Microsoft backs OpenAI showed off how its product Sora could use text hints to make videos that looked like full-length movies.

Some entertainment industry technologists are excited about using these kinds of tools to speed up and improve the making of films, while others are wary of using systems that look like they were trained on copyright works without permission.

There are also worries among lawmakers about how deepfakes, which are fakes made by AI, are being used in elections around the world, such as in the US, Pakistan, India, and Indonesia.

Meta representatives said the company probably wouldn’t let developers use Movie Gen for free, like it did with its Llama line of large-language models, because the company thinks about the risks for each model on its own. They wouldn’t say anything about Meta’s review of Movie Gen in particular.

Instead, they said Meta was working directly with the entertainment industry and other content creators to find new ways to use Movie Gen. Meta would then add it to its own goods sometime next year.

This year, OpenAI has been meeting with Hollywood executives and agents to talk about future partnerships involving Sora. However, as of now, no deals have been reported to have come out of those talks. In May, actor Scarlett Johansson said that the company that made ChatGPT copied her voice for its robot without her permission, which made people more worried about how they did business.

Lions Gate Entertainment, which made “The Hunger Games” and “Twilight,” said in September that it would let the AI startup Runway use its collection of movies and TV shows to teach an AI model. It said that the company and its filmmakers could use the model to improve their work in exchange.