All it takes for the AI to generate realistic footage is a few lines of text prompt and the video will be generated in no time.
While the world is gradually coming to terms with the fact that Artificial Intelligence is going to be an inevitable part of our lives, technology experts are coming up with new applications. The outcome of some of these applications is quite hard to process for the common man. Recently, the world's leading AI research organization, OpenAI has released a new AI model called "Sora" that can develop lifelike and imaginative scenes based on text instructions. The organization has shared on X (@OpenAI) some jaw-dropping videos that Sora had created and it has caused a stir on the internet.
Introducing the new AI model to the world, Open AI posted, "Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions." A 17-second video of a snowy Tokyo city scenario amidst the Sakura blooms took the internet by surprise and over 48 million people had viewed it. The prompt that generated this realistic video was, "Beautiful, snowy Tokyo city is bustling. The camera moves through the bustling city street, following several people enjoying the beautiful snowy weather and shopping at nearby stalls. Gorgeous Sakura petals are flying through the wind along with snowflakes."
Introducing Sora, our text-to-video model.
— OpenAI (@OpenAI) February 15, 2024
Sora can create videos of up to 60 seconds featuring highly detailed scenes, complex camera motion, and multiple characters with vibrant emotions. https://t.co/7j2JN27M3W
Prompt: “Beautiful, snowy… pic.twitter.com/ruTEWn87vf
Open AI shared a few more AI-generated videos in the same thread. Each one of them was unique and seemed to accurately depict what was mentioned in the prompts. The video of giant woolly mammoths walking amidst a backdrop of snowy mountains would make one think that those creatures weren't extinct anymore. Prompts like, "their long wooly fur lightly blows in the wind as they walk," and "the low camera view is stunning capturing the large furry mammal with beautiful photography," worked out pretty effectively. Could this be a new hope or a bane to the future of animated movies? No one knows yet.
Prompt: “Several giant wooly mammoths approach treading through a snowy meadow, their long wooly fur lightly blows in the wind as they walk, snow covered trees and dramatic snow capped mountains in the distance, mid afternoon light with wispy clouds and a sun high in the distance… pic.twitter.com/Um5CWI18nS
— OpenAI (@OpenAI) February 15, 2024
Sora was able to generate a 17-second movie trailer featuring, "the adventures of the 30-year-old spaceman wearing a red wool knitted motorcycle helmet, blue sky, salt desert, cinematic style, shot on 35mm film." The people generated by this AI could be easily mistaken for real actors and the attention to detail is what makes this unbelievable. In yet another cinematic video, Sora brought to life a scene from a Tokyo street and the AI even managed to bring out the city's vibe that was expected in the prompt. "She walks confidently and casually," read the prompt and that's what the woman in the video does exactly.
Prompt: “A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. she wears a black leather jacket, a long red dress, and black boots, and carries a black purse. she wears sunglasses and red lipstick. she walks confidently and casually.… pic.twitter.com/cjIdgYFaWq
— OpenAI (@OpenAI) February 15, 2024
As much as this innovation amazed people, it also raised a lot of concerns about trust. "We used to say 'I wouldn't believe it if I saw it with my own eyes,' but now it's true. I wouldn't believe it if I saw it with my own eyes," commented @pastperfecttens. "People, what you are doing is absolutely amazing, but you are killing massive jobs doing this," wrote @FranchScatman.
within a couple years, we will be watching regular movies fully made by AI
— Danky (@0xdanky) February 15, 2024
Apart from deep fake concerns. All the stolen artwork. No compensation or royalties for artists. Who will be getting rich from this I wonder?
— tiana loret (@tia_loret) February 16, 2024
The creators have, however, assured in the thread that they would watch out for safety concerns in this model which is still in development. "We’ll be taking several important safety steps ahead of making Sora available in OpenAI’s products. We are working with red teamers — domain experts in areas like misinformation, hateful content, and bias — who are adversarially testing the model," they wrote. "This is going to be another really cool way to improve storytelling capabilities," Bill Peebles, a researcher on the Sora project told Wired and added, "We’re going to be very careful about all the safety implications for this."