Generative Extend in Premiere Pro: Adobe’s AI Tool That Could Change Video Editing

Adobe Premiere Pro’s new AI tool could save video editors hours of time

Adobe has released Photoshop 25.9, the latest public beta of its image-editing software, adding a range of generative AI capabilities powered by its new Firefly Image 3 AI model. One of the first AI tools released was generative fill in Photoshop, which lets creators fill specific shapes or areas with AI-generated imagery. Now, generative fill is one of the most popular Photoshop tools, on par with the crop tool. Of the 11 billion images created using Adobe’s AI model Firefly, 7 billion of them were generated in Photoshop. Put another way, an average of 23 million images a day are made using generative fill, Nielson said. Part of the appeal of Adobe’s updates is that they are legitimate use cases for generative AI for professionals.

The possibility of “losing a generation of artists,” as she put it, is worrisome. There’s no shortage of experts arguing about whether AI is capable of producing art, but artists have already lost jobs in favor of AI, especially in entry-level or freelance positions. Job experts predict that AI is likely to reduce the number of overall job opportunities as it gets better at automating more menial tasks.

How Generative AI is unlocking creativity – the Adobe Blog

How Generative AI is unlocking creativity.

Posted: Thu, 17 Oct 2024 07:00:00 GMT [source]

Lightroom’s generative remove has better object detection and selection to remove photobombers and other intrusive elements. One is the text and image to video generation that Adobe previewed last month, accessible in the Firefly web app at firefly.adobe.com. This enables users to create five-second, 720p-resolution videos from natural-language text prompts. It’s also possible to generate video using still images as a prompt, meaning a photograph or illustration could be used to create b-roll footage. Adobe’s Firefly cloud service, which provides access to AI-based design tools, is also receiving new video editing capabilities. One of the additions is a feature that generates five-second clips based on text prompts.

Adobe’s Firefly ‘Bulk Create’ lets users edit thousands of images at once

We actively engage with policymakers and industry groups to help shape policy that balances innovation with ethical considerations. Our discussions with policymakers focus on our approach to AI and the importance of developing technology to enhance human experiences. Regulators seek practical solutions to address current challenges and by presenting frameworks like our AI Ethics principles—developed collaboratively and applied consistently in our AI-powered features—we foster more productive discussions.

It is fascinating how Adobe discusses and frames generative AI tools compared to its competitors. Unlike companies like OpenAI and Stability AI, Adobe has been serving creative professionals for decades — Adobe didn’t just pop up when the AI door opened. Adobe’s 30-plus years of creating tools for visual artists means its core audience is not universally champing at the bit for more generative AI technology; many are concerned about how AI may harm their business and the art space at large. Adobe pledges to attach Content Credentials to assets produced within its applications so users can see how it was made and plans to apply its approaches to the planned integration of third-party AI models. As you can see when comparing the sets of images in the figures above and below… you can have great influence over your set of generated images through this control.

Expand videos that are too short without reshooting

While Generative Extend might give them the footage they need, other creatives may be less enthused. It may mean that reshoots are no longer required, taking days of work (and income) away from the cast and crew. Generative Extend is a Premiere Pro feature that Adobe previewed earlier this year. It enables editors to add generated footage and audio to the start or end of a clip. Adobe says the tool can also correct eyelines and actions that change unexpectedly in the middle of a shot. Generative AI is already reshaping digital experiences in India, particularly in ecommerce and travel.

This technology also enables the extension of video clips and the smoothing of transitions, with integration into Adobe’s video editing software, Premiere Pro. Adobe has expanded its Firefly family of creative generative AI models to video, in addition to new breakthroughs in its Image, Vector and Design models. The Firefly Video Model, now in limited public beta, is the first publicly available video model designed to be “commercially safe,” Adobe said.

The company says it’s committed to taking a creator-friendly approach and developing AI following the company’s AI Ethics with principles of accountability, responsibility and transparency. This includes respecting creators’ rights and never training the development of AI by using customer content. The new tools will also help across all design workflows whether that’s creating variations of advertising and marketing graphics or mocking up digital drawings and illustrations. For example, it’s now easier to add patterns to fashion silhouettes for mood boards. And since Adobe Firefly’s features are integrated into the products you already know and likely use so often, you won’t have to waste time navigating new software. Instead, users will need to either click on their profile picture on the Firefly website or do the same inside Adobe’s Creative Cloud desktop or web app.

This is markedly different from most AI art programs that are targeted at amateurs and non-artists — professional photographers and illustrators can create better images than an AI image generator, after all. Making it quicker to fix those kinds of errors is the goal of Adobe’s AI, Stephen Nielson, senior director of product manager for Photoshop, told me. Photoshop also has new and intuitive features to accelerate core creative workflows and streamline repetitive tasks by using the Selection Brush Tool, Adjustment Brush Tool and enhancements to the Type Tool and Contextual Taskbar.

• For those who want it, it’s available in all versions of Adobe Lightroom beginning today as an “early access” feature.
• Several of Photoshop’s existing AI tools are designed for tasks like eliminating power lines, garbage cans, and other distractions from the background of a photo.
• When it comes to generative artificial intelligence (AI), one company that has been at the forefront on the software side is Adobe (ADBE -0.43%).
• Retype is another nifty tool that converts static text in images into editable text.

This is great for taking pre-made designs and color schemes and applying your brand to them, without spending hours recoloring or changing fonts and other elements. Photoshop Beta’s Generative Workspace allows your generated images to have a new home. Previously, when generating images, you had to manually click to open them and save them each as a file or an artboard—but the Generative Workspace allows you to keep track of all your generated images across the Adobe suite. “AI tools can either be used for evil or to steal stuff, but it can also be used for good, to make your process a lot more efficient,” said Acevedo.

Adobe also hopes that by building this AI for professionals, it won’t raise the typical red flags that other AI programs do. If it’s integrated well, creators might be more inclined to take advantage of it, said Alexandru Costin, vice president of generative AI at Adobe. Another feature, Lens Blur, allows you to blur any part of a photo to create more professional-looking cityscapes, portraits, or street photography. If you have a photo you love but want to swap the background, the latest Photoshop update allows you to generate a replacement background that matches the lighting, shadows, and perspective of the subject in the forefront.

Well, that’s possible to change, too, and like style variations, users change the composition with a descriptive text prompt. I saw this new direction for myself at this year’s Adobe MAX, where new announcements focused on AI as tools rather than gimmicks. New tools like Project Turntable, that enables you to easily rotate 2D vector art in 3D by generating the missing data to fill in the image – a 2D horse now has four legs as its turned.

Google ups Workspace price, makes Gemini AI features available for free

Adobe said it only trains the video model on stock footage and public domain data that it has rights to use for training its AI models. Adobe has also released more info about its own promises for “responsible innovation” for Firefly and this new generative AI video model. Adobe promises that its Firefly generative AI models are trained only on licensed content, such as Adobe Stock and public domain content. It also gets new intuitive features like the Generate Image feature, powered by the new Firefly Image 3 Model. Additionally, the Enhance Detail feature for Generative Fill has been improved to provide greater sharpness and detail for large images. Moreover,the new Selection Brush tool simplifies the process of selecting specific objects for editing.

In this article, we’ll be exploring some of the more detailed features of Firefly in general. While we will be doing so from the perspective of the text-to-image module, much of what we cover will be applicable to other modules and procedures as well. The Substance 3D Collection is revolutionizing the ideation stage of 3D creation with powerful generative AI features in Substance 3D Sampler and Stager.

What to do if Generative Fill is grayed out in Adobe Photoshop AI

One of the biggest announcements for videographers during Adobe Max 2024 is the ability to expand a clip that’s too short. Dubbed generative extend, the tool uses AI to add both video and sound to the end of an existing clip. In demonstrations of the tool, Adobe showed off generated video that looked very similar to the original clip. I would prefer to continue paying Adobe USD 9.99 monthly, just as I have been doing for the most part of my professional career. I definitely don’t want to have to pay over 50% more at USD 14.99 just to continue paying monthly instead of an upfront annual fee. What could make a lot of us photographers happy is if Adobe continued to allow us to keep this plan at 9.99 a month and exclude all the generative AI features they claim to so generously be adding for our benefit.

From playground to production: How to jump-start your content transformation with generative AI – the Adobe Blog

From playground to production: How to jump-start your content transformation with generative AI.

Posted: Thu, 20 Jun 2024 07:00:00 GMT [source]

Each step in the creative process can be enhanced with generative AI in Adobe Photoshop. Similarly, Adobe’s newly-announced Generative Remove tool in Lightroom — a tool that is classified as “Early Access beta” — also incurs a Generative Credit per use. These usage number exist now because it says it wants to be transparent about usage so that when it does start enforcing these limits, users can see how much they’ve used historically. It’s not clear when Adobe will actually start to enforce limits, such as app slowdowns, if Credits are expended. Adobe tells PetaPixel that for most of its plans, it has not started enforcement when users hit a monthly limit even if it is actively tracking use. The company recorded $504 million in new digital media annualized recurring revenue (ARR), ending the quarter with digital media ARR of $16.76 billion.

• The concern for creatives is seeing their work potentially lumped in with those tasks.
• Adobe could improve the user experience dramatically by simply including the reason a generation gets flagged as a guideline violation.
• Note that Content Credentials are applied in this case just the same as they are when downloading an image.

Adobe also announced its plans to bring third-party generative AI models directly into its applications, including Premiere Pro, although the timeline is murky for now. Clicking the Favorite control will add the generated image to your Firefly Favorites so that you can return to the generated set of images for further manipulation or to download later on. Choose one of the generated images to work with and hover your mouse across the image to reveal a set of controls. After Effects now also has an RTX GPU-powered Advanced 3D Renderer that accelerates the processing-intensive and time-consuming task of applying HDRI lighting — lowering creative barriers to entry while improving content realism. Rendering can be done 30% faster on a GeForce RTX 4090 GPU over the previous generation. The latest After Effects release features an expanded range of 3D tools that enable creators to embed 3D animations, cast ultra-realistic shadows on 2D objects and isolate effects in 3D space.

“Generative Extend” is among the most interesting generative AI tools Adobe plans to bring to Premiere Pro. It promises to seamlessly add frames in clips to make them longer, allowing editors to create smoother transitions. Adobe says this “breakthrough technology” will enable editors to create extra media for fine-tuning edits, hold a shot for an extra beat, and better cover transitions.

Firefly applies metadata to any generated image in the form of content credentials and the image download process begins. One reason is to get general user feedback to improve the experience of using the product… and the other is to influence the generative models so that users receive the output that is expected. There are also new Firefly-powered features in Substance 3D Viewer, like Text to 3D and 3D Model to Image, that combine text prompts and 3D objects to give artists more control when generating new scenes and variations. Just a few weeks ago, the company introduced Magic Fixup, a technique that applies more sophisticated image editing capabilities than normal image editors after being trained on video instead of still images. Another new tool, Generative Extend, enables editors to lengthen existing clips, smoothing transitions and adjusting timing to align perfectly with audio cues. Moreover, the AI can address video timeline gaps, helping to resolve continuity issues in editing by contextually connecting two clips within the same timeline—a feature that distinguishes Adobe from its competitors.

Adobe is also investing in better ways to help differentiate content created by AI, which is one of the biggest issues with AI-generated content. Recently Adobe launched a new Content Authenticity app for artists to create content credentials, a kind of digital signature that lets artists invisibly sign their work and disclose any AI used. “I think Adobe has done such a great job of integrating new tools to make the process easier,” said Angel Acevedo, graphic designer and director of the apparel company God is a designer. “We saw stuff that’s gonna streamline the whole process and make you a little bit more efficient and productive.”

Adobe does not seem to have any plans to put warnings or notifications in its apps to alert users when they are running low on Credits either, even when the company does eventually enforce these limits. This biggest issue, though, was the company’s projection of about $550 million in new digital media ARR for the quarter. In Q4 of last year, the company generated $569 million in new digital media ARR, so this would be a deceleration and could lead to lower revenue growth in the future. Adobe said the lower new ARR forecast was due to timing issues, such as Cyber Monday falling into the next quarter this fiscal year.