Refining your vision: A guide to AI image editing

Key Concepts

• Gemini Image (Nano Banana): A generative AI model on Vertex AI for image editing, leveraging Gemini's reasoning capabilities for contextual understanding and conversational interaction.
• Conversational Editing: The ability to edit images using natural language prompts, allowing for iterative changes and a dialogue-based workflow.
• Inpainting Removal: The process of removing an object from an image and intelligently filling the resulting space based on the surrounding image context.
• Style Transfer: Generating a new image that adopts the style, aesthetic, color palette, and material textures of a reference image.
• Consistency Maintenance: Preserving the appearance of specific objects or characters within an image while modifying other elements.
• Reference to Image (Multi-Image Context): Using multiple input images to inform a single output, allowing the model to fuse elements and understand spatial relationships.
• VO (Video Generation Model): Google's model for generating video from static images, enabling animation and motion.
• Image to Video Capability: A feature of VO that transforms a static image into a dynamic video sequence.
• Vertex AI Studio: A platform on Google Cloud for accessing and utilizing AI models, including Gemini Image.
• SDK (Software Development Kit): Tools and libraries for programmatic interaction with AI models, such as the generate content method for image fusion and removal.

Image Editing with Gemini Image (Nano Banana) on Vertex AI

This video introduces the capabilities of Gemini Image, also referred to as Nano Banana, a powerful AI model available on Google Cloud's Vertex AI for editing existing images. Unlike generating new images, Nano Banana focuses on refining and transforming existing visual assets through conversational prompts.

Conversational Editing: Precise, Iterative Changes

The core innovation highlighted is conversational editing. This feature allows users to make precise edits to an image using natural language, eliminating the need for manual masking.

• Example: A user can upload a photo of a runner in a gray jacket and instruct Nano Banana to "change the runner's jacket color to a deep navy blue." The model processes this prompt, applies the color change while preserving the rest of the image, and allows for further iterative edits.
• Further Iteration: Following the color change, a user can then prompt, "slightly blur the background," demonstrating the ability to build upon previous edits within a single conversational flow.

Object Removal with Text Prompts

Nano Banana also excels at removing objects from images using only text instructions.

• Process:
  1. Select a starting image (e.g., a fire hydrant on a lawn).
  2. Upload the image to Vertex AI Studio.
  3. Provide a text prompt such as, "remove the red fire hydrant and fill the space in naturally."
• Mechanism: Gemini intelligently handles the inpainting removal, using the surrounding image context to seamlessly fill the void left by the removed object.

Style Transfer for Complex Designs

For intricate design tasks, style transfer is a key capability. This involves uploading a reference image and generating a new image that mirrors its style and aesthetic.

• Example:
  1. Start with an image of a living room described as "mid-century modern meets minimalist comfort with a warm neutral palette."
  2. Upload this image and prompt the model to "Generate a brand new office space that uses the exact color palette, material textures, and minimalist style of the attached living room photo."
• Outcome: The model generates a new office space that adheres to the stylistic elements of the original living room.

Maintaining Consistency of Unmodified Objects

A significant benefit of AI image editing is the ability to maintain the consistency of elements that should not be altered, while modifying other aspects. Nano Banana is particularly effective for:

• Character Consistency: Ensuring a character's appearance remains the same across different scenes or edits.
• Product Consistency: Preserving the exact look of a product, such as a coffee mug, even when the context changes.
• Example: A product shot of a person drinking from a coffee mug can be edited by asking Gemini to "place the person drinking out of the cup on a beach, all while maintaining the exact look of the original cup and model."

Advanced Editing: Multi-Image Context (Reference to Image)

A more advanced capability involves using multiple images as context for a single output. This is where the term reference to image comes into play.

• Process: Upload two or more images, and the model fuses them into one cohesive composition.
• Example: Virtual Staging:
  1. Upload an image of an empty room.
  2. Upload an image of a new blue velvet sofa.
  3. Prompt: "Take the sofa from the reference image and place it realistically in the empty room. Adjust the sofa's lighting and shadows to match the lighting from the window."
• Result: Gemini generates a single, cohesive image, understanding the spatial context and realistically integrating the sofa with appropriate lighting and shadows.

Programmatic Editing with SDK

For developers, programmatic editing is possible using the SDK.

• Method: The generate content method is used for fusion removal or consistency tasks.
• Implementation: All input image bytes are included in the contents list alongside the text prompt, providing the model with necessary visual data and instructions.

Integrating Image Editing with Video Generation (VO)

The true power of this technology is realized when combining image editing capabilities with VO, Google's video generation model.

• Integrated Workflow Example:
  1. Image Editing: Use Nano Banana's conversational flow to create a perfect static asset (e.g., the runner with the navy blue jacket).
  2. Video Generation: Take the edited static image and use VO's image to video capability as the starting frame.
  3. Motion Instruction: The prompt to VO becomes a motion instruction, such as: "Animate this runner. The camera slowly tracks her. Adds subtle mist and lens flare."
• Benefit: This integrated workflow of editing a static image and then animating it with VO unlocks the creation of powerful assets across various industries.

Conclusion and Next Steps

The video concludes by emphasizing the broad applicability of these AI image editing and video generation capabilities. For those ready to start building, links to documentation, code samples, and getting started guides are provided in the video description.