פשטו תמונות מורכבות: כוחו של AI חדש

Okay. Hi everyone. I'm Adi. I'm excited to be here. I'm going to present our work, Progressive Photovallistic Simplification. I would like to start by thanking my collaborators. Thank you, collaborators Dana Behrman, Eric Shamir and Yadid Hossen. So we present Progressive Photorealistic simplification, a new method to simplify complex and busy scenes while maintaining perfect photorealism. The concepts of obstruction and perception have a long history in both art theory and vision science. A central insight is that we don't just look passively. We actively organize a scene and naturally prioritize some elements over others. From this perspective, abstraction is a way of simplifying what we see so that the underlying important structure stands out, like we can see in this famous Bull series by Picasso in computer graphics. These classical concepts have been explored in the context of non photorealistic rendering using classical image processing methods and more recently, generative models to produce sketch based or style representations. But notice what all of these have in common. They change the medium. The photo becomes non photorealistic and the realism is gone. Our work asks a different can we simplify an image while keeping it completely photorealistic? Instead of changing the style, we treat abstraction as a gradual removal of less important elements from the scene while keeping both its realism and semantic currents. In our approach, scene elements are removed step by step, creating photorealistic abstraction trajectory. Each step simplifies the image until it becomes completely abstract. This raises a practical where do you stop? You can just immediately declutter the scene. You can go far enough and remove elements from the main subject, or you can push all the way to a minimalist, fully abstract image. Our main goal is to figure out what to remove and when. We want every step to look like a realistic photograph that conveys the original content. Given an initial image composed of n elements, we want to find the best order to remove them from least important to most important. But we face two main challenges. The first challenge is subjectivity of ordering. There is no math formula to rank what is most important. The relative importance of objects is often ambiguous. For example, in this scene, the pole and the end might be equally distracting. There is no strict right or wrong order to remove them. The second challenge is complexity. We can think of this problem as searching for the best path in a tree structure where each edge represents the removal of a single element. The number of possible paths grows exponentially in each node. We must choose which element to remove from the remaining elements in the scene. So in total, there are n factorial possible permutations of the element. To address this, we introduce a simple taxonomy that aligns with human preferences. We group objects into four levels of importance. Level one is distracting elements, things like trash or stray cobbles that actively hurt the quality and composition. Level 2 is secondary elements, contextual objects that support a scene but are not essential, like lamp or train tracks. Level three is primary subject, the core of the image. Remove this and you change the whole meaning of the photo. And level four is the background, the environment itself, which we simplify. Last, to verify that this taxonomy matches human preferences, we asked professional photographers to annotate a set of images, assigning each element to one of these four levels. We found strong agreement between the photographers. We also observed high agreement on the order in which object should be removed. The consensus was overwhelming, which confirmed the intuitive ordering we built into our system. We then tested whether a vision language model, specifically Gemini, classifies elements the same way humans do. We found strong agreement between the photographers and Gemini. Using this taxonomy, we apply an iterative select, remove and verify process. In the select step, a vision language model evaluates the current image, extracts a list of objects from the active category, and then picks the single least important element to remove next. Then an image editing model erases that element to keep the rest of the photo completely untouched. We align, mask and blend the edited pixels back into the input image. This prevents quality drift over multiple steps. And finally, we verify. We train the classifier to check each removal and decide whether the result still looks like a realistic, simpler photo. If an edit fails the check, we reject it and try another candidate. If it passes, we accept it and move on. We then repeat this loop on the updated image. Because removing an element can change the scene, our system dynamically updates its choices at every single step. If no element remains in the current semantic category, we move to the next level in the hierarchy. This search based pipeline produces highly realistic, high quality simplifications. However, it's very slow, taking a couple of hours to process a single image. To solve this, we distill the entire search process into a single, much faster image to video model. The trick here is to view the sequences of simplified images as a subtractive stop motion video. Moving forward in time simply means going deeper into simplification. Using Lora, we fine tune an image to video model on the trajectories generated from stage one. This allows the model to learn both the optimal removal order and the inpainting task directly. This fine tuned model returns a smooth video. Early frames gently declutter the scene while later frames converge to a minimal representation. We maintain the high quality of the search method, but at a fraction of the cost, reducing the time from hours to just minutes. Surprisingly, the distilled model nearly matches the search based method in removal order. Accuracy. In the paper, we provide details about the evaluation process and comparison to additional baselines. Let's look at some results. Here you can see the change from a full cluttered scene all the way down to a minimalist scene using our search based baseline. And here are results generated by our distilled model. Notice how the video stays smooth and consistent the whole time. It's very useful to clean my room so quickly. Beyond just generating videos, our method enables some applications like interactive simplification, semantic decluttering and image layering. First, interactive simplification where the user can explore scene using a dynamic simplification slider and can even branch off the sequence to explore alternative removal trajectories. Second, semantic decluttering. Because distractors are removed first, the early frames of our sequence automatically provide a cleanup version of the original photo, requiring absolutely no manual masking. And third, image layering. By stepping backward through the sequence, we can decompose the scene into clean semantic layers. In summary, we have proposed a novel approach to image simplification that progressively removes elements from the scene while keeping it looking perfectly real. More importantly, our work shows that with today's powerful generative models, abstraction no longer has to be non photorealistic. AI tools are opening up entirely new directions in visual simplifications and creative editing, showing that realism and abstraction can coexist on a continuous spectrum. Thank you for listening.