Yael-vinker

clipasso

yael-vinker

8

clipasso is a method for converting an image of an object into a sketch, allowing for varying levels of abstraction. Developed by researchers at Replicate, clipasso uses a differentiable vector graphics rasterizer to optimize the parameters of Bézier curves directly with respect to a CLIP-based perceptual loss. This combines the final and intermediate activations of a pre-trained CLIP model to achieve both geometric and semantic simplifications. The level of abstraction is controlled by varying the number of strokes used to create the sketch. clipasso can be compared to similar models like CLIPDraw, which explores text-to-drawing synthesis through language-image encoders, and Diffvg, a differentiable vector graphics rasterization technique. Model inputs and outputs clipasso takes an image as input and generates a sketch of the object in the image. The sketch is represented as a set of Bézier curves, which can be adjusted to control the level of abstraction. Inputs Target Image**: The input image, which should be square-shaped and without a background. If the image has a background, it can be masked out using the mask_object parameter. Outputs Output Sketch**: The generated sketch, saved in SVG format. The level of abstraction can be controlled by adjusting the num_strokes parameter. Capabilities clipasso can generate abstract sketches of objects that capture the key geometric and semantic features. By varying the number of strokes, the model can produce sketches at different levels of abstraction, from simple outlines to more detailed renderings. The sketches maintain a strong resemblance to the original object while simplifying the visual information. What can I use it for? clipasso could be useful in various creative and design-oriented applications, such as concept art, storyboarding, and product design. The ability to quickly generate sketches at different levels of abstraction can help artists and designers explore ideas and iterate on visual concepts. Additionally, the semantically-aware nature of the sketches could make clipasso useful for tasks like visual reasoning or image-based information retrieval. Things to try One interesting aspect of clipasso is the ability to control the level of abstraction by adjusting the number of strokes. Experimenting with different stroke counts can lead to a range of sketch styles, from simple outlines to more detailed renderings. Additionally, using clipasso to sketch objects from different angles or in different contexts could yield interesting results and help users understand the model's capabilities and limitations.

Read more