Today’s release is a new feature for NovelAI Diffusion V3 called “Vibe Transfer”. With Vibe Transfer, you can upload an input image that will be used as a sort of inspiration for generations while enabled. You can control the strength of the transfer, as well as how much information is extracted from the input image and considered during the transformation.
Reference Strength
The “Reference Strength” parameter is fairly intuitive. The closer it is to 1, the stronger the AI will try to emulate visual cues, such as stylistic aspects, colors etc., from the given input image. You might need to experiment a bit to get the best results, as the AI may tend to ignore your text prompt if the Vibe Transfer is too strong, but it will also be able to replicate features from the input more precisely.
Information Extracted
The “Information Extracted” setting can be a bit more complicated, and we recommend using the default value. The Vibe Transfer feature refers to the input image, extracting concepts or features before transferring them to the output generation. In some cases, it can be tricky to control exactly which concepts are transferred from the input. For example, if your input image has a white background, the Vibe Transfer tool will likely also force the output image to have a white background. By reducing the information extracted, you can sometimes avoid unwanted elements from being considered.
Tip: By using the combined powers of your Prompt tags and Undesired Content tags, along with the raised Prompt Guidance and Undesired Content Strength settings, you can make this tool a little easier to control. For example, list existing aspects via tags to reinforce them to be generated or add new tags to try and add or change the content of the base image, or even force them out of the generation by placing them into Undesired Content.
Future Research
The Vibe Transfer tool is part of our ongoing research into new, precise forms of control for our image-generation AI models. We are actively working on better versions and other new tools that will make our NovelAI Diffusion models even more useful for practical use, while improving the user experience in general.
Example: The Starry Night — Painting by Vincent van Gogh’s abstract vibes are transferred to a NovelAIDiffusion V3 generation.
Example: The Swing — Painting by Fragonard’s whimsical colors are vibe transferred to create a NovelAIDiffusion V3 generation.
Conclusion
We’re excited to see what you’ll create with this new tool, and how it will allow you to explore new creative possibilities! As always, feel free to share and tag us so we can see what you make!
