Hello everyone, I know many of you began with video creation tools such as Runway and Pika, which are excellent for generating cinematic styles. However, today I'm going to demonstrate how to use Stable Diffusion Deforum to create engaging and trippy video art, for free!
Table of Contents
- Table of Contents
- Step 1: Install Deforum extension and Launch Stable Diffusion WebUI using Stability Matrix
- Step 2: Adjust Deforum settings and use two artwork images as guidance
- Step 3: Enter prompts and generate a sequence of images
- Step 4: Use Premiere Pro to turn sequence images into a video
- Common questions: Can I run Stable Diffusion on a Macbook? How long does it take to generate an image?
- Deforum web version is available!
⭐ Prerequisite: How to one-click install Stable Diffusion using Stability Matrix
Step 1: Install Deforum extension and Launch Stable Diffusion WebUI using Stability Matrix
First, launch Stability Matrix, click Packages, click the tree dots button at the top right of Stable Diffusion WebUI, then click Extensions.
Search and install Deforum.
After finishing installing Deforum, Launch Stable Diffusion WebUI
A new window will pop up. Let’s check if Deforum is installed.
Click Extension tab, scroll down, and deforum-for-automatic1111-webui is active!
Step 2: Adjust Deforum settings and use two artwork images as guidance
Click Deforum tab and adjust the following settings:
- Click Run tab, set desired output resolution. The generation of a larger image size will take more time.
- Change output folder name (optional)
Next, click Keyframes tab and adjust the following:
- Change Max frames. Traditional animation is 25 frames per second. The default is 120 frames, which is around 5 seconds. Feel free to increase the frame number.
- Click Guide Images and check Enable guided images mode.
- Follow the JSON format, and paste image URLs as guidance
{ "0": "https://www.meisterdrucke.us/kunstwerke/1260px/Wassily%20Kandinsky%20-%20Intersecting%20Lines%201923%20-%20%28MeisterDrucke-557692%29.jpg", "20":"https://upload.wikimedia.org/wikipedia/commons/thumb/e/ea/Van_Gogh_-_Starry_Night_-_Google_Art_Project.jpg/800px-Van_Gogh_-_Starry_Night_-_Google_Art_Project.jpg" }
At frame 0, I use Kandinsky's Intersecting Lines as guidance, and at frame 20, I switch the guidance to Van Gogh's The Starry Night.
Then switch to Init tab, check Use Init, and paste the first image URL here again.
Step 3: Enter prompts and generate a sequence of images
Click Prompts tab, follow the JSON format, and enter your prompts to generate abstract art transitioning in style from Kandinsky to Van Gogh.
{ "0": "Abstract art by Wassily Kandinsky, high quality", "20": "Abstract art by Vincent van Gogh, The Starry Night, high quality, vibrant color" }
Open the output folder and you will see that all images are ready!
Step 4: Use Premiere Pro to turn sequence images into a video
Now Open Premiere Pro, Click File > Import (Shortcut Command + I)
In the pop up window, go to your output folder, and click the first image.
Click Show Options, and Check Image Sequence. Then, Import!!!
Now your images are imported as a single video. Drag the video into Timeline, and play it!!
Everything looks good! It’s time to export the video.
Click File > Export > Media
Change the file name and location. Change Preset to Match Source - Adaptive High Bitrate, set Format as H.264, and click export, and your video will be ready in few seconds!
Common questions: Can I run Stable Diffusion on a Macbook? How long does it take to generate an image?
We love RTX 4090, but it's not within everyone's budget😣
So can I run Stable Diffusion on my Macbook? The answer is yes, but it might take longer to generate images.
- My Computer: Apple M2 Pro, 32 GB RAM, 1TB storage.
- Image Resolution: 720 x 720px
- Steps: 20
- Generation time for a single image: 8–9 seconds.
- Generation time for 120 images: around 20 minutes.
This means it takes 20 minutes to generate a 4-second video🥱
Deforum web version is available!
Another solution is to use the web version of Deforum to generate videos. However, this is a paid service. The pricing starts at $8 for a one-time purchase of 20 credits, which allows you to generate 20 15-second animations.
