Home
Creators
Search Recent Random
Posts
Search DMs Popular Hash Lookup Tags Random
Importer
Import FAQ
Register Favorites Keys Review DMs Logout
Community
Telegram ThePornDude
Home Creators Posts Import Register Favorites Logout

Stable Diffusion ComfyUI Lightweight Workflow For Animated Video (Patreon)

Published:
2024-01-29 13:20:23
Imported:
2026-06
Tags:
haven't archived this post yet. have a subscription? use the importer!

Downloads

  • AnimateDiff LCM-V2V-Lightweight_workflow.zip
  • AnimateDiff LCM-V2V-Lightweight_workflow.zip

Missing 1 full-res photo, 2 files, as this post has not yet been imported.

Content



VIDEO:https://youtu.be/Wtjcr5EtMCg

We have developed a lightweight version of the Stable Diffusion ComfyUI workflow that achieves 70% of the performance of AnimateDiff with RAVE. This means that even if you have a lower-end computer, you can still enjoy creating stunning animations for platforms like YouTube Shorts, TikTok, or media advertisements.

In this video, we take you through the new workflow step by step. We start by loading our checkpoints and setting the clip layers. We then proceed to the IP adapter, which is a basic and fast-loading component designed for lightweight usage. Next, we explore the control net, which focuses on minimalistic control of character movements using the DW pose model. After that, we load the source videos and apply resizing and preprocessing techniques. Finally, we delve into the sampling steps, using the LCM method and applying latent upscales to enhance details and colors.

By following this simple and quick workflow, you can generate animations with different styles without being influenced by the characters or backgrounds of the source videos. The key is to prepare a high-quality, bright, and detailed image in the IP adapter for styling. This image will serve as a reference for the animations. We demonstrate various examples in the video, showcasing the results achieved with different IP adapter images. Although this workflow may not match the quality of the complete element diff workflow, it provides a lightweight and fast solution that is accessible to users with lower computing power. 

Things You Need For This Workflow Default Setting:

Checpoints Model: Any checkpoint you prefer

Lora animatediff/v3_sd15_adapter.ckpt

https://huggingface.co/guoyww/animatediff/blob/main/v3_sd15_adapter.ckpt

AnimateDiff Model: CiaraRowles/TemporalDiff

https://huggingface.co/CiaraRowles/TemporalDiff/blob/main/temporaldiff-v1-animatediff.ckpt

ControlNet: DWPose Preprocesser With OpenPose : 

https://huggingface.co/lllyasviel/sd-controlnet-openpose


Workflow : In This Post Attachment

Files

Previews only

Comments

bw

Hello Benji, first of all thank you for these great posts. They are all amazing. I'm just curious about how to speed up the workflow. I use A100 80GB and the previous workflow takes about 20 minutes. This workflow also takes 15 minutes. Is it normal that it takes this long? Do you have any suggestions on how I can optimize the A100 to shorten the workflow runtime? Thank you very much.

Innovate Futures @ Benji
>>128310329

Nice! A100 love it, I want to buy one also. hehe :) So, which one are you running for 20 mins? The RAVE+ AnimateDiff or the AnimateDiff Full version? Usually, it does takes time to run animate video generation. It's generating frame by frame, most of the time the system are consuming on that. The difference of this one compare with the other 2 workflow(RAVE+animatediff and Animatediff Full Ver.) , this workflow take less memory caching in the VRAM because of less models, controlnet, and sampling processers. If try to speed up with A100, I suggest tranc the frame size smaller. Then scale up later. But then that also consum your time manually to upscale it or another processing time by AI to upscale it.

polaraju p

Error occurred when executing DWPreprocessor: Unknown C++ exception from OpenCV code this error came please tell me

Nelson

Hi Benji, when I queue prompt, the comfy ui got disconnect and here is the error message from my terminal: To see the GUI go to: http://0.0.0.0:8188 FETCH DATA from: /home/nelson/ComfyUI/custom_nodes/ComfyUI-Manager/extension-node-map.json [AnimateDiffEvo] - WARNING - This warning can be ignored, you should not be using the deprecated AnimateDiff Combine node anyway. If you are, use Video Combine from ComfyUI-VideoHelperSuite instead. ffmpeg could not be found. Outputs that require it have been disabled got prompt Killed Any idea how to solve the issue?