Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Diffutoon 是否可以生成多种风格不同的视频? #206

Open
LesenlyMu opened this issue Sep 12, 2024 · 3 comments
Open

Diffutoon 是否可以生成多种风格不同的视频? #206

LesenlyMu opened this issue Sep 12, 2024 · 3 comments

Comments

@LesenlyMu
Copy link

我想要生成更多风格的动漫视频,应该在什么地方进行修改。我尝试过修改 stable diffusion 模型,但似乎没有明显的变化。

@LesenlyMu LesenlyMu changed the title 是否可以生成多种风格不同的视频? Diffutoon 是否可以生成多种风格不同的视频? Sep 12, 2024
@wenxianxin2000
Copy link

修改提示词,提示词很重要

@Artiprocher
Copy link
Collaborator

Diffutoon 是只考虑动漫风格视频的渲染方案,如果想要生成其他风格的视频,建议把 ControlNet 模型修改成 Depth 和 SoftEdge,并使用其他 Stable Diffusion 底模,另外分辨率需要适当降低。

@nitinmukesh
Copy link

nitinmukesh commented Sep 23, 2024

Diffutoon is a rendering solution that only considers anime-style videos. If you want to generate videos of other styles, it is recommended to modify the ControlNet model to Depth and SoftEdge, and use other Stable Diffusion base models. In addition, the resolution needs to be appropriately reduced.

Thanks for the tip.
I tried this combination before (Depth and SoftEdge), for some reason it take large amount of time. I tried this again today
for 1s / 30 frames / 30 fps / W:576 / H:1024 / 8 GB VRAM / 8 GB Shared

100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [00:41<00:00,  1.37s/it]
100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [06:54<00:00, 13.83s/it]
 10%|███████▊                                                                      | 1/10 [26:45<4:00:53, 1605.96s/it]

Is there anything that I maybe doing wrong. I have tried other controlnets but the processing time is not so much.

[EDIT]
Can confirm it's depth that takes so much time. Without depth

100%|█████████████████████████████████████████████████████████████████████████████████| 30/30 [00:04<00:00,  6.06it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 10/10 [04:04<00:00, 24.48s/it]
Saving images: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:17<00:00,  1.74it/s]
Saving video: 100%|███████████████████████████████████████████████████████████████████| 30/30 [00:00<00:00, 51.22it/s]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants