Hi there!
I'm Danqi Qian.
A Motion/Visual Designer based in NYC.
︎︎︎
BAKA AI Visual Research / 2023
Stable Diffusion, Stable Diffusion API, MidJourney, Firefly, Lora, ChatGPT
Product Manager: Dillon Chen
AI Visual Researcher: Danqi Qian
Baka is a AI comic product where you can generate multiple manga characters, you can interact with your characters by adding them to your chat. For my task, the challenge is to find a solution to create consistent manga characters and scenes, and to move the AI generation process to backstage using Stable Diffusion API.
TLDR - For this project, I managed to automate Lora training process (dataset image generation + image caption generation) in Google Colab. Meanwhile, with Stable Diffusion API, I wrote API code to automate eg. txt2image with ControlNet, image inpaint, img2img etc, so users only need to deal with writing prompts .
Character Consistensy
The best solution for character consistensy is LoRA in Stable Diffusion, it allows you to train models based on 20-30 images and the result it pretty good.
Link here to see my Lora documentation
We ask users to give us a prompt, and then we generate 20 images based on that prompt as the dataset.
![]()
Scene Consistensy
For use case eg. I want to assign a specific location for my character and I want to consistently show my character in that location.
The solution is to use img2img, we provide users a library of location stock images, so they can tweak the style or look based on their promts, which can later be saved for future usage.
This works well as it’s very easy to implement through SD API.
![]()
Conversation Tone Consistensy
How to give each character a consistent tone and they have the memory of their unique stories? We create a GPT for each character and feed them their story. This is implemented through using OPEN AI API.
Video Generation Exploration
I tested many ways to generate video with a single image in Stable Diffusion: Deforum, img2img, Deforum + Controlnet + video referenc, Animatediff. Animatediff has the best and fastest result.
![]()
I can’t show all the exploration I did but I’d love to chat with you if you have any questions or any insignts in generating AI content!
TLDR - For this project, I managed to automate Lora training process (dataset image generation + image caption generation) in Google Colab. Meanwhile, with Stable Diffusion API, I wrote API code to automate eg. txt2image with ControlNet, image inpaint, img2img etc, so users only need to deal with writing prompts .
Character Consistensy
The best solution for character consistensy is LoRA in Stable Diffusion, it allows you to train models based on 20-30 images and the result it pretty good. Link here to see my Lora documentation
We ask users to give us a prompt, and then we generate 20 images based on that prompt as the dataset.
Scene Consistensy
For use case eg. I want to assign a specific location for my character and I want to consistently show my character in that location.The solution is to use img2img, we provide users a library of location stock images, so they can tweak the style or look based on their promts, which can later be saved for future usage.
This works well as it’s very easy to implement through SD API.
Conversation Tone Consistensy
How to give each character a consistent tone and they have the memory of their unique stories? We create a GPT for each character and feed them their story. This is implemented through using OPEN AI API.Video Generation Exploration
I tested many ways to generate video with a single image in Stable Diffusion: Deforum, img2img, Deforum + Controlnet + video referenc, Animatediff. Animatediff has the best and fastest result.
I can’t show all the exploration I did but I’d love to chat with you if you have any questions or any insignts in generating AI content!
︎︎︎