Character Consistency with AI in Creative Projects — Ammar Alghamdi

Storyboard for a short animation. Tools used: Midjourney, Scenario AI, Adobe Firefly

In art and design, artificial intelligence (AI) opens up new possibilities for keeping characters consistent, especially during the early stages, like storyboarding, concept designs, and pre-production. The idea is to use AI to ensure that characters look the same across different story parts, which is both challenging and exciting.

However, there’s a hurdle when using AI to visualize stories. One big issue is that AI can be unpredictable and doesn’t always fit well with the need for precise, consistent character appearances in storytelling. But, if we take a closer look at how these AI tools work — learning from tons of images to create new ones based on what they’re asked — it becomes more
apparent how they can be a significant asset in creative processes.

In this blog, I will share my experimentation with different AI tools to create a consistent character.

Understanding AI Image Generation Basics

To make the most out of AI for keeping characters looking the same, it helps to know a bit about how these image-making AI tools work. They often use natural language processing (NLP) with text prompts to determine what we’re asking for. Once they get what we’re saying, they use models like diffusion models to turn those instructions into images. This process starts with digital noise and carefully adds and takes away bits until an image that matches the prompt emerges.

Here are also other fantastic techniques like Generative Adversarial Networks (GANs), where two parts of the AI are playing a game — one creating images and the other judging them — to get better and better results. And then there’s something called Neural Style Transfer, which isn’t about making new images from scratch but more about applying a specific look or style to an image. So, by getting a handle on how these tools work, you can guide them to produce characters that stick to your original vision and match in style and vibe across your whole project.

Consistent character with ChatGPT 4: (Custom GPT)

One of the effective ways in using ChatGPT for character consistency is to create your own custom GPT with custom instructions (prompt engineering). I know this may sounds technical, but trust me it is not!

Process:

First we need to create a new GPT. (You will need the PLUS plan to be able to create a custom GPT).

Then head to the Configure tab to create the custom instructions set.

Now, we need to create the custom instruction set. There are a lot of sets that can be found if you research the Internet very well. However, I will share a set that I put together that seems to be working great: click here to access.

After pasting the set in the instruction, we need to add a description and a name of the GPT. Then save the GPT.

After you open your custom GPT, the bot will start providing interactive questionnaires for character details such as style, age, gender, clothing etc. Here is an example of the interaction:

After the GPT gathers your character details, it will start creating the desired characters. Here are some results I got very quickly:

Most of my uses with GPTs for characters are creating a character only and not focusing on the environment and scenes. I usually get characters with different poses and then take the characters generated by our custom GPT to Midjourney or Scenario AI to create scenes for my storyboard.

Some Pros & Cons of using ChatGPT to develop character consistency.

Pros:

• Easy experience: No parameters or technical knowledge are needed.
  Interactive: You engage with the bot by chatting or talking, and it will generate images or edits.
• Excellent quality: The quality of images produced by GPT4 is excellent.

Cons:

• Sometimes, it takes many prompts to get what you want.
• The GPT can hallucinate and produce something that doesn’t reflect the prompt.

Consistent character with Midjourney:

In order to use Midjourney, you will need to have a Discord account. To get started with Midjourney, please visit this Quick
Start guide.

If you want to learn more about the parameters and their use in Midjourney, make sure to check out their documentation
here.

Process:

Here are the key steps to create a consistent character in Midjourney:

Generate reference images using the “panel/split” technique. Prompt Midjourney to split an image into multiple parts:

panel front view, side view, full body of a [Character Features], [Style], multiple expressions and poses, character sheet, [Character clothing] — ar 16:9 — style raw — v 6.0

Let’s put it into work:

3 panel front view, side view, full body of a tall boy, black hair, Pixar animation styled, black pants, blue t-shirt — ar 16:9 — style raw — v 6.0

NOTE: Midjourney (and almost all AI art tools can sometimes give inaccurate outputs during your first prompt, which is fine because we will fix it as we go). Now, we can choose our character from one of these 4 variations. Let’s proceed with the second one by upscaling it (U2)!

The next step is essential. In this step, we will use the Vary (Region) button to select and regenerate specific parts of the chosen and upscaled image. We can assign different expressions to the character with the Vary Region editor.

NOTE: The size of your selection will affect your results. Larger selections give the Midjourney Bot more room to generate new creative details. Smaller selections will result in smaller, more subtle changes.

Click on Vary Region > Choose the Lasso Tool > Select the area we want to edit/regenerate.

Then, we will delete the prompt and write the expression we want.

Now, this is good. We have the front view of our character with a sad expression in 4 different ways. Next, we will return to the upscale image on which we used the Vary Region and make another expression. Until we have enough of what we want for our story. This step of gathering expressions will help us in our next step.

Before proceeding to the next step, we need to crop our character’s 3 main profile views (front and sides). Then, we also need to crop the facial expressions we generated. After that, I prefer to remove the backgrounds. I use iloveimg.com or Photoshop, depending on how easy or difficult the background removal process is.

Profile view (Background removed):

Facial emotions:

Next step is to upload each cropped image to Midjourney. Just drag the images and drop them to the chat. Then hit Enter so they can be uploaded to the chat.

Now the cooking starts. We will use the (prefer option set) command for our following prompt. This command will help us store these images and make them easily accessible.

Type /prefer option set into the message. Then, in the options box, type the character’s name; I choose “Cyan” for our example. Then, click (+1 more) and select the “value” option. Now, inside the value box, drag each uploaded image into the box. Make sure you leave a SPACE in between each image link. After you have all the images dropped, hit ENTER!

Now, there are 2 ways to start using our new character. In both ways, we can ask Midjourney to refer to these photos by simply using the keyword ( — cyan).

1st Way:

We need to generate an environment to put our character in. The environment/settings can be anything if they have the same style as our character, such as the Pixar animation style in our example.

Next, we will hit the Variation button, and include our character’s keyword ( — cyan) with an action/description.

The results:

We can keep creating different environments/scenes and placing our character in that scene with an action, pose, etc. Keep in mind that your prompts will also have an influence on the overall look of the characters. For example, suppose you don’t stick to 1 art style, in our case, Pixar. In that case, Midjourney might be influenced by the other style, affecting the overall look. Keep the style, medium, and parameters consistent when prompting a new scene.

2nd way:

This way will require the use of Vary Region again. Here, you can create a whole scene with a random character. Then, using the Lasso Tool through the Vary Region button, you can replace that random character with your new character.

This is a good but not perfect approach to get consistency with your characters in Midjourney. There are also additional parameters that can be used to enhance the accuracy of the characters, such as Image Weight ( — iw) and (::) after the link of the character images we uploaded.

Creating a consistent character with Midjourney is much more fun now. Recently, Midjourney released a new feature called “Style References“, designed to work similarly to image prompts where you give a URL to one or more images that ‘describe’ the consistent style you want to work over. This parameter is such a game-changer. Although it does not directly help with character consistency, it helps with the overall look of the image of your character when used in a plot/storyboard, etc.

How to use Style Reference:

• Type --sref after your prompt and put one (or more) URLs to images like this --sref url1 url2 url3
• The image model will look at the image urls as ‘style references’ and try to make something that ‘matches’ their aesthetics
• Set relative weights of styles like this --sref urlA::2 urlB::3 urlC::5
• Set the total strength of the stylization via --sw 100 (100 is default, 0 is off, 1000 is maximum)
• Regular image prompts must go before --sref like this/imagine cat ninja ImagePrompt1 ImagePrompt2 --sref stylePrompt1 stylePrompt2