How to come up with good prompts for Stable Diffusion

Updated Categorized as Tutorial Tagged , 3 Comments on How to come up with good prompts for Stable Diffusion

In this post, I’ll teach you how to create good prompts for generating AI art work images for Stable Diffusion.

This is part 2 of the beginner’s guide series.
Read part 1: Absolute beginner’s guide.
Read part 3: Inpainting.
Read part 4: Models.

What is Stable Diffusion?

Stable Diffusion is a text-to-image AI model. It is trained on millions of image and text description pairs found on the internet. Because it has seen so much, the model understands what text description associates with what images.

As a result, if you put in a prompt like “A Photo of a cat sitting on top of a building”, it would give you images like these:

Cat images generated with Stable Diffusion.

You may be thinking what’s the big deal? Couldn’t we get millions of them in a Google search? What’s intriguing about this technology is that you can prompt the model to generate high quality images that do not exist before. For example, you can ask for a portrait painting of Emma Watson by the 19th century American painter John Singer Sargent:

Prompt stable diffusion to generate portrait of Emma Watson in John Sargent's style.
John Sargent’s painting of Emma Watson.

It is incredible that such images can be produced from keyword-pixel correlations! What’s mind-boggling is that it gets the artistic style, faces (which our brains are very unforgiving of tiny mistakes) and shadows correct, and blends them all together in an aesthetically pleasing manner. The wonder of large numbers is beyond the comprehension of human minds.

Where can I try my prompts?

Check out our list of free sites offering free images generations. Read the Quick Start Guide for other setups.

Anatomy of a good prompt

There are proven techniques to generate high quality, specific images. Your prompt should cover most if not all of these areas

  1. Subject (required)
  2. Medium
  3. Style
  4. Artist
  5. Website
  6. Resolution
  7. Additional details
  8. Color

First you will need a description of the subject with as much detail as possible. E.g.

Subject

A young woman with light blue dress sitting next to a wooden window reading a book.

We got the following image, which matches the prompt pretty well.

We can be more specific. Let’s add a medium. Some examples are: digital painting, photograph, oil painting. Let’s use

Medium

Digital painting

The new prompt is

Digital painting of a young woman with light blue dress sitting next to a wooden window reading a book

The resulting image is

You can see the image changes from a photograph to a digital art.

You get the idea. Let’s add the rest of them

Artist

by Stanley Artgerm Lau

Website

artstation

Resolution

8k

Additional details

extremely detailed, ornate, cinematic lighting

color

vivid

Putting them all together, the prompt is

Digital painting of a young woman with light blue dress sitting next to a wooden window reading a book, by Stanley Artgerm Lau, artstation, 8k, extremely detailed, ornate, cinematic lighting, vivid.

which generates this image:

By adding keywords to the prompt, we can engineer the image to get the style we want.

Tips for good prompts

  • Be detailed and specific when describing the subject.
  • Use multiple brackets () to increase its strength and [] to reduce.
  • Use an appropriate medium type consistent with the artist. E.g. photograph should not be used with van Gogh.
  • Artist name is a very strong style modifier. Use wisely.
  • Experiment with blending styles.
  • Head to the prompt section to study the high-quality prompts. If you like a particular image, use the prompt as a starting point.

Some good keywords for you

Below are some of my favorite keywords and their effects. (Used with Stable Diffusion v1.4 and v1.5)

Enjoy!

Medium

Medium defines a category of the artwork.

keywordNote
PortraitFocuses image on the face / headshot.
Digital paintingDigital art style
Concept artIllustration style, 2D
Ultra realistic illustrationdrawing that are very realistic. Good to use with people
Underwater portraitUse with people. Underwater. Hair floating
Underwater steampunkunderwater with wash color

Style

These keywords further refine the art style.

keywordNote
hyperrealisticIncreases details and resolution
pop-artPop-art style
Modernistvibrant color, high contrast
art nouveauAdd ornaments and details, building style

Artist

Mentioning the artist in the prompt is a strong effect. Study their work and choose wisely.

keywordNote
John Collier19th century portrait painter. Add elegancy
Stanley Artgerm LauStrong realistic modern drawing.
Frida KahloQuite strong effect following Kahlo’s portrait style. Sometimes result in picture frame
John Singer SargentGood to use with woman portrait, generate 19th delicate clothings, some impressionism
Alphonse Mucha2D portrait painting in style of Alphonse Mucha

Website

Mentioning an art or photo site is a strong effect, probably because each site has its niche genre.

keywordNote
pixivJapanese anime style
pixabayCommercial stock photo style
artstationModern illustration, fantasy

Resolution

keywordNote
unreal engineVery realistic and detailed 3D
sharp focusIncrease resolution
8kIncrease resolution, though can lead to it looking more fake. Makes the image more camera like and realistic
vray3D rendering best for objects, landscape and building.

Additional details

Add specific details to your image.

keywordNote
dramaticIncreases the emotional expressivity of the face. Overall substantial increase in photo potential / variability. +1 for variability, important for getting the max hit.
silkAdd silk to clothing
expansiveMore open background, smaller subject
low angle shotshot from low angle **
god rayssunlight breaking through the cloud
psychedelicvivid color with distortion

Color

Add additional color scheme to the image.

keywordNote
iridescent goldShinny gold
silverSilver color
vintagevintage effect

Summary

We have gone through the basic structure of a good prompt. This should be used as a guide rather than rules. The Stable Diffusion model is very flexible. Let it surprise you with some creative combination of keywords!

If you have problem generating stunning artworks, this Stable Diffusion prompt generator would be able to help you.

This is part 2 of the beginner’s guide series.
Read part 1: Absolute beginner’s guide.
Read part 3: Inpainting.
Read part 4: Models.


Buy Me A Coffee

3 comments

  1. Found a good way to generate prompts. Copy this entire article into ChatGPT and ask it to understand how to generate prompts. From there, just ask it what you want to achieve and ChatGPT will give you a good prompt to use.

    Use AI to make AI Art

Leave a Reply