/ AI

그림 AI

그림 AI의 개념과 유형을 간단히 정리하였다.

그림 그려주는 AI를 한번 제대로 써보고 싶었지만 그동안 학업 때문에 여유가 없었다. 드디어 겨울방학이 되면서 시간도 많아졌으니 본격적으로 조사해봤다.

현재 주요한 그림 AI들이 공통적으로 채택하는 학습 방식은 Diffusion Model이라고 한다.

이용 방법

현재 StableDiffusion 기반의 그림 AI들은 공통적으로 프롬프트 를 입력해 이미지로 변환하는 기술을 적용하고 있다.

가장 접근성이 높은 방법은 NovelAI를 이용하는 것이다.

반면 자유롭게 모델을 선택하여 사용이 가능한 오픈 소스 AI를 사용할 수도 있다. 인터넷에서 NovelAI에서 유출된 모델을 포함해 각종 파생 모델을 구할 수 있다.

기존 모델에 원하는 이미지를 추가로 학습시켜 새로운 모델을 만들어 사용할 수도 있다. 이 경우 DreamBooth를 이용해 간편하게 고품질의 모델을 생성할 수 있다.

AUTOMATIC1111이라는 개발자가 제작한 Web UI를 사용하면 img2img 기능 등을 통해 NovelAI를 이용할 때처럼 원하는 모델을 사용하여 간편하게 AI 그림을 제작하는 것이 가능하다.

AI 종류

  • StableDiffusion (SD)

    • 오픈소스로 배포되면서 파생형 AI가 많이 등장했다. NovelAI를 포함하여 현재 널리 사용되고 있는 대부분의 그림 AI들이 이 StableDiffusion을 기반으로 한다.

    • StableDiffusionWebUI 웹 기반 인터페이스로 편리하게 AI를 이용할 수 있는 툴이다. 후술할 각종 파생형 AI도 이 툴을 통해 사용 가능하다. 제작자명을 따서 AUTOMATIC1111이라고로도 한다.

  • WaifuDiffusion (WD)

  • NovelAI (NAI)

    • 소스코드와 모델이 유출되면서 WebUI를 통해 쉽게 로컬 컴퓨터로도 유출 모델을 이용 및 추가 학습 등의 가공이 가능해졌다.

모델 학습 및 생성 AI

직접 소스 이미지를 제공하여 학습을 시키고 새로운 모델을 생성할 수 있는 툴이다. 생성된 모델은 AUTOMATIC1111 에서 사용되는 체크포인트 파일 .ckpt로 변환하여 사용이 가능하다.

  • DreamBooth

    • Google Colab 노트북이 제공되는데, 이를 이용할 경우:

      • Hugging Face Hub의 아이디 및 액세스 토큰이 필요하다.

      • Accelerate 환경 설정이 필요하다.

    • DreamBooth WebUI Extension

      • AUTOMATIC1111의 WebUI 환경에서 간편하게 DreamBooth 학습 기능을 이용할 수 있는 확장이다. WebUI 상의 확장 기능 탭에서도 다운로드 받을 수 있다.

    • kohya_ss

      • CLI 인터페이스로 간편하게 DreamBooth 학습 기능을 이용할 수 있는 툴이다. LoRA 생성도 지원한다.

현재 나는 kohya_ss의 툴을 이용하고 있는데, 추후 작성할 DreamBooth 가이드에서 자세한 설명을 제공할 것이다.

  • DreamArtist

    • 소스 이미지 단 한 장만으로 학습이 가능한 특징이 있다.

    • 로컬 WebUI환경을 지원한다.

프롬프트

유용한 프롬프트는 프롬프트를 공유하는 사이트 등에서 참고할 수 있다. Novel AI 모델로 한정하면 고퀄리티의 그림을 얻을 수 있는 프롬프트를 총정리한 문서인 중국제 원소법전도 유명하다.

사용법

참고문헌

  • NovelAI 에서는 {}, AUTOMATIC1111에서는 () 로 특정 프롬프트의 가중치를 증가 (강화) 시킬 수 있다. 수치상으로 1.1배가 된다.

  • AUTOMATIC1111에서 NAI에서 사용하던 괄호 ()를 문자 그대로 사용하기 위해서는 \(, \) 와 같이 escape 해야 한다.

  • [] 로 특정 프롬프트의 가중치를 약화시킬 수 있다.

  • [] 와 () (AUTOMATIC1111 기준) 등은 각각 여러 개 중첩이 가능하다. 예를 들어 (()) 와 같이 작성하면 가중치가 1.1*1.1=1.21배로 적용된다.

  • (prompt:가중치) 와 같이 직접 가중치를 설정할 수도 있다.

  • prompt:비율|prompt:비율 로 프롬프트를 혼합할 수 있다. (예시: prompt:6|prompt:4)

  • [시작:끝:언제] 로 서로 다른 프롬프트를 특정 스텝 구간에만 각각 적용할 수 있다. 이때 언제에는 0~1사이의 숫자가 들어간다. 예를 들어, mountains:lake:0.25 라는 프롬프트를 명시하면 총 샘플링 스텝 수 중 0.25만큼의 비율에 해당하는 구간에서는 mountains를 적용하고, 나머지 0.75 비율의 구간에서는 lake를 적용한다.

  • 같은 내용의 태그라도 공백 ``과 언더바 _의 사용에 따라 결과가 달라지기도 한다.

Positive vs. Negative Prompt

  • Positive prompt: AI에게 그림에 적용할 요소를 명시하는 프롬프트이다. 일반적으로 ‘프롬프트’는 이것을 가리킨다.

  • Negative prompt: AI에게 그림에 절대로 적용하지 말아야 할 요소를 명시하는 프롬프트이다. 여기에 명시된 프롬프트는 AI가 그림을 그릴 때 절대로 참조하지 않는다. 잘 활용하면 매우 강력한 도구가 될 수 있다.

프롬프트 모음

프롬프트 쓰는 팁

NovelAI 모델에 입력 가능한 프롬프트들을 일부 정리했다.

  • 1990 : 90년대 만화풍
  • scenery : 배경 묘사에 집중
  • 1girl : 여자아이 1명
  • white dress : 흰 드레스
  • looking from afar : 멀리서 바라보는 느낌
  • looking sideways : 옆으로 틀어서 바라보는 느낌
  • windy : 바람이 부는 느낌
  • 작품명(style) : 해당 작품의 스타일을 따라 그린다. (예시: idolmaster(style))

이외에도 여러가지 프롬프트가 있다.

적용 예시

low quality lowres multiple breasts, low quality lowres mutated hands and fingers, low quality lowres long body, low quality lowres mutation poorly drawn, low quality lowres black-white, low quality lowres bad anatomy, low quality lowres liquid body, low quality lowres liquid tongue, low quality lowres disfigured, low quality lowres malformed, low quality lowres mutated, low quality lowres anatomical nonsense, low quality lowres text font ui, low quality lowres error, low quality lowres malformed hands, low quality lowres long neck, low quality lowres blurred, low quality lowres lowers, low quality lowres low res, low quality lowres bad proportions, low quality lowres bad shadow, low quality lowres uncoordinated body, low quality lowres unnatural body, low quality lowres fused breasts, low quality lowres bad breasts, low quality lowres huge breasts, low quality lowres poorly drawn breasts, low quality lowres extra breasts, low quality lowres liquid breasts, low quality lowres heavy breasts, low quality lowres missing breasts, low quality lowres huge haunch, low quality lowres huge thighs, low quality lowres huge calf, low quality lowres bad hands, low quality lowres fused hand, low quality lowres missing hand, low quality lowres disappearing arms, low quality lowres disappearing thigh, low quality lowres disappearing calf, low quality lowres disappearing legs, low quality lowres fused ears, low quality lowres bad ears, low quality lowres poorly drawn ears, low quality lowres extra ears, low quality lowres liquid ears, low quality lowres heavy ears, low quality lowres missing ears, low quality lowres fused animal ears, low quality lowres bad animal ears, low quality lowres poorly drawn animal ears, low quality lowres extra animal ears, low quality lowres liquid animal ears, low quality lowres heavy animal ears, low quality lowres missing animal ears, low quality lowres text, low quality lowres ui, low quality lowres missing fingers, low quality lowres missing limb, low quality lowres fused fingers, low quality lowres one hand with more than 5 fingers, low quality lowres one hand with less than 5 fingers, low quality lowres one hand with more than 5 digit, low quality lowres one hand with less than 5 digit, low quality lowres extra digit, low quality lowres fewer digits, low quality lowres fused digit, low quality lowres missing digit, low quality lowres bad digit, low quality lowres liquid digit, low quality lowres colorful tongue, low quality lowres black tongue, low quality lowres cropped, low quality lowres watermark, low quality lowres username, low quality lowres blurry, low quality lowres JPEG artifacts, low quality lowres signature, low quality lowres 3D, low quality lowres 3D game, low quality lowres 3D game scene, low quality lowres 3D character, low quality lowres malformed feet, low quality lowres extra feet, low quality lowres bad feet, low quality lowres poorly drawn feet, low quality lowres fused feet, low quality lowres missing feet, low quality lowres extra shoes, low quality lowres bad shoes, low quality lowres fused shoes, low quality lowres more than two shoes, low quality lowres poorly drawn shoes, low quality lowres bad gloves, low quality lowres poorly drawn gloves, low quality lowres fused gloves, low quality lowres bad cum, low quality lowres poorly drawn cum, low quality lowres fused cum, low quality lowres bad hairs, low quality lowres poorly drawn hairs, low quality lowres fused hairs, low quality lowres big muscles, low quality lowres ugly, low quality lowres bad face, low quality lowres fused face, low quality lowres poorly drawn face, low quality lowres cloned face, low quality lowres big face, low quality lowres long face, low quality lowres bad eyes, low quality lowres fused eyes poorly drawn eyes, low quality lowres extra eyes, low quality lowres malformed limbs, low quality lowres more than 2 nipples, low quality lowres missing nipples, low quality lowres different nipples, low quality lowres fused nipples, low quality lowres bad nipples, low quality lowres poorly drawn nipples, low quality lowres black nipples, low quality lowres colorful nipples, low quality lowres gross proportions, short arm, low quality lowres missing arms, low quality lowres missing thighs, low quality lowres missing calf, low quality lowres missing legs, low quality lowres mutation, low quality lowres duplicate, low quality lowres morbid, low quality lowres mutilated, low quality lowres poorly drawn hands, low quality lowres more than 1 left hand, low quality lowres more than 1 right hand, low quality lowres deformed, low quality lowres extra arms, low quality lowres extra thighs, low quality lowres more than 2 thighs, low quality lowres extra calf, low quality lowres fused calf, low quality lowres extra legs, low quality lowres bad knee, low quality lowres extra knee, low quality lowres more than 2 legs, low quality lowres bad tails, low quality lowres bad mouth, low quality lowres fused mouth, low quality lowres poorly drawn mouth, low quality lowres bad tongue, low quality lowres tongue within mouth, low quality lowres too long tongue, low quality lowres big mouth, low quality lowres cracked mouth, low quality lowres dirty face, low quality lowres dirty teeth, low quality lowres dirty pantie, low quality lowres fused pantie, low quality lowres poorly drawn pantie, low quality lowres fused cloth, low quality lowres poorly drawn cloth, low quality lowres bad pantie, low quality lowres yellow teeth, low quality lowres thick lips, low quality lowres bad camel toe, low quality lowres colorful camel toe, low quality lowres bad asshole, low quality lowres poorly drawn asshole, low quality lowres fused asshole, low quality lowres missing asshole, low quality lowres bad anus, low quality lowres bad pussy, low quality lowres bad crotch, low quality lowres bad crotch seam, low quality lowres fused anus, low quality lowres fused pussy, low quality lowres fused crotch, low quality lowres poorly drawn crotch, low quality lowres fused seam, low quality lowres poorly drawn anus, low quality lowres poorly drawn pussy, low quality lowres poorly drawn crotch seam, low quality lowres bad thigh gap, low quality lowres missing thigh gap, low quality lowres fused thigh gap, low quality lowres liquid thigh gap, low quality lowres poorly drawn thigh gap, low quality lowres bad collarbone, low quality lowres fused collarbone, low quality lowres missing collarbone, low quality lowres liquid collarbone, low quality lowres strong girl, low quality lowres obesity, low quality lowres worst quality, low quality lowres low quality, low quality lowres normal quality, low quality lowres liquid tentacles, low quality lowres bad tentacles, low quality lowres poorly drawn tentacles, low quality lowres split tentacles, low quality lowres fused tentacles, low quality lowres missing clit, low quality lowres bad clit, low quality lowres fused clit, low quality lowres colorful clit, low quality lowres black clit, low quality lowres liquid clit, low quality lowres QR code, low quality lowres bar code, low quality lowres censored, low quality lowres safety panties, low quality lowres safety knickers, low quality lowres beard, low quality lowres furry, pony, low quality lowres pubic hair, low quality lowres mosaic, low quality lowres excrement, low quality lowres faeces, low quality lowres shit, low quality lowres futa, low quality lowres testis, low quality lowres lowres, low quality lowres terrible, low quality lowres dirty, low quality lowres feces, low quality lowres organs, low quality lowres fat, low quality lowres thick thighs, low quality lowres low resolution rough, low quality lowres pedophile, low quality lowres bestiality, low quality lowres parody, low quality lowres traditional media, low quality lowres koma, low quality lowres comic, low quality lowres scary, low quality lowres severe, low quality lowres insects, low quality lowres gross scars, low quality lowres twisted human body, low quality lowres irrational human body, low quality lowres sharp fingers, low quality lowres parts of the body out of common sense, low quality lowres murder, low quality lowres beheading, low quality lowres zombie, low quality lowres mummy, low quality lowres graffiti, low quality lowres unfinished picture, low quality lowres terrible quality, low quality lowres Coprophilia, low quality lowres muscular, low quality lowres bald, low quality lowres monk, low quality lowres wrinkly, low quality lowres simple background, low quality lowres realistic, low quality lowres old, low quality lowres scan, low quality lowres touhou, low quality lowres yaoi, low quality lowres gay, low quality lowres femboy, low quality lowres trap, low quality lowres pee, low quality lowres doujinshi, low quality lowres monochrome, low quality lowres meme, low quality lowres demon, low quality lowres monstrous creature, low quality lowres tentacle, low quality lowres self harm, low quality lowres vomit, low quality lowres suicide, low quality lowres death, low quality lowres corpse, low quality lowres bone, low quality lowres skeleton, low quality lowres fingers over 6, low quality lowres framed, low quality lowres historical picture, low quality lowres futanari, low quality lowres shemale, low quality lowres trans gender, low quality lowres dick girl, low quality lowres flat breasts, low quality lowres degenerate ass, low quality lowres retro artstyle, low quality lowres anime screencap, low quality lowres stitched, low quality lowres pokemon, low quality lowres ryona, low quality lowres animal, low quality lowres male focus, low quality lowres nipple penetration, low quality lowres sonic (series), low quality lowres bondage, low quality lowres bdsm, low quality lowres 2D, low quality lowres 2D game, low quality lowres 2D game scene, low quality lowres 2D character, low quality lowres game cg, low quality lowres watercolor (medium), low quality lowres 2koma, low quality lowres interlocked fingers, low quality lowres gloves, low quality lowres nitroplus, low quality lowres greyscale, low quality lowres sketch, low quality lowres line drawing, low quality lowres gorilla, low quality lowres meat, low quality lowres gundam, low quality lowres multiple views, low quality lowres cut, low quality lowres concept art, low quality lowres reference sheet, low quality lowres turnaround, low quality lowres chart, low quality lowres comparison, low quality lowres artist progress, low quality lowres lineup, low quality lowres before and after, low quality lowres orc, low quality lowres tusks, low quality lowres goblin, low quality lowres kobold, low quality lowres pony, low quality lowres longbody, low quality lowres nsfw, low quality lowres owres, low quality lowres Humpbacked, low quality lowres text error, low quality lowres extra digits, low quality lowres standard quality, low quality lowres large breasts, low quality lowres shadow, low quality lowres nude, low quality lowres artist name, low quality lowres skeleton girl, low quality lowres bad legs, low quality lowres missing fngers, low quality lowres extra digt, low quality lowres wort quality, low quality lowres jpeg, low quality lowres artifacts, low quality lowres bad body, low quality lowres optical illusion, low quality lowres Glasses, low quality lowres jpegartifacts, low quality lowres girl, low quality lowres women, low quality lowres more than 1 moon, low quality lowres Multi foot, low quality lowres Multifold, low quality lowres Multi fingering, low quality lowres colored sclera, low quality lowres monster girl, low quality lowres Black hands, low quality lowres The background is incoherent, low quality lowres abnormal eye proportion, low quality lowres Abnormal hands, low quality lowres abnormal legs, low quality lowres abnormal feet abnormal fingers, low quality lowres sharp face, low quality lowres tranny, low quality lowres mutated hands, low quality lowres extra limbs, low quality lowres too many fingers, low quality lowres unclear eyes, low quality lowres bad, low quality lowres mutated hand and finger, low quality lowres malformed mutated, low quality lowres boken limb, low quality lowres incorrect limb, low quality lowres fusion finger, low quality lowres lose finger, low quality lowres multiple finger, low quality lowres multiple digit, low quality lowres fusion hand, low quality lowres lose leg, low quality lowres fused leg, low quality lowres multiple leg, low quality lowres bad cameltoe, low quality lowres colorful cameltoe, low quality lowres low poylgon 3D game, chibi, loli

일명 ‘네거티브 법전’ 중 일부로, negative prompt에 입력하면 좋은 결과물을 얻을 수 있다.

용어 정리

  • vae: variational autoencoder (.pt 확장자 파일), 후처리를 담당한다.

  • hypernetwork: 모델에서 주어진 프롬에 대해 데이터(화풍 등)에 해당하는 프롬으로 변환하는 방법을 배움

  • 임베딩(embedding): 모델에 변형을 주지 않은 채로 태그만을 생성함

  • LoRA: 기존 모델에 영향을 주지 않고 레이어 사이에 새로운 레이어를 injection하는 방식으로 학습하는 기술이다.