[Docker support] enabling docker, adding description to readme by JustinGuese · Pull Request #134 · CompVis/stable-diffusion

JustinGuese · 2022-08-30T15:21:22Z

adding docker support and a proposal for usage in the readme

krainboltgreene · 2022-08-30T19:59:18Z

+# trigger first download to prevent re-downloading in the future
+# the script will fail as we do not have the weights yet, therefore the exit 0 
+RUN python scripts/txt2img.py; exit 0 


I've been wondering how to do this, is there really no better way to download the deps?

yeah it must be simple code somewhere, didn't find it so far though
must be some tqdm part, maybe:

stable-diffusion/ldm/models/diffusion/ddpm.py

Line 532 in 69ae4b3

for zd in tqdm(samples, desc=desc):

?

Co-authored-by: eggplants <w10776e8w@yahoo.co.jp>

JustinGuese · 2022-09-01T07:51:05Z

thx @eggplants the weight(th)s gets me every time

JustinGuese · 2022-09-01T08:00:17Z

okay the part that does not get initialized yet is
Downloading: "https://github.com/DagnyT/hardnet/raw/master/pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth" to /root/.cache/torch/hub/checkpoints/checkpoint_liberty_with_aug.pth

so I will try to add that to the docker image as well...
and it gets really huge anyways, so I thought about just including the weights? what do you think?

it's already like 10 GB, so 14 GB wouldn't be to different neither :D it's absurd yes

Co-authored-by: Harrison Kiang <hkiang01@gmail.com>

hkiang01

Optimizations re: Docker's build cache

hkiang01 · 2022-09-07T02:07:49Z

+RUN ln -s /root/miniconda3/bin/conda /usr/local/bin/conda
+RUN conda update -y conda
+# i am using git clone instead during development of this dockerfile
+COPY . /app/ 


So as to take advantage of Docker's build cache, COPYs should be towards the bottom of the Dockerfile as possible. In this case, the only files required before the RUN conda env create... layer is the environment.yml and setup.py.

Suggested change

COPY . /app/

COPY environment.yml setup.py ./

need to check if really only the environment.yml is needed, I think the .yml uses the setup.py to build the local package as well, that is why I thought it's needed as well. will check it out though

hkiang01 · 2022-09-07T02:10:49Z

+RUN wget https://github.com/DagnyT/hardnet/raw/master/pretrained/train_liberty_with_aug/checkpoint_liberty_with_aug.pth -P /root/.cache/torch/hub/checkpoints/checkpoint_liberty_with_aug.pth
+# and now just grab the weights as well
+RUN wget https://www.googleapis.com/storage/v1/b/aai-blog-files/o/sd-v1-4.ckpt?alt=media  -P weights/sd-v1-4.ckpt
+CMD [ "python", "scripts/txt2img.py", \


Along with the other suggestion for COPY, to make sure the files are still available, we can move the original COPY towards the bottom, here

Suggested change

CMD [ "python", "scripts/txt2img.py", \

COPY . /app/

CMD [ "python", "scripts/txt2img.py", \

magnusviri · 2022-09-07T23:26:52Z

@JustinGuese, would you be willing to add this to https://github.com/lstein/stable-diffusion?

JustinGuese · 2022-09-12T10:11:15Z

@magnusviri can do

JustinGuese · 2022-09-14T11:49:33Z

@magnusviri i added it to lstein's version: invoke-ai/InvokeAI#560

0x2b3bfa0 · 2022-09-15T06:27:01Z

+RUN rm /usr/local/bin/python
+RUN ln -s /root/miniconda3/envs/ldm/bin/python /usr/local/bin/python


Have you considered this simpler alternative?

Suggested change

RUN rm /usr/local/bin/python

RUN ln -s /root/miniconda3/envs/ldm/bin/python /usr/local/bin/python

ENV PATH /root/miniconda3/envs/ldm/bin:$PATH

yeah at least for me it didn't work when calling python later on... but feel free to try it out

you can also look at PYTHONPATH

Suggested change

RUN rm /usr/local/bin/python

RUN ln -s /root/miniconda3/envs/ldm/bin/python /usr/local/bin/python

ENV PYTHONPATH=/root/miniconda3/envs/ldm

0x2b3bfa0 · 2022-09-15T06:30:06Z

+FROM python:3.8-slim
+RUN apt update && apt install -y wget git libglib2.0-0 libsm6 libxext6 libxrender-dev
+RUN apt clean && rm -rf /var/lib/apt/lists/*
+# RUN useradd -ms /bin/bash stablediff
+# install conda
+RUN wget \
+    https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \
+    && mkdir /root/.conda \
+    && bash Miniconda3-latest-Linux-x86_64.sh -b \
+    && rm -f Miniconda3-latest-Linux-x86_64.sh \
+    && echo PATH="/root/miniconda3/bin":$PATH >> .bashrc 
+RUN chmod +x /root/miniconda3/bin/conda
+RUN ln -s /root/miniconda3/bin/conda /usr/local/bin/conda
+RUN conda update -y conda


The official continuumio/miniconda3 container images would spare a few lines

Suggested change

FROM python:3.8-slim

RUN apt update && apt install -y wget git libglib2.0-0 libsm6 libxext6 libxrender-dev

RUN apt clean && rm -rf /var/lib/apt/lists/*

# RUN useradd -ms /bin/bash stablediff

# install conda

RUN wget \

https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh \

&& mkdir /root/.conda \

&& bash Miniconda3-latest-Linux-x86_64.sh -b \

&& rm -f Miniconda3-latest-Linux-x86_64.sh \

&& echo PATH="/root/miniconda3/bin":$PATH >> .bashrc

RUN chmod +x /root/miniconda3/bin/conda

RUN ln -s /root/miniconda3/bin/conda /usr/local/bin/conda

RUN conda update -y conda

FROM continuumio/miniconda3

i was hesitant about that because it was last updated 3 months ago... but yeah maybe using that and just do a apt update upgrade...?

krainboltgreene · 2022-09-23T09:57:59Z

Fun fact: 3 of the dependencies that you install store files in stable-diffusion/src/ and if you don't cache those files you have to reinstall every time.

Co-authored-by: Helio Machado <0x2b3bfa0+git@googlemail.com>

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

JustinGuese · 2022-09-23T10:02:50Z

@krainboltgreene

Fun fact: 3 of the dependencies that you install store files in stable-diffusion/src/ and if you don't cache those files you have to reinstall every time.

feel free to edit the dockerfile, or you can name me the URLs of them and the paths where they are supposed to be and I will add it ;)

krainboltgreene · 2022-09-23T18:37:06Z

@JustinGuese Will do when I have time. It's the CLIP and taming-transform dependency that make files in src/.

krainboltgreene · 2022-09-23T18:37:57Z

Oh also, don't run conda env update because it will update libraries past what works.

JustinGuese added 3 commits August 30, 2022 17:19

enabling docker, adding description to readme

1f065ce

cleaner dockerfile

241808f

entrypoint -> cmd

ed57090

krainboltgreene reviewed Aug 30, 2022

View reviewed changes

eggplants reviewed Aug 31, 2022

View reviewed changes

Comment thread README.md Outdated

eggplants reviewed Aug 31, 2022

View reviewed changes

Comment thread README.md Outdated

eggplants reviewed Aug 31, 2022

View reviewed changes

Comment thread Dockerfile Outdated

JustinGuese and others added 3 commits September 1, 2022 09:50

typo weights

56652e0

Co-authored-by: eggplants <w10776e8w@yahoo.co.jp>

typo weights

a14d536

Co-authored-by: eggplants <w10776e8w@yahoo.co.jp>

typo weights

c035a67

Co-authored-by: eggplants <w10776e8w@yahoo.co.jp>

just download all requirements now

0890fda

hkiang01 reviewed Sep 6, 2022

View reviewed changes

Comment thread docker-compose.yaml

add platform to docker image for mac builds

12169c5

Co-authored-by: Harrison Kiang <hkiang01@gmail.com>

hkiang01 reviewed Sep 7, 2022

View reviewed changes

0x2b3bfa0 reviewed Sep 15, 2022

View reviewed changes

jackalcooper reviewed Sep 16, 2022

View reviewed changes

Comment thread Dockerfile Outdated

jackalcooper reviewed Sep 16, 2022

View reviewed changes

Comment thread README.md Outdated

jackalcooper reviewed Sep 16, 2022

View reviewed changes

Comment thread README.md Outdated

JustinGuese and others added 4 commits September 23, 2022 12:01

Update Dockerfile

a275d9d

Co-authored-by: Helio Machado <0x2b3bfa0+git@googlemail.com>

Update Dockerfile

a4e7663

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

Update README.md

7fdee4e

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

Update README.md

011e831

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

	CMD [ "python", "scripts/txt2img.py", \
	COPY . /app/
	CMD [ "python", "scripts/txt2img.py", \

		RUN rm /usr/local/bin/python
		RUN ln -s /root/miniconda3/envs/ldm/bin/python /usr/local/bin/python

	RUN rm /usr/local/bin/python
	RUN ln -s /root/miniconda3/envs/ldm/bin/python /usr/local/bin/python
	ENV PATH /root/miniconda3/envs/ldm/bin:$PATH

	RUN rm /usr/local/bin/python
	RUN ln -s /root/miniconda3/envs/ldm/bin/python /usr/local/bin/python
	ENV PYTHONPATH=/root/miniconda3/envs/ldm

Conversation

JustinGuese commented Aug 30, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JustinGuese commented Sep 1, 2022

Uh oh!

JustinGuese commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

hkiang01 left a comment

Choose a reason for hiding this comment

Uh oh!

hkiang01 Sep 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

magnusviri commented Sep 7, 2022

Uh oh!

JustinGuese commented Sep 12, 2022

Uh oh!

JustinGuese commented Sep 14, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

krainboltgreene commented Sep 23, 2022

Uh oh!

JustinGuese commented Sep 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

krainboltgreene commented Sep 23, 2022

Uh oh!

krainboltgreene commented Sep 23, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

JustinGuese commented Sep 1, 2022 •

edited

Loading

hkiang01 Sep 7, 2022 •

edited

Loading

JustinGuese commented Sep 23, 2022 •

edited

Loading