Automatic1111 Dreambooth extension suddenly OOM #308

FugueSegue · 2022-11-23T19:39:36Z

FugueSegue
Nov 23, 2022

I have a RTX 2090 Ti with 12GB VRAM. I got the Dreambooth extension to work almost as soon as it was first released. I was able to train several checkpoints and I was satisfied with the results.

Now it doesn't work anymore. I think something updated and it broke. Whereas before I was able to train in Dreambooth as much as I wanted, now it just plain won't. OOM. I didn't change anything on my computer. I had configured the webui-user.bat to automatically update A1111 and the extension each time I started it. Never doing that again.

The OOM error doesn't happen when I try to train in Dreambooth. Ever since last night I haven't been able to get that far! Now whenever I try to CREATE a model, I get this error:

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 11.00 GiB total capacity; 10.06 GiB already allocated; 0 bytes free; 10.26 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I never had this error before. I swear I changed nothing other than allow A1111 and Dreambooth to update themself on their own.

So I have spent all of this morning wiping my workstation, re-installing Windows 10, and without installing anything else I have re-installed Python, Git, Pytorch, CUDA, etcetera ad nauseam infinitum. Automatic1111 works fine. Dreambooth doesn't.

Any help is appreciated.

Renaldas111 · 2022-11-23T20:27:22Z

Renaldas111
Nov 23, 2022

Yes, Dreambooth extension is broken after the huge update, OOMs on all possible options.

3 replies

FugueSegue Nov 24, 2022
Author

Okay. I'm rolling back to an earlier commit. Best of luck to the coders.

terminal2727 Nov 26, 2022

If you don't mind me asking, have you gotten it working again? If so, which version did you roll back to? I'm new to dreambooth, so I haven't actually used a version of it that works

FugueSegue Nov 26, 2022
Author

I rolled both Automatic1111 and the Dreambooth extension to the ones just before midnight on November 18. This was the last day I was able to train checkpoint models. Perhaps some of the commits after that might function but I don't know for sure. I just chose that time and date arbitrarily. And, yes, it works fine for me now.

Someday soon I'm going to create a separate installation of Automatic1111 and update that. Just to check if it works. Meanwhile I'm sticking with the one that definitely works.

d8ahazard · 2022-11-26T20:31:35Z

d8ahazard
Nov 26, 2022
Maintainer

SO, rolling back doesn't help anybody, really. I understand your frustration, but as a developer, I can't fix a problem with the current software if people aren't using it.

That said, I've pushed a new update. The default parameters for a model aren't necessarily for everybody, but there's a performance wizard button that should help get you close to the right settings to not OOM.

3 replies

FugueSegue Nov 26, 2022
Author

That's good news! I understand completely. I will install a separate copy of Automatic1111 and update it and the extension to the latest version. I will report back here with the results ASAP.

FugueSegue Nov 26, 2022
Author

I'm sorry, but I got the same error.

I started with a copy of my current installation of Automatic1111 and updated the Dreambooth extension to the latest version. I attempted to create a new model to train and it resulted in the same error I got before.

Here is the entirety of the output to my CLI:

venv "C:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Commit hash:
Installing requirements for Web UI
Checking Dreambooth requirements...
Checking/upgrading existing torch/torchvision installation
Checking torch and torchvision versions
Dreambooth revision is a66a2d6
[X] bitsandbytes version 0.35.0 installed.
[X] diffusers version 0.8.1 installed.
[X] transformers version 4.21.0 installed.
[X] torch version 1.12.1+cu116 installed.
[X] torchvision version 0.13.1+cu116 installed.
[*] xformers version 0.0.14.dev0 installed.

Launching Web UI with arguments: --listen --xformers --theme dark
Got preload file: C:\stable-diffusion-webui\extensions\sd_dreambooth_extension\preload.py
Patching transformers to fix kwargs errors.
Dreambooth API layer loaded
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Loading weights [44ccbca4] from C:\stable-diffusion-webui\models\Stable-diffusion\toni221126a_9000.ckpt
Applying xformers cross attention optimization.
Model loaded.
Loaded a total of 0 textual inversion embeddings.
Embeddings:
Running on local URL: http://0.0.0.0:7860

To create a public link, set share=True in launch().
Loading weights [a9263745] from C:\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned.ckpt
Global Step: 840000
Applying xformers cross attention optimization.
Weights loaded.
Extracting checkpoint...
Cleanup completed.
Allocated: 0.0GB
Reserved: 0.0GB

Allocated 0.0/2.0GB
Reserved: 0.0/2.0GB

Checkpoint loaded from CheckpointInfo(filename='C:\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned.ckpt', title='v1-5-pruned.ckpt [a9263745]', hash='a9263745', model_name='v1-5-pruned', config='C:\stable-diffusion-webui\repositories\stable-diffusion\configs/stable-diffusion/v1-inference.yaml')
Checkpoint C:\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned.ckpt has both EMA and non-EMA weights.
In this conversion only the non-EMA weights are extracted. If you want to instead extract the EMA weights (usually better for inference), please make sure to add the --extract_ema flag.
Error completing request
Arguments: ('toni221126b', 'v1-5-pruned.ckpt [a9263745]', 'ddim', 'runwayml/stable-diffusion-v1-5', '', False) {}
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\ui.py", line 185, in f
res = list(func(*args, **kwargs))
File "C:\stable-diffusion-webui\webui.py", line 54, in f
res = func(*args, **kwargs)
File "C:\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\conversion.py", line 971, in extract_checkpoint
pipe = pipe.to(map_location)
File "C:\stable-diffusion-webui\venv\lib\site-packages\diffusers\pipeline_utils.py", line 227, in to
module.to(torch_device)
File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 927, in to
return self._apply(convert)
File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
module._apply(fn)
File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
module._apply(fn)
File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
module._apply(fn)
[Previous line repeated 2 more times]
File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 602, in _apply
param_applied = fn(param)
File "C:\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 925, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 11.00 GiB total capacity; 10.06 GiB already allocated; 0 bytes free; 10.26 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Traceback (most recent call last):
File "C:\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 284, in run_predict
output = await app.blocks.process_api(
File "C:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 983, in process_api
data = self.postprocess_data(fn_index, result["prediction"], state)
File "C:\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 913, in postprocess_data
if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range

discordianbelle Nov 27, 2022

I'm also having the same issue, not really willing to roll back either

Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
Extracting checkpoint...
Cleanup completed.
Allocated: 0.0GB
Reserved: 0.0GB

Allocated 0.0/2.0GB
Reserved: 0.0/2.0GB

Checkpoint loaded from CheckpointInfo(filename='D:\AI\stable-diffusion-webui\models\Stable-diffusion\HassanBlend1.4_0.35-TotalPreg_e100_0.65-Weighted_sum-merged.ckpt', title='HassanBlend1.4_0.35-TotalPreg_e100_0.65-Weighted_sum-merged.ckpt [6d9cfb49]', hash='6d9cfb49', model_name='HassanBlend1.4_0.35-TotalPreg_e100_0.65-Weighted_sum-merged', config='D:\AI\stable-diffusion-webui\v1-inference.yaml')
Checkpoint D:\AI\stable-diffusion-webui\models\Stable-diffusion\HassanBlend1.4_0.35-TotalPreg_e100_0.65-Weighted_sum-merged.ckpt has both EMA and non-EMA weights.
In this conversion only the non-EMA weights are extracted. If you want to instead extract the EMA weights (usually better for inference), please make sure to add the --extract_ema flag.
Error completing request
Arguments: ('recursive_test_1', 'HassanBlend1.4_0.35-TotalPreg_e100_0.65-Weighted_sum-merged.ckpt [6d9cfb49]', 'dpm', 'runwayml/stable-diffusion-v1-5', '', False) {}
Traceback (most recent call last):
File "D:\AI\stable-diffusion-webui\modules\ui.py", line 185, in f
res = list(func(*args, **kwargs))
File "D:\AI\stable-diffusion-webui\webui.py", line 56, in f
res = func(*args, **kwargs)
File "D:\AI\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\conversion.py", line 971, in extract_checkpoint
pipe = pipe.to(map_location)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\diffusers\pipeline_utils.py", line 227, in to
module.to(torch_device)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 927, in to
return self._apply(convert)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
module._apply(fn)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
module._apply(fn)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 579, in _apply
module._apply(fn)
[Previous line repeated 2 more times]
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 602, in _apply
param_applied = fn(param)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 925, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
RuntimeError: CUDA out of memory. Tried to allocate 114.00 MiB (GPU 0; 12.00 GiB total capacity; 9.10 GiB already allocated; 0 bytes free; 9.32 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Traceback (most recent call last):
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 284, in run_predict
output = await app.blocks.process_api(
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 983, in process_api
data = self.postprocess_data(fn_index, result["prediction"], state)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 913, in postprocess_data
if predictions[i] is components._Keywords.FINISHED_ITERATING:
IndexError: tuple index out of range
Returning [0.9, 0.999, 1e-08, 0.01, 'default', True, '', 1, True, False, False, 1.72e-06, 'constant', 0, 1, 75, 1000, 'fp16', 'D:\AI\stable-diffusion-webui\models\dreambooth\recursivetest0\working', False, 1, True, '', 1.0, 512, 0, 1, 500, 500, False, 'dpm', 'HassanBlend1.4_0.35-TotalPreg_e100_0.65-Weighted_sum-merged.ckpt [6d9cfb49]', 1, True, True, False, False, True, '', 7.5, 60, '', '', '', 'Description', '', '', '', -1, 1, 0, -1, 7.5, 60, '', '', '', 7.5, 60, '', '', '', 'Description', '', '', '', -1, 1, 0, -1, 7.5, 60, '', '', '', 7.5, 60, '', '', '', 'Description', '', '', '', -1, 1, 0, -1, 7.5, 60, '', '', 'Loaded config.']
Gensample?
Cleanup completed.
Allocated: 0.0GB
Reserved: 0.0GB

Allocated 0.0/0.0GB
Reserved: 0.0/0.0GB

Exception generating sample!
Traceback (most recent call last):
File "D:\AI\stable-diffusion-webui\extensions\sd_dreambooth_extension\dreambooth\dreambooth.py", line 58, in generate_sample_img
pipeline = StableDiffusionPipeline.from_pretrained(model_path, safety_checker=None)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\diffusers\pipeline_utils.py", line 482, in from_pretrained
config_dict = cls.load_config(cached_folder)
File "D:\AI\stable-diffusion-webui\venv\lib\site-packages\diffusers\configuration_utils.py", line 312, in load_config
raise EnvironmentError(
OSError: Error no file named model_index.json found in directory D:\AI\stable-diffusion-webui\models\dreambooth\recursivetest0\working.
Restored system models.
Allocated: 2.0GB
Reserved: 2.0GB

orionflame · 2022-11-28T13:46:38Z

orionflame
Nov 28, 2022

I am having similar issues.

0 replies

mykeehu · 2022-11-28T15:04:16Z

mykeehu
Nov 28, 2022

It's worth updating to today's version, there have been many optimizations.

0 replies

Eleksar387 · 2022-11-28T18:36:55Z

Eleksar387
Nov 28, 2022

Same here. I get "IndexError: tuple index out of range". Updated a few minutes ago. But no change.

0 replies

FugueSegue · 2022-11-28T21:19:40Z

FugueSegue
Nov 28, 2022
Author

I updated Dreambooth to 1d7d341.
I updated Automatic to 0b5dcb3d7ce397ad38312dbfc70febe7bb42dcc3.
Same errors as always. I wish I had better news.

1 reply

mykeehu Nov 28, 2022

It should be reported in Issues then, because they are constantly checked by @d8ahazard .

orionflame · 2022-12-02T12:02:05Z

orionflame
Dec 2, 2022

In the latest version I am getting the same error. But this time it said loading dreambooth when I pressed Train. But none of the iterations incremented and just showed 0/10000 and finished. It obviously didn't train anything. Not sure if this issue is known.

1 reply

FugueSegue Dec 2, 2022
Author

If you have low VRAM, consider configuring these settings in the Advanced section of the Parameters tab:

Check "Use 8bit Adam"
Set Mixed Precision to fp16
Set Memory Attention to xformers
Check "Don't Cache Latents"
Uncheck "Train Text Encoder"
Uncheck "Train EMA"
Uncheck "Gradient Checkpointing"

Or may be checking Gradient Checkpointing doesn't matter? I don't know.

orionflame · 2022-12-02T17:06:53Z

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automatic1111 Dreambooth extension suddenly OOM #308

{{title}}

Replies: 8 comments 10 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Automatic1111 Dreambooth extension suddenly OOM #308

Replies: 8 comments · 10 replies

FugueSegue Nov 24, 2022 Author

FugueSegue Nov 26, 2022 Author

d8ahazard Nov 26, 2022 Maintainer

FugueSegue Nov 26, 2022 Author

FugueSegue Nov 26, 2022 Author

FugueSegue Nov 28, 2022 Author

FugueSegue Dec 2, 2022 Author

Replies: 8 comments 10 replies

FugueSegue Nov 24, 2022
Author

FugueSegue Nov 26, 2022
Author

d8ahazard
Nov 26, 2022
Maintainer

FugueSegue Nov 26, 2022
Author

FugueSegue Nov 26, 2022
Author

FugueSegue
Nov 28, 2022
Author

FugueSegue Dec 2, 2022
Author