An error during merging about sd-webui-model-mixer

[miasik]

Fantastic! ;-)
I see my new grid and it looks like we've finally done here :-D
Thank you so much!

Next I want to ensure that Rebasin works not worse than Train Difference. Because TD is amazing.

[miasik]

It looks working now for Sum mode, but Add-Diff/Rebasing still does not.
It shows its progress and a1111 renders a picture in this mode thу picture never changes. If I change mode the picture follows corresponding merges but it freezes again if I switch back to Add-Diff/Rebasing.
Also, what Rebasing and Fast Rb are?

debugs =  ['elemental merge', 'merge', 'adjust']█████████████████                                            | 32/64 [00:08<00:08,  3.85it/s]
use_extra_elements =  True
 - mm_max_models =  4
config hash =  73453409346ffcb55fca326b0e055e898aef8cacd938fcbc304fd6e4a1143e13
  - mm_use [True, False, False, False]
  - model_a Cartoons\fluffity_v21.safetensors [68536065c9]
  - base_model v1-5-pruned-emaonly.safetensors [6ce0161689]
  - max_models 4
  - models ['Photo\\epicphotogasm_lastUnicorn.safetensors [62bb78983a]']
  - modes ['Add-Diff']
  - calcmodes ['Rebasin']
  - usembws [[]]
  - weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  - alpha [0.25]
  - adjust 1,1,1,0,0,0,0,0
  - use elemental [False]
  - elementals ['']
  - Parse elemental merge...
model_a = Cartoons_fluffity_v21
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Cartoons\fluffity_v21.safetensors...
isxl = False
compact_mode =  False
 - check possible UNet partial update...
 - partial changed blocks =  ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'IN09', 'IN10', 'IN11', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08', 'OUT09', 'OUT10', 'OUT11']
 - UNet partial update mode
 - Dynamic loading rebasin module...
Rebasin mode
 - Calulation device for Rebasin is  cuda
 - LAP library is lap
Loading model Photo_epicphotogasm_lastUnicorn...
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\epicphotogasm_lastUnicorn.safetensors...
mode = Add-Diff, alpha = 0.25
Stage #1/2: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:03<00:00, 237.75it/s]
Check uninitialized #2/2: 100%|████████████████████████████████████████████████████████████████████████| 876/876 [00:00<00:00, 146382.88it/s]
Rebasin calc...
maximize weight matching using scipy linear_sum_assignment...
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 12.03it/s]
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 11.53it/s]
P_bg324: new - old = 0.0
weight order changed layers = ['P_bg358', 'P_bg371', 'P_bg324']
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3482, in before_process
        theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in apply_permutation
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in <dictcomp>
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 773, in get_permuted_param
        for axis, p in enumerate(ps.axes_to_perm[k]):
    KeyError: 'cond_stage_model.logit_scale'

---
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:08<00:00,  3.66it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.71it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.89it/s]

[miasik]

Rebasin only support SD1.5 and does not support SD2.1 / SDXL currently. cond_stage_model.logit_scale key is not for SD1.5

It couldn't be the reason, because both models are 1.5 — they were posted as 1.5 and they look fully functional as 1.5 models. Actually, I can merge them as 1.5 models by Sum mode.
Is there a way to just skip unexpected keys and continue merging?

[miasik]

As I see — nothing has changed. Still getting the same error and the rendered picture doesn't chande

*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3487, in before_process
        theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 839, in apply_permutation
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if _valid_key(k)}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 839, in <dictcomp>
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if _valid_key(k)}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 773, in get_permuted_param
        for axis, p in enumerate(ps.axes_to_perm[k]):
    KeyError: 'cond_stage_model.logit_scale'

[wkpark]

Thank you for your reporting,
I have the same issue with the recent 1.7.x sd-webui (currently, I'm using sd-webui dev branch)

Please check some settings "Settings"->"Optimization" and turn off "FP8" optimization (Enable to Disable)

[miasik]

Thank you for your reporting, I have the same issue with the recent 1.7.x sd-webui (currently, I'm using sd-webui dev branch)

Please check some settings "Settings"->"Optimization" and turn off "FP8" optimization (Enable to Disable)

I don't see/find such an option

[wkpark]

I guess this is a similar issue. #29

[miasik]

I guess this is a similar issue. #29
Should I add more logs?

[wkpark]

It looks working now for Sum mode, but Add-Diff/Rebasing still does not. It shows its progress and a1111 renders a picture in this mode thу picture never changes. If I change mode the picture follows corresponding merges but it freezes again if I switch back to Add-Diff/Rebasing. Also, what Rebasing and Fast Rb are?

debugs =  ['elemental merge', 'merge', 'adjust']█████████████████                                            | 32/64 [00:08<00:08,  3.85it/s]
use_extra_elements =  True
 - mm_max_models =  4
config hash =  73453409346ffcb55fca326b0e055e898aef8cacd938fcbc304fd6e4a1143e13
  - mm_use [True, False, False, False]
  - model_a Cartoons\fluffity_v21.safetensors [68536065c9]
  - base_model v1-5-pruned-emaonly.safetensors [6ce0161689]
  - max_models 4
  - models ['Photo\\epicphotogasm_lastUnicorn.safetensors [62bb78983a]']
  - modes ['Add-Diff']
  - calcmodes ['Rebasin']
  - usembws [[]]
  - weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  - alpha [0.25]
  - adjust 1,1,1,0,0,0,0,0
  - use elemental [False]
  - elementals ['']
  - Parse elemental merge...
model_a = Cartoons_fluffity_v21
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Cartoons\fluffity_v21.safetensors...
isxl = False
compact_mode =  False
 - check possible UNet partial update...
 - partial changed blocks =  ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'IN09', 'IN10', 'IN11', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08', 'OUT09', 'OUT10', 'OUT11']
 - UNet partial update mode
 - Dynamic loading rebasin module...
Rebasin mode
 - Calulation device for Rebasin is  cuda
 - LAP library is lap
Loading model Photo_epicphotogasm_lastUnicorn...
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\epicphotogasm_lastUnicorn.safetensors...
mode = Add-Diff, alpha = 0.25
Stage #1/2: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:03<00:00, 237.75it/s]
Check uninitialized #2/2: 100%|████████████████████████████████████████████████████████████████████████| 876/876 [00:00<00:00, 146382.88it/s]
Rebasin calc...
maximize weight matching using scipy linear_sum_assignment...
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 12.03it/s]
P_bg324: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 11.53it/s]
P_bg324: new - old = 0.0
weight order changed layers = ['P_bg358', 'P_bg371', 'P_bg324']
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3482, in before_process
        theta_0 = apply_permutation(permutation_spec, first_permutation, theta_0)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in apply_permutation
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 833, in <dictcomp>
        return {k: get_permuted_param(ps, perm, k, params) for k in params.keys() if "model_" not in k}
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\rebasin\weight_matching.py", line 773, in get_permuted_param
        for axis, p in enumerate(ps.axes_to_perm[k]):
    KeyError: 'cond_stage_model.logit_scale'

---
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:08<00:00,  3.66it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.71it/s]
Total progress:  50%|████████████████████████████████████████████                                            | 32/64 [00:08<00:08,  3.89it/s]

Rebasin only support SD1.5 and does not support SD2.1 / SDXL currently.
cond_stage_model.logit_scale key is not for SD1.5

"Fast rebasin" only checks special weights groups.
(rebasin itself checks all permutation groups and sorts keys internally results possible permutations. but almost all orders of permutation groups are not changed (identical) except special groups. so Fast Rebasin will result identical results if it does not found special groups on merging blocks)

[wkpark]

Rebasin only support SD1.5 and does not support SD2.1 / SDXL currently. cond_stage_model.logit_scale key is not for SD1.5

It couldn't be the reason, because both models are 1.5 — they were posted as 1.5 and they look fully functional as 1.5 models. Actually, I can merge them as 1.5 models by Sum mode. Is there a way to just skip unexpected keys and continue merging?

OK. I see, It needs to be checked the validity or acceptable keys in the weight matching routine.

[wkpark]

This issue has been automatically closed. Please reopen if this issue is not resolved

[wkpark]

thanks for your reporting. I've fixed logic bug again

[miasik]

Nothing new again. Rebasin doesn't change the image

debugs =  ['elemental merge', 'merge', 'adjust']████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.44it/s]
use_extra_elements =  True
 - mm_max_models =  4
config hash =  0abd1b6d1456626614caacd09c282b776869dcac07788f0289caa9e1a80dbfb5
  - mm_use [True, False, False, False]
  - model_a Cartoons\fluffity_v21.safetensors [68536065c9]
  - base_model Cartoons\fluffity_v21.safetensors [68536065c9]
  - max_models 4
  - models ['Photo\\epicphotogasm_lastUnicorn.safetensors [62bb78983a]']
  - modes ['Add-Diff']
  - calcmodes ['Rebasin']
  - usembws [[]]
  - weights ['0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5']
  - alpha [0.01]
  - adjust
  - use elemental [False]
  - elementals ['']
  - Parse elemental merge...
model_a = Cartoons_fluffity_v21
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Cartoons\fluffity_v21.safetensors...
isxl = False , sd2 = False
compact_mode =  False
 - check possible UNet partial update...
 - partial changed blocks =  ['BASE', 'IN00', 'IN01', 'IN02', 'IN03', 'IN04', 'IN05', 'IN06', 'IN07', 'IN08', 'IN09', 'IN10', 'IN11', 'M00', 'OUT00', 'OUT01', 'OUT02', 'OUT03', 'OUT04', 'OUT05', 'OUT06', 'OUT07', 'OUT08', 'OUT09', 'OUT10', 'OUT11']
 - UNet partial update mode
 - Dynamic loading rebasin module...
Rebasin mode
 - Calulation device for Rebasin is  cuda
 - LAP library is lap
Loading model Photo_epicphotogasm_lastUnicorn...
Loading from file E:\SD\automatic1111\models\Stable-diffusion\Photo\epicphotogasm_lastUnicorn.safetensors...
mode = Add-Diff, alpha = 0.01
Stage #1/2: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:13<00:00, 62.83it/s]
Check uninitialized #2/2: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 876/876 [00:00<00:00, 125476.75it/s]
Rebasin calc...
maximize weight matching using scipy linear_sum_assignment...
P_bg324: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 10.52it/s]
P_bg324: new - old = 0.0
weight order changed layers = ['P_bg358', 'P_bg371', 'P_bg324']
Save unchanged weights #2/2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<?, ?it/s]
Clip is fine
 - merge processing in 17.5s (prepare: 0.5s, merging: 17.1s).
 - loading scripts.patches...
 - Fail to patch lora
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 3684, in before_process
        base_dict[key] = theta_0[k]
    KeyError: 'cond_stage_model.logit_scale'

---
Using already loaded model Cartoons_fluffity_v21 + (Photo_epicphotogasm_lastUnicorn - v1-5-pruned-emaonly.safetensors [6ce0161689]) x alpha_0(0.25).safetensors [955683620d]: done in 0.5s (send model to device: 0.5s)
Using already loaded model Cartoons_fluffity_v21 + (Photo_epicphotogasm_lastUnicorn - v1-5-pruned-emaonly.safetensors [6ce0161689]) x alpha_0(0.25).safetensors [955683620d]: done in 0.0s
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.37it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.37it/s]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:12<00:00,  2.43it/s]

[wkpark]

OK. this is another bug. rebasin works just fine. fixed another bug found at the partial update routine.

[miasik]

Ok, this time i noticed some minor changes and I cant get if it's OK
I set XYZ plot with [Model Mixer] alpha B for X and 0,0.25,0.5,0.75,1 as parameter
Then I got a grid but the changes almost invisible

New errors, one for each X value:

debugs =  ['elemental merge', 'merge', 'adjust']█▍                                                                                                                 | 30/150 [00:07<00:28,  4.26it/s]
use_extra_elements =  True
 - mm_max_models =  4
XYZ: alpha b -> 0.25
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in before_process
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in <listcomp>
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
    ValueError: could not convert string to float: ''

[wkpark]

Ok, this time i noticed some minor changes and I cant get if it's OK I set XYZ plot with [Model Mixer] alpha B for X and 0,0.25,0.5,0.75,1 as parameter Then I got a grid but the changes almost invisible

New errors, one for each X value:

debugs =  ['elemental merge', 'merge', 'adjust']█▍                                                                                                                 | 30/150 [00:07<00:28,  4.26it/s]
use_extra_elements =  True
 - mm_max_models =  4
XYZ: alpha b -> 0.25
*** Error running before_process: E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py
    Traceback (most recent call last):
      File "E:\SD\automatic1111\modules\scripts.py", line 710, in before_process
        script.before_process(p, *script_args)
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in before_process
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
      File "E:\SD\automatic1111\extensions\sd-webui-model-mixer\scripts\model_mixer.py", line 2756, in <listcomp>
        mm_finetune = ",".join([str(int(float(x))) if float(x) == int(float(x)) else str(x) for x in fines])
    ValueError: could not convert string to float: ''

thanks again. you just found another bug again.

and fixed now

[wkpark]

Thank you for your patience and quick feedback!