exponentialml / comfyui_visualstyleprompting Goto Github PK

Ive been trying to generate images larger than 512 by 512 and realized that the method breaks if the reference image is 1024 by 1024, When the Image gets bigger than 752 x 1024 (not necessarily in that order or distribution of width and height) It stops working altogether and i have a feeling that the effect gets reduced the further away the reference image gets from 512 by 512. An easy way to test with a reference image of 512x512 and confirming that the output is capturing the style, then try with 752 x 1024 (captures the style somewhat strongly) then use 768 x 1024 and the style is gone pretty much completely finally a test with 1024 by 1024 confirms that the intended effect is not present at all.

When using the Visual Style Prompting node, I noticed that changing the denoise parameter had no effect effect. I tested at denoise: 1.00, 0.50, and 0.10 - all results were the same.

This is my workflow:

Hi. I am looking forward to using VSP, the examples shown in the papers look awesome, and look indeed better than the 'competition'. However ... I can not get the expected outcome. I installed as described, via git clone, then opened the provided workflow, in which I chose model realisticvison4, loaded an image, wrote 'purple fur' in the style prompt and 'dog' in the pos prompt, expecting to get a purple fur dog. Please see the image I attached. Can you point me in the proper direction to get it working?

Greetings, Ruud.

this is my old workflow before update look at the result:

but after update it just didn't work even with the same settings :

and here is the reference image :

model: dreamshaper_8LCM

please help me :(

After uploading the picture about 10 times, the memory will be exceeded. it is very easy to exceed the memory,this may be a problem.
I am using a 24G GPU N3090, and this does not happen when running other processes.

Hi I am looking for a workflow with controlnet please.

comfyui.log

The paper said the switching self-attention should only be applied to late upblocks of Unet to achieve the best results.
But this codes seem that the switching self-attention is applied to all stages.

After loading the workflow, the "VisualStylePrompt" node appears to be missing. I have updated and reinstalled the custom node.

The "Apply Visual Style Prompting" is fine and shows up.
The extension is loaded successfully in the Comfy booting.

Any suggestions?

I tested the updated plugin and several problems occurred:

1. The resulting pictures will be a bit blurry.
2. If you add a condition to connect this node, an error will occur. The first version was available before. Conditional connection can avoid prompt word pollution, which is very useful.
   

I hope the author can notice

Is it possible to allow for longer positive prompts?

I kept getting the error:

RuntimeError: Sizes of tensors must match except in dimension 0. Expected size 77 but got size 154 for tensor number 1 in the list.

I had to keep reducing the length of my positive prompt until it was short enough to work successfully.

Hi! I am having the following error (it has worked fine before):

got prompt
Failed to validate prompt for output 15:

• ApplyVisualStyle 22:
  • Required input is missing: clip
  • Required input is missing: reference_latent
  • Required input is missing: reference_cond
  • Required input is missing: positive
  • Required input is missing: negative
    Output will be ignored
    invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}

I tried the workflow, not worked!

'NoneType' object has no attribute 'detach'

What is the problem, please

I have installed it via github link

but still...

The input model is mutated in place, because the forward functions are replaced by monkey patching. This makes the input model unusable in alternative paths in the workflow, for example to denoise 15 steps with visual style applied and 5 steps without.

I tried to load the model twice like this for it to work, it takes 2x the memory as a result:

But it doesn't even work: comfyui ignores the second model and reuses the first one. A really dirty workaround, which has the same memory requirements as I intended above, is to add this at the start of the node fn:

        model = model.clone()
        model.model = copy.deepcopy(model.model)

As a rule of thumb, nodes should never mutate their input in place. This will lead to some very awkward situations and strange bugs.

If you really need to patch the forward functions and can't do it in any other way, I suggest deepcopying the shell of the model object and subobjects while reusing the weights at the leafs. Off the top of my head, you can do this by creating a second model on the meta device, and then loading the weights in from the existing model. Maybe there's an easier way.

im running an LCM model at a CFG of 1. I quickly realized that the tool does not work at all with a CFG of 1, but that makes sense due to things not being encoded fully at that value. That being said, I have noticed it works well on LCM with a CFG of 0.5. anything higher than 1 does a pretty lack luster result. 0.5 seems the best

After testing that a bit with different prompts, I found that the reference prompt does nothing at all to the output. This may be to the low CFG value, but I'm not sure

This node cannot be used with the text encoder in the screenshot.
conditioning>advanced>CLIP Text Encode (Advanced)

Prompt outputs failed validation
ApplyVisualStyle:

• Required input is missing: reference_cond
• Required input is missing: positive
• Required input is missing: negative