[CLI]: autologger not working for Stable Video Diffusion about wandb

[ArtsiomWB]

Hi @bdytx5, thank you for writing in. Can you send me your whole error stack that you are seeing when running the code above. Just ran the code myself and would love to see if we are seeing a similar behavior.

[bdytx5]

when I get a chance, I will run it again. I had created a few different versions of the script (1 of which did have a wandb"init" error of some sort). But I was able to somehow get rid of that error, with the issue still being that no media was being uploaded to wandb. Can you confirm it was an error related to wandb.init ?

[ArtsiomWB]

Not sure if this error is fully related to wandb. Are you still not able to see any media in wandb, or have you been able to resolve this already as well?

[ArtsiomWB]

Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.

[bdytx5]

As far as I know the logger simply does not log the videos. Can you confirm that you have tested this? I tried several variations of the script, and videos are not logged.

[ArtsiomWB]

Testing the script on my side right now and getting a:
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc) on my side. Are you seeing any errors when executing the script?

[bdytx5]

That seems to be a cuda error. heres my cuda information that should work if you can obtain a system running similar specs and the newest versions of the packages. Additionally, here is a run I did https://wandb.ai/byyoung3/uncategorized/runs/8pwsl8ca?nw=nwuserbyyoung3 --- Note I added wandb.init() to the script as well

[ArtsiomWB]

Thank you so much for sending your setup over, have sent the last couple of days this week trying to repro, will keep you posted

[bdytx5]

Ok. I used an A5000 on Jarvis labs specifically which I'm guessing you should be able to gain access to

[ArtsiomWB]

Apologies for the delay in response. Finally got my hands on a Nvidia GPU via Sagemaker, but still see the CUDA error regardless. I do see what you are talking about though. Even though the run is done, no video got recorded to wandb at all which is strange. Lets look into it.

[ArtsiomWB]

If possible, could you try to rerun into this issue one more time and then send me another workspace? This time with wandb code saving activated so i can send this behavior straight to our eng team

[ArtsiomWB]

Hi there, I wanted to follow up on this request.

[bdytx5]

Hey, yeah sorry I've been pretty busy lately. I'll try to get to it as soon as I can

[ArtsiomWB]

Sounds like a plan, @bdytx5!

[ArtsiomWB]

Hi @bdytx5, We will close this ticket on our side for tracking purposes, but when you do follow up here we will go ahead and resume the conversation.

[exalate-issue-sync]

WandB Internal User commented:
bdytx5 commented:
when I get a chance, I will run it again. I had created a few different versions of the script (1 of which did have a wandb"init" error of some sort). But I was able to somehow get rid of that error, with the issue still being that no media was being uploaded to wandb. Can you confirm it was an error related to wandb.init ?

[exalate-issue-sync]

WandB Internal User commented:
bdytx5 commented:
As far as I know the logger simply does not log the videos. Can you confirm that you have tested this? I tried several variations of the script, and videos are not logged.

[exalate-issue-sync]

WandB Internal User commented:
bdytx5 commented:

That seems to be a cuda error. heres my cuda information that should work if you can obtain a system running similar specs and the newest versions of the packages. Additionally, here is a run I did https://wandb.ai/byyoung3/uncategorized/runs/8pwsl8ca?nw=nwuserbyyoung3 --- Note I added wandb.init() to the script as well

[exalate-issue-sync]

WandB Internal User commented:
bdytx5 commented:
Ok. I used an A5000 on Jarvis labs specifically which I'm guessing you should be able to gain access to

[bdytx5]

sorry for the delay: https://wandb.ai/byyoung3/uncategorized/runs/iaqp8pde?nw=nwuserbyyoung3

[ArtsiomWB]

No worries!

Could you try running something like this?

import wandb
import torch
from diffusers import StableVideoDiffusionPipeline
from diffusers.utils import load_image
import numpy as np
from wandb.integration.diffusers import autolog

# Initialize W&B autologging
autolog(init=dict(project="stable_video_diffusion_example", entity="<your_entity>"))

# Initialize the video diffusion pipeline
pipe = StableVideoDiffusionPipeline.from_pretrained(
    "stabilityai/stable-video-diffusion-img2vid", torch_dtype=torch.float16, variant="fp16"
)
pipe.enable_model_cpu_offload()

# List of image URLs
image_urls = [
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/svd/rocket.png",
]

# Set a random seed for reproducibility
generator = torch.manual_seed(42)

# Example motion bucket IDs
motion_buckets = [120, 180, 240, 300, 360]
noise_aug_strength = 0.1  # Noise augmentation strength

# Iterate over each motion bucket
for bucket in motion_buckets:
    # Initialize Weights & Biases run for each bucket
    run = wandb.init(project='stable_video_diffusion_example', name=f'bucket_{bucket}', reinit=True, group='exp_1')

    # Process each image for the current motion bucket
    for i, url in enumerate(image_urls):
        # Load the conditioning image
        image = load_image(url).convert("RGB")  # Ensure image is in RGB format
        image = image.resize((1024, 576))

        # Generate video frames with the current motion bucket
        result = pipe(image, decode_chunk_size=8, generator=generator, motion_bucket_id=bucket, noise_aug_strength=noise_aug_strength)

        # Convert frames to numpy array and transpose the axes to (time, channel, height, width)
        frames_np = np.stack([np.array(frame) for frame in result.frames[0]])
        frames_np = frames_np.transpose((0, 3, 1, 2))  # Transpose to (time, channels, height, width)

        # Log video to wandb with specific motion bucket ID
        wandb.log({f"video_{i}_bucket_{bucket}": wandb.Video(frames_np, fps=7, format="mp4")})

    # Finish the wandb run for the current motion bucket
    wandb.finish()

# Finish the final W&B run
wandb.finish()

[ArtsiomWB]

Hi there, I wanted to follow up on this request!

[bdytx5]

Hey, yeah I will try this out when I get a chance.

[ArtsiomWB]

Hi! Have you had a chance to try it out?

[ArtsiomWB]

Hi, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

[bdytx5]

Yes, I can confirm your script does work for logging videos.

[exalate-issue-sync]

WandB Internal User commented:
bdytx5 commented:
Yes, I can confirm your script does work for logging videos.