[Q] Initialize multiple runs and update them simultaneously about wandb

[umakrishnaswamy]

@ksmdnl - got it, unfortunately we do not support this right now since we can only update one run at a time, and you would have to terminate the other run in order to log to the new run. I'd be happy to make a feature request for this, and I'll update this thread if any progress arises on it.

[umakrishnaswamy]

hey @ksmdnl - currently we don't support logging more than 1 run in the same process. this would have to be done sequentially. alternatively, perhaps you can utilize Python's multiprocessing library to spin up separate processes. Each process can handle its own W&B run, allowing you to independently log different forward passes or experiments. Please let me know if you have further questions or would like to discuss further!

[ksmdnl]

Hi @umakrishnaswamy, thanks for the input!
If I understood it correctly, I would then need to initialize the API object in as many processes as the number of forward pass and log the corresponding data there. However, when each process finished, so as each corresponding run is also finished. Thus, each step the same thing should be started again. But I'm not sure if this will help because I'd like to keep each run (without initializing them again every step) and update them one step at a time. Is my understanding correct or do you imagine another use case/approach using multiprocessing?

To be clear, in my scenario it's only one experiment with multiple forward passes per step (which includes the optimization) and I would like to log the output of each pass.

[ksmdnl]

@umakrishnaswamy yes, that would be very helpful. Thanks!