Comments (16)
# # log the initial model and architecture to an artifact
# with tempfile.TemporaryDirectory() as temp_dir:
# model_name = (
# f"model-{self._wandb.run.id}"
# if (args.run_name is None or args.run_name == args.output_dir)
# else f"model-{self._wandb.run.name}"
# )
# model_artifact = self._wandb.Artifact(
# name=model_name,
# type="model",
# metadata={
# "model_config": model.config.to_dict() if hasattr(model, "config") else None,
# "num_parameters": self._wandb.config.get("model/num_parameters"),
# "initial_model": True,
# },
# )
# model.save_pretrained(temp_dir)
# # add the architecture to a separate text file
# save_model_architecture_to_file(model, temp_dir)
# for f in Path(temp_dir).glob("*"):
# if f.is_file():
# with model_artifact.new_file(f.name, mode="wb") as fa:
# fa.write(f.read_bytes())
# self._wandb.run.log_artifact(model_artifact, aliases=["base_model"])
# badge_markdown = (
# f'[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge'
# f'-28.svg" alt="Visualize in Weights & Biases" width="20'
# f'0" height="32"/>]({self._wandb.run.get_url()})'
# )
# modelcard.AUTOGENERATED_TRAINER_COMMENT += f"\n{badge_markdown}"
I just commented out the following in the integration_utils.py
of transformers of Hugginggface.
from wandb.
@umarbutler, what version currently works for you?
@ArtsiomWB 4.40.2
from wandb.
Hi @jubueche, could you please talk a bit about your workflow, and why you are interested in turning off the artifact logging ?
from wandb.
Hi,
I don't need to turn off the artifact logging. I just don't want my model to get synced to wandb. My models are multiple GB and it takes quite some space and time when I upload them.
from wandb.
Gotcha, could you please try setting os.environ["WANDB_LOG_MODEL"] = "false"
from wandb.
Gotcha, could you please try setting
os.environ["WANDB_LOG_MODEL"] = "false"
This does not work.
from wandb.
@ArtsiomWB I can confirm. This does not work.
from wandb.
Hey @jubueche, thank you so much for the workaround. It is strange that os.environ["WANDB_LOG_MODEL"] = "false"
is not working on your side. What version of wandb are you currently on? Will try reproducing this on my side.
from wandb.
Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.
from wandb.
Hi sorry, my wandb version is
>>> wandb.__version__
'0.16.4'
For now I am just using the code with the commented out section.
from wandb.
# # log the initial model and architecture to an artifact # with tempfile.TemporaryDirectory() as temp_dir: # model_name = ( # f"model-{self._wandb.run.id}" # if (args.run_name is None or args.run_name == args.output_dir) # else f"model-{self._wandb.run.name}" # ) # model_artifact = self._wandb.Artifact( # name=model_name, # type="model", # metadata={ # "model_config": model.config.to_dict() if hasattr(model, "config") else None, # "num_parameters": self._wandb.config.get("model/num_parameters"), # "initial_model": True, # }, # ) # model.save_pretrained(temp_dir) # # add the architecture to a separate text file # save_model_architecture_to_file(model, temp_dir) # for f in Path(temp_dir).glob("*"): # if f.is_file(): # with model_artifact.new_file(f.name, mode="wb") as fa: # fa.write(f.read_bytes()) # self._wandb.run.log_artifact(model_artifact, aliases=["base_model"]) # badge_markdown = ( # f'[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge' # f'-28.svg" alt="Visualize in Weights & Biases" width="20' # f'0" height="32"/>]({self._wandb.run.get_url()})' # ) # modelcard.AUTOGENERATED_TRAINER_COMMENT += f"\n{badge_markdown}"
I just commented out the following in the
integration_utils.py
of transformers of Hugginggface.
Do you know if this was a new addition to transformers
? Maybe the problem is on their side? This problem doesn't arise for me on a different system which has an older version of transformers
.
from wandb.
Thank you so much for the follow up, @umarbutler , hey @jubueche , are you able to try a different version of transformets and see if that fixes it?
@umarbutler, what version currently works for you?
from wandb.
Hi there, I wanted to follow up on this request. Please let us know if we can be of further assistance or if your issue has been resolved.
from wandb.
Thank you for the update! @jubueche, could you see if this version works on you side as well?
from wandb.
It's weird, I have 4.41.0.dev0
and it doesn't work for me. My wandb version is 0.16.4
. Even in the current version, it seems that the uploading of the model is still there. See this line.
from wandb.
Ah, actually model.save_pretrained(temp_dir)
is probably the problem, which is not in the updated code anymore. I think this was the problem so should be resolved now. @ArtsiomWB you could make sure that this was the problem by putting that line back in and checking what gets uploaded.
from wandb.
Related Issues (20)
- [App]: Please replace OS-specific _reflink_linux using FICLONE = 0x40049409 with Python-supported os.copy_file_range HOT 3
- [Q] How to download a lot of histories? HOT 3
- [App]: Line plots not rendered correctly HOT 12
- [App]: I cannot see the query charts HOT 2
- [Q] Cant login, not authorized despite using the correct API key HOT 3
- [CLI]: ERROR Internal wandb error: file data was not synced When using sync_tensorboard with Jupyter notebook HOT 11
- [Q] Wandb does not log when using tmux or screen HOT 3
- [CLI]: no good exit when terminated script by CTRL+C HOT 2
- [App]: t.userInfo is null trying to access https://wandb.ai/settings HOT 3
- [llamaindex]: Failed to log trace tree to W&B: `<EventPayload.RESPONSE: 'response'>` HOT 1
- [CLI]: Numpy 2.0 breaks login command HOT 9
- Distributed training with wandb.sweep HOT 6
- does not exist and causes issue with numpy 2.0 release HOT 15
- [Solved][App]: Unable to see the runs in workspace even if the run is taking place successfully HOT 2
- [App]: runs showing from link but not in project page HOT 3
- [Q] Upgrade to gqlparser 2.5.14 in next release? HOT 1
- [App]: All tests fail to run because some dependencies are missing HOT 5
- [Feature]: Halving Random Grid Search for Hyperparameter Tuning HOT 1
- Issue showing runs in the groups from mobile browsers HOT 6
- [Q] "None" was logged when wandb.sweep using pytorch_lightning HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wandb.