Comments (12)
what would be easier to understand
with_events
orverbose
flag ?
@andreyvelich Well, you'er right, it would be better to use verbose
option without exposing event
definition.
from training-operator.
Good idea. +1
from training-operator.
It sounds good!
Also, at the same time, showing the event of FrameworkJob (e.g., TFJob) at the top might be helpful.
@andreyvelich WDYT?
from training-operator.
Should this be a different API for providing more clarity ?
from training-operator.
It would be helpful!
Maybe we should add new API get_job_events
to print events of job and pods. And we add new arg with_events=False
in API get_job_logs
to make it possible to get all the information in the same API.
from training-operator.
Also, at the same time, showing the event of FrameworkJob (e.g., TFJob) at the top might be helpful.
@tenzen-y That sounds good. The question is how to identify which Job user created ? get_job_logs
doesn't have job_type
as an input argument, and we just check which pod has these labels:
training.kubeflow.org/job-name=my-job
training.kubeflow.org/job-role=master
The same labels could have multiply jobs (e.g. PyTorchJob, XGBoostJob).
That ties to my other question, if we are going to introduce mandatory job_type
argument to our get_job_logs
API, should we follow the same pattern for all APIs ?
E.g. instead of get_pytorchjob
, get_tfjob
, we are going to have a single API called: get_job
which takes job_type
as a mandatory argument and we are going to get appropriate job based on this type.
We can do the same for all other APIs: create_job
, create_job_from_func
, get_job
, delete_job`, etc.
After refactoring our SDK: #1719, I noticed that it is very confusing for the user that we have some CRUD operations job specific (e.g. create_tfjob
), but some of them are not (e.g. get_job_pod_names
, get_job_logs
).
I can create separate issue to discuss this, and I am going to provide more feedback from the users soon.
cc @kubeflow/wg-training-leads
from training-operator.
Should this be a different API for providing more clarity ?
I am not sure, if users who are not familiar with Kubernetes should know differences between events and logs.
Usually, when Data Scientists create a ML Job, they want to directly check the logs from this job (e.g. run get_job_logs
API).
Otherwise, we should somehow explain them if get_job_logs
API fails, they should run get_job_events
API.
WDYT @johnugeorge ?
And we add new arg with_events=False in API get_job_logs to make it possible to get all the information in the same API.
I like the idea @kuizhiqing, what would be easier to understand with_events
or verbose
flag ?
from training-operator.
+1 Agree with you @andreyvelich
from training-operator.
The same labels could have multiply jobs (e.g. PyTorchJob, XGBoostJob).
That ties to my other question, if we are going to introduce mandatory job_type argument to our get_job_logs API, should we follow the same pattern for all APIs ?
E.g. instead of get_pytorchjob, get_tfjob, we are going to have a single API called: get_job which takes job_type as a mandatory argument and we are going to get appropriate job based on this type.We can do the same for all other APIs: create_job, create_job_from_func, get_job, delete_job`, etc.
After refactoring our SDK: #1719, I noticed that it is very confusing for the user that we have some CRUD operations job specific (e.g. create_tfjob), but some of them are not (e.g. get_job_pod_names, get_job_logs).
I can create separate issue to discuss this, and I am going to provide more feedback from the users soon.
@andreyvelich Thanks for the clarification.
I agree with you. Let's work on events for XXXJob in another issue.
from training-operator.
/assign @andreyvelich
from training-operator.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
from training-operator.
/assign @andreyvelich
from training-operator.
Related Issues (20)
- Why manifests/base/service.yaml does not include webhook server port (443) in version 1.7.0~1.5.0? HOT 7
- Not getting Kubeflow Training SDK v1.7 when installing `kubeflow-training` HOT 13
- Flaky Test: [It] should create desired Pods and Services: Distributed TFJob (4 workers, 2 PS) is succeeded
- MPIJob requires service names for the pods. HOT 3
- Add DeepSpeed Example with MPI Operator HOT 9
- chore(style): provide type for `STORAGE_INITIALIZER_VOLUME` constant
- fix(compatability): match-case syntax only compatible with Python3.10 HOT 5
- Export Fine-Tuned LLM after Trainer is Complete HOT 6
- Vulnerability - CVE-2023-44487 HOT 3
- Support ARM64 platform in PyTorch examples HOT 4
- Support ARM64 platform in TensorFlow examples HOT 4
- Support ARM64 platform in XGBoost examples HOT 2
- Update third party worflows in the gh actions HOT 4
- mpijob will stuck if LastReconcileTime is updated in 1 second
- Worker failed without exit code
- PyTorchJobClient not found HOT 3
- The actual default RestartPolicy of PyTorch is inconsistent with its description in the CRD HOT 1
- spatial dataset training functions HOT 1
- TfJob creation failed due to webhook validation failure HOT 1
- [GSOC] Tracking Issue: Integrate JAX in Kubeflow Training Operator
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from training-operator.