Comments (27)
If you have multiple Jupyter Servers open with the Jupyter Scheduler server extension running, that might be related to the behavior you're seeing. If you keep only one such server running, does that fix the problem?
(This may be an enhancement opportunity to run jobs only once, even if multiple servers are running.)
from jupyter-scheduler.
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋
Welcome to the Jupyter community! 🎉
from jupyter-scheduler.
This job ran from its daily schedule and now I received 4 emails and it appears the job executed 4 times today (1 more time than yesterday). Is this going to continue to increase each day by 1?
Here you can see the notebook jobs completed:
Here is the current scheduled job:
from jupyter-scheduler.
Hi @TallGibbs. Thank you for opening this issue and proactively providing jupyter troubleshoot
output, this is very helpful.
Could you please send a screenshot of "Run on a schedule" section of the Create Job screen for the job that this happened/happening with? For the sake of reproduction I'd like to understand if you define the schedule via Interval > Day
or Interval > Custom schedule > cron expression
option.
![Screenshot 2024-01-25 at 11 18 26 AM](https://private-user-images.githubusercontent.com/26686070/299783602-d402fedc-1ba2-42f1-99eb-5b66a29decaf.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDg1NDcyNjAsIm5iZiI6MTcwODU0Njk2MCwicGF0aCI6Ii8yNjY4NjA3MC8yOTk3ODM2MDItZDQwMmZlZGMtMWJhMi00MmYxLTk5ZWItNWI2NmEyOWRlY2FmLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDAyMjElMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwMjIxVDIwMjI0MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTMzMjdmYjI5NWRlM2U1ZThiNTIzMTg0MjMxZDVkNTRkYmE0Y2IyZDFjYzkzYTNlMTU4YTIxNzBlODViNGQ5MDcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.wYas4fv7mW864tkG2KWbJgvOahw2l_7-ZyTtD4uax8Q)
from jupyter-scheduler.
Here you go!
Final output of the scheduled job from the "Job Definition" page:
from jupyter-scheduler.
Thank you for the details @TallGibbs. I was not able to reproduce this through clock manipulation, trying to reproduce "organically" by waiting 12 PM.
from jupyter-scheduler.
@andrii-i Today the job completed two times. However, yesterday afternoon I re-installed jupyter notebook version 6.5.4 through Anaconda Navigator GUI, then updated (back) to 7.0.6 and then deleted all my jobs in Jupyter lab. Then I scheduled this job again and now it resulted in the job running twice:
from jupyter-scheduler.
I'm not going to touch anything today on the job scheduler and see what happens on Saturday, Sunday and Monday. It should NOT run on the weekend and then should run again on Monday.
from jupyter-scheduler.
I set up a daily job run on my Windows 10 machine that has run exactly once per day, every day, since I first configured it.
If you run jupyter server list
on a command prompt in your same Conda environment, how many entries do you see under "Currently running servers"? If you have multiple Jupyter Server instances running, that might be related to this.
from jupyter-scheduler.
Here is the update for today: on Saturday and Sunday the job did NOT run (which is a good thing; only scheduled Mon through Fri). Today though (Monday), the job ran 3 times which is once more than what happened on Friday.
Somehow the scheduler is running the job once more each time? Is there any way that my code can be causing this? I'm assuming NO because I see on Jupyter Lab's "Notebook jobs" tab indicating the whole job ran (and not that my code just looped through it).
Here is the job's result from today:
from jupyter-scheduler.
I'm going to leave everything the same for tomorrow and see if it run 4 times. Then, I'm going to make a new schedule that is NOT at 12:00 PM and see if that is related to the issue. @andrii-i
from jupyter-scheduler.
Thanks! For what it's worth, my schedule was at 8:00 am, and the run time typically started within 10 seconds of the hour.
from jupyter-scheduler.
I set up a daily job run on my Windows 10 machine that has run exactly once per day, every day, since I first configured it.
If you run
jupyter server list
on a command prompt in your same Conda environment, how many entries do you see under "Currently running servers"? If you have multiple Jupyter Server instances running, that might be related to this.
I just ran that code on my activated environment and I see 5 servers running (I'm not sure if it is safe to post screenshot of full server addresses). It may be confusing though because I also currently have Jupyter Notebook open with some code I am working on, while also running Jupyter Lab.
I'm guessing a better test it to close everything down, then open just an Anaconda Prompt and run that code (after activating my environment)?
from jupyter-scheduler.
If you have multiple Jupyter Servers open with the Jupyter Scheduler server extension running, that might be related to the behavior you're seeing. If you keep only one such server running, does that fix the problem?
(This may be an enhancement opportunity to run jobs only once, even if multiple servers are running.)
What do you think is most valuable for testing/documentation purposes? I see 3 options I could do right now.
Option 1 = Do nothing, and see if script runs 1 additional time tomorrow at noon.
Option 2 = Close all jupyter servers, then proceed with essentially option 1 (let scheduled job run at Noon).
Option 3 = Close all jupyter servers, make a new scheduled job that runs today at 3:00 PM and see what happens
from jupyter-scheduler.
@TallGibbs please try option 2. Please make sure you have only 1 instance of the jupyter server with jupyter_scheduler
extension installed running and see if the duplication goes away. I would expect this to solve the problem. I tried the same schedule on windows with 1 instance of the jupyter server running and every job is created only once:
from jupyter-scheduler.
@andrii-i Ok, I physically closed all open tabs with anything Jupyter related and then opened an Anaconda Prompt and ran the command: jupyter server list
Here are the results from that:
It appears there are 3 servers still running even though I do not physically have any open browsers related to Jupyter. So, to try and shut them down I ran this command: jupyter notebook stop 8888
I get the response "Could not stop server on 8888" within my Anaconda Prompt. Same response for all 3 servers when I try the different ports. It makes me think there is some conflicts between my currently installed Anaconda prompt and previous versions? I had to uninstall and reinstall several times in the past few weeks due to some package conflicts.
from jupyter-scheduler.
@andrii-i sorry for the additional post: it looks like I had to completely close Anaconda Navigator and now I show 0 servers running.
from jupyter-scheduler.
Today's update: with all the servers shutdown the scheduled job did not run at all. Is that the expected response? Is there a way to execute a scheduled job without a server running?
Now that we verified no job ran, with no servers running, I'm going to schedule the file to run this afternoon and see what happens.
from jupyter-scheduler.
Looks good today, job ran successfully and only ran 1 time. Now, I will leave it untouched through tomorrow's scheduled job and verify it still only runs once.
from jupyter-scheduler.
@TallGibbs thank you for testing and confirming that there is no duplication when single server is running.
Today's update: with all the servers shutdown the scheduled job did not run at all. Is that the expected response? Is there a way to execute a scheduled job without a server running?
It is an expected response. Jupyter Scheduler is composed of jupyter_scheduler
server extension and @jupyterlab/scheduler lab extension. jupyter_scheduler
server extension owns and manages job execution. So with no instances of server running there would be no jobs created, with multiple instances of the server running there would be multiple jobs created.
Looks good today, job ran successfully and only ran 1 time. Now, I will leave it untouched through tomorrow's scheduled job and verify it still only runs once.
Sounds good. Please let us know if problem goes away.
from jupyter-scheduler.
It seems like your issue has been resolved, so I'll close this for now. Feel free to continue discussion if needed however! 👋
from jupyter-scheduler.
Thank you everyone for all the help.
I'm not sure this is directly the right place for this question, but thought I would ask anyways even if just for a redirect: is there a way for me to have a jupyter server session running without Anaconda Navigator open? We are trying to find a replacement for Alteryx Server running scheduled jobs (at my company) and I'm hoping I can find a way with Jupyter where a schedule job can run even when my work computer is shutdown. I'm not an expert in computer networks/servers (clearly), so I am kind of figuring this out as I go...
from jupyter-scheduler.
I run JupyterLab by running jupyter lab
from an Anaconda-enabled command prompt, and I haven't run Anaconda Navigator at all. You'll need to have a server running all the time, ideally something running in a data center or in a hosted environment (disclosure: I work for AWS), that stays up so that jobs get rerun.
from jupyter-scheduler.
@JasonWeill awesome info, thank you that is helpful and gives me some ideas.
from jupyter-scheduler.
Another follow-up closely related to this issue. Is this normal that using the command "jupyter server stop 8888" gives back the response "Could not stop server on 8888" but then when you run "jupyter server list" the server on port 8888 no longer shows running? Seems like a flaw in the code when it tells me "could not stop server on 8888."
Here is my screenshot of this exact sequence:
from jupyter-scheduler.
Jupyter Server has a separate issue queue: https://github.com/jupyter-server/jupyter_server/issues
from jupyter-scheduler.
@TallGibbs below are some good places to ask questions if you don't want to create an issue:
- Discourse https://discourse.jupyter.org
- Appropriate community meeting. Jupyter Server Contributing Hour happening on Thursday (tomorrow for me) is probably the most appropriate for your question. Please find calendar of community events with links to each event here: https://discourse.jupyter.org/t/jupyter-community-calendar/2485
from jupyter-scheduler.
Related Issues (20)
- Telemetry HOT 2
- Why do we use Multiprocessing.Process to execute notebook HOT 1
- RuntimeError: "This event loop is already running" with nbclient==0.8.0 HOT 17
- No such kernel named conda-env-root_tf_2.13.0-py when using mamba
- Queue of jobs with max concurrency HOT 1
- On Safari browser, the jobs section for in JobDefintion page is not visible HOT 7
- New jobs always show up as "Stopped" on Linux
- add unit testing on Linux that runs jobs
- Delete job button is calling the delete API twice
- Output files open path error (%5C instead of backslash issue) HOT 2
- Ability to Prevent Saving Copies of Notebooks on Scheduled runs. HOT 1
- Better issue guidance for new users
- Extend Jupyter-scheduler to create and manage jobs with multiple tasks HOT 2
- Ability to associate kernelspec with a jobdefinition
- Emit telemetry events on create job forms HOT 2
- Scheduler not working despite being installed and enabled HOT 7
- Upgrade to SQLAlchemy 2.x HOT 9
- Lab extension makes a blocking call to the server extension on startup
- Clarify support lifecycle after JupyterLab 3 end of maintenance
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jupyter-scheduler.