100% CPU usage about s2a_fm

[MrYsLab]

What version of s2a_fm and pymata are you using? Are you running this on a PC or something like Raspberry Pi. If on a PC, what CPU are you using and how much RAM do you have?

[mirko314]

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?

—
Reply to this email directly or view it on GitHub
#11 (comment).

[MrYsLab]

Thanks for the info. I will be away from my computer for the next few days. I will investigate and get back to you later this week. Are you running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko [email protected] wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?

—
Reply to this email directly or view it on GitHub
#11 (comment);.

—
Reply to this email directly or view it on GitHub.

[mirko314]

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko [email protected] wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?

—
Reply to this email directly or view it on GitHub
#11 (comment);.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment).

[MrYsLab]

No need to check it on Windows since I run linux myself. Windows most likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko [email protected] wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko [email protected] wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?

—
Reply to this email directly or view it on GitHub
#11 (comment);;.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment);.

—
Reply to this email directly or view it on GitHub.

[mirko314]

Alright then. Do you have this skyrocketing CPU-Usage?
I am using:
Python 2.7.10 (default, Sep 7 2015, 13:51:49)
[GCC 5.2.0] on linux2

On 27.10.2015 14:49, Alan Yorinks wrote:

No need to check it on Windows since I run linux myself. Windows most
likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko [email protected] wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko [email protected] wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?

—
Reply to this email directly or view it on GitHub
#11 (comment);;.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment);.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment).

[MrYsLab]

The last I checked it was much less than 100%. I am going to download and install from scratch and will get back to you later in the week.

On October 27, 2015, at 9:52 AM, Mirko [email protected] wrote:

Alright then. Do you have this skyrocketing CPU-Usage?
I am using:
Python 2.7.10 (default, Sep 7 2015, 13:51:49)
[GCC 5.2.0] on linux2

On 27.10.2015 14:49, Alan Yorinks wrote:

No need to check it on Windows since I run linux myself. Windows most
likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko [email protected] wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko [email protected] wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?

—
Reply to this email directly or view it on GitHub
#11 (comment);;;.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment);;.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment);.

—
Reply to this email directly or view it on GitHub.

[mirko314]

Thank you!
BTW, I really like it, we are currently putting together a course for
Highschool with it :)

On 27.10.2015 17:26, Alan Yorinks wrote:

The last I checked it was much less than 100%. I am going to download
and install from scratch and will get back to you later in the week.

On October 27, 2015, at 9:52 AM, Mirko [email protected] wrote:

Alright then. Do you have this skyrocketing CPU-Usage?
I am using:
Python 2.7.10 (default, Sep 7 2015, 13:51:49)
[GCC 5.2.0] on linux2

On 27.10.2015 14:49, Alan Yorinks wrote:

No need to check it on Windows since I run linux myself. Windows most
likely will not be an improvement.

Is Python 2.7.10 or an earlier version?

On October 27, 2015, at 9:45 AM, Mirko [email protected] wrote:

Alright, I am using it on Linux ( Arch Linux) using it with python 2.7.
I will check it on Windows on the same machine later.

On 27.10.2015 14:41, Alan Yorinks wrote:

Thanks for the info. I will be away from my computer for the next few
days. I will investigate and get back to you later this week. Are you
running Windows and what version of Python are you using?

On October 27, 2015, at 8:36 AM, Mirko [email protected] wrote:

I cloned your current master branch and tested it, Pymata 2.10, pyserial
2.7.
Well it has to be a Bug in the Software because i am running it on my
main Workstation with 16GB Ram and and Intel® Core™ i7-4770K CPU @
3.50GHz × 8. I never have that much CPU-Usage.

On 26.10.2015 22:14, Alan Yorinks wrote:

What version of s2a_fm and pymata are you using? Are you running this on
a PC or something like Raspberry Pi. If on a PC, what CPU are you using
and how much RAM do you have?

—
Reply to this email directly or view it on GitHub
#11 (comment);;;.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment);;.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment);.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#11 (comment).

[MrYsLab]

I got back earlier than I originally thought I would, so I was able to run the program and get some results. First, let me share what my system is:
Ubuntu 15.10, 64 bit, 8 gb ram. My processor is an i5-2600k.

When I run the system monitor, the average CPU utilization across all cores is reported at between 38-40% (across all cores). If I look at the individual cores, on occasion, I do see a short 100% peak on one of the cores, but this peak appears on random cores while the other 3 are pretty close to idle.

I am able to run other programs while s2a_fm is up and running without issues (the browser, libre office, etc.).

Is the 100% you are seeing on average or peaking on a single core? Also, are you locked out of access to other programs as a result?

What tool are you using to monitor CPU utilization?

BTW, If I run "top", I see a consistent total usage of 145%. If all 4 cores were fully occupied, then I would see 400 %.

[mirko314]

Well, It is not an overall 100% its just one Core is at 100% even tough
it switches between the cores. I do not get impaired performance Really,
because I still got 7 different Cores. I just hear that my fans getting
loud because of the cpu usage.
I am using the gnome-system-monitor.

On 28.10.2015 00:00, Alan Yorinks wrote:

I got back earlier than I originally thought I would, so I was able to
run the program and get some results. First, let me share what my system is:
Ubuntu 15.10, 64 bit, 8 gb ram. My processor is an i5-2600k.

When I run the system monitor, the average CPU utilization across all
cores is reported at between 38-40% (across all cores). If I look at the
individual cores, on occasion, I do see a short 100% peak on one of the
cores, but this peak appears on random cores while the other 3 are
pretty close to idle.

I am able to run other programs while s2a_fm is up and running without
issues (the browser, libre office, etc.).

Is the 100% you are seeing on average or peaking on a single core? Also,
are you locked out of access to other programs as a result?

What tool are you using to monitor CPU utilization?

BTW, If I run "top", I see a consistent total usage of 145%. If all 4
cores were fully occupied, then I would see 400 %.

—
Reply to this email directly or view it on GitHub
#11 (comment).

[MrYsLab]

Ok, you had me worried. The program is actually running as expected. The reason for the high percentage is that there is a thread running in a tight loop looking for characters to come in from the Arduino (this is in pymata). Because both pymata and s2a_fm are multithreaded, the overall throughput performance (handling data bidirectionally for the Arduino and Scratch program) is pretty good, but CPU utilization is relatively high. Python has a limitation in running threads. The global interpreter lock (GIL) prevents multiple threads running simultaneously, and threads cannot be distributed amongst the cores. Just a fact of life. Dave Beazly does a nice job describing this.

Recently I have used the asyncio library in another project and in the second generation of pymata (it's called pymata-aio). Running similar applications and comparing the 2 libraries, PyMata shows the same performance as described in a previous comment (~35-40%). When I run pymata-io, the average performance drops from 35% to 4% and no core ever runs at 100%. It is not multi-threaded, but uses event driven asyncio, a much more efficient way of handling concurrency.

I am thinking of porting s2a_fm to asyncio (no decision yet). The pymata end is already done, and being used in other Scratch projects rb4s. Stay tuned.

Is it ok for me to close this issue?

[SebCanet]

Anything to enhance s2a will be welcomed ! :-D
Thanks, regards.

[MrYsLab]

@SebCanet Ok, you convinced me. I will take a look at porting it. ;-). I am not sure what is entailed yet, but hopefully it shouldn't be too bad.

[SebCanet]

Yahou !!!! Thanks a lot !

[MrYsLab]

@SebCanet If I document the new program on a github wiki instead of providing a user's manual, would it make things easier for you, since the wiki pages can be translated in the browser?

All of the block translations should remain the same and the current Scratch files should continue to work.

Also, the latest pymata (pymata-aio) auto detects com ports. I will try to have the program auto launch Scratch.

[mirko314]

Thank you for explaining this behavior! It would be really appriciated if you port it to the async PyMata lib.
BTW: I restructured your s2a_fm.py to enable autmatic startup of scratch(under Linux)
Looking forward for the implementation!
Thank you!
Mirko

[MrYsLab]

@mirko314, @SebCanet I just ran a prototype version of s2a_fm using Python asyncio (I am calling the new version s2aio). You can see the results here. The good news is that you can continue to develop Scratch programs using the current s2a_fm and these programs should be able to be used without modification when s2aio becomes available (at least that is the goal). Thanks Mirko for bringing this issue up.

[SebCanet]

Thansk for everything !!! That' exactly in this way of mind that I created my "s2a control panel", modular way.
And that's why they choose "our" version of software (whereas mBlock, or Scratchbot, or any other) to create documentation for teacher sold here : http://www.xynops.com/scratch-arduino.html