100% CPU Usage 😢 about eternalterminal

[MisterTea]

That log message is only at verbosity level 4 and higher. You shouldn't see it in normal operation: https://github.com/MisterTea/EternalTerminal/blob/8ce83c9239a881bf433aa29bbd7f0cef3777ea1e/src/terminal/TerminalClient.cpp#L213 Is the server cfg file set to verbose prints?

…

On Sat, Jul 22, 2023, 12:14 PM rahulraj80 ***@***.***> wrote: I am getting 100% CPU usage because of ET : Can someone help and guide? I had installed et v6.2.4 on an embedded debian setup (Storage device) : ***@***.***:~# et --version et version 6.2.4 ***@***.***:~# uname -a Linux wdmc 4.14.4 #14 SMP Wed Dec 20 01:09:34 +05 2017 armv7l GNU/Linux It works well and keeps the connection etc, for the reverse tunnel that I had setup to monitor the server using another on a remote, but accessible machine. After some time - the CPU usage goes up to 100% and stays there forever. This happens within 5 minutes of the connection being triggered. [image: image] <https://user-images.githubusercontent.com/11489601/255342255-ee9c7d9a-0516-4f4c-b3bb-5ebfab03ad63.png> etclientLogs.tar.gz <https://github.com/MisterTea/EternalTerminal/files/12136884/etclientLogs.tar.gz> etserverLogs.tar.gz <https://github.com/MisterTea/EternalTerminal/files/12136885/etserverLogs.tar.gz> Line in crontab which triggers the ET connection - I have started it from terminal and also without the -x option, and that gives me the same results: ***@***.*** 0 * * * * /usr/bin/et -x ***@***.***:pppp -r 8xxx:10xxxx & While retrieving the logs: This commands fails as I don't have these log files: tar -cvzPhf /tmp/etclientLogs.tar.gz /tmp/etclient_err /tmp/etclient.INFO But I had these which I have attached: tar -cvzPhf /tmp/etclientLogs.tar.gz /tmp/etclient-2023-07-22_14-20-02.log /tmp/etclient-stderr-2023-07-22_14-20-02.log Similarly this failed for the server as I did not have these files: tar -cvzPhf /tmp/etserverLogs.tar.gz /tmp/etserver_err /tmp/etserver.INFO But I had these - attached: sudo tar -cvzPhf /tmp/etserverLogs.tar.gz /tmp/etserver-2023-07-20_11-33-10_2532693.log /tmp/etserver-stderr-2023-07-20_11-33-10_2532693.log The etclient log file is flooded with "Got data from stdin" - Not sure if it is a bug. Tail of the log file without "Got data" lines ***@***.***:~# cat /tmp/etclient-2023-07-22_14-20-02.log | grep -v "Got data" | tail [V4 2023-07-22 14:51:10,259 client-main UnixSocketHandler.cpp:23] socket 10 has data [V4 2023-07-22 14:51:10,259 client-main UnixSocketHandler.cpp:45] Unixsocket handler read from fd: 10 [V2 2023-07-22 14:51:10,259 client-main BackedReader.cpp:68] Reading message of length: 95 [V2 2023-07-22 14:51:10,259 client-main BackedReader.cpp:71] bytes remaining: 95 [V4 2023-07-22 14:51:10,259 client-main UnixSocketHandler.cpp:23] socket 10 has data [V4 2023-07-22 14:51:10,259 client-main UnixSocketHandler.cpp:45] Unixsocket handler read from fd: 10 [V3 2023-07-22 14:51:10,259 client-main TerminalClient.cpp:280] Got terminal buffer [INFO 2023-07-22 14:51:10,260 client-main TerminalClient.cpp:326] Window size changed: row: 60638 column: 25787 width: 45510 height: 3856 [V2 2023-07-22 14:51:10,260 client-main BackedWriter.cpp:50] Message length with header: 37 [V4 2023-07-22 14:51:10,260 client-main UnixSocketHandler.cpp:61] Unixsocket handler write to fd: 10 The log file has 128k lines in total: ***@***.***:~# cat /tmp/etclient-2023-07-22_14-20-02.log | wc 128004 1159042 10840264 Of these, 118k lines were the "Got data from stdin lines" - and 9.5k other lines: ***@***.***:~# cat /tmp/etclient-2023-07-22_14-20-02.log | grep -v "Got data" | wc 9543 92893 889540 The 'Got data line looks like this: ***@***.***:~# tail -2 /tmp/etclient-2023-07-22_14-20-02.log [V4 2023-07-22 14:51:10,261 client-main TerminalClient.cpp:210] Got data from stdin [V4 2023-07-22 14:51:10,261 client-main TerminalClient.cpp:210] Got data from stdin — Reply to this email directly, view it on GitHub <#588>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACK5P2W6CZJTLZ3MW3OWSLXRQDBBANCNFSM6AAAAAA2T6LFBU> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[rahulraj80]

The conf file was not changed.

; et.cfg : Config file for Eternal Terminal
;

[Networking]
port = 2022
# bind_ip = 0.0.0.0

[Debug]
verbose = 0
silent = 0
logsize = 20971520
telemetry = true
logdirectory = /tmp

[rahulraj80]

I found a ubuntu/focal based firmware instead of the jessie/stretch one. Let me see if that was the reason for my issue. Will upgrade OS and report.

[rahulraj80]

Sorry for the delay: I ran into a ton of issues with my embedded system due to a wrong flash. Fixed it after a ton of reading and effort.

Update:
On the new OS as well, I have the same issue. The et executable starts using up 100% of the CPU a few minutes after being activated, and then stays at 100% until killed. It also overheats the device enclosure and I can hear the fan working at a higher speed.

I see no errors in the error logs, but if I change verbosity to 1 in et.cfg, I see these two lines repeated continuously.

[INFO 2023-08-03 17:26:38,830 client-main TerminalClient.cpp:326] Window size changed: row: 28811 column: 46730 width: 14152 height: 46849
[INFO 2023-08-03 17:26:38,830 client-main TerminalClient.cpp:326] Window size changed: row: 28811 column: 46730 width: 50832 height: 46848
[INFO 2023-08-03 17:26:38,899 client-main TerminalClient.cpp:326] Window size changed: row: 28811 column: 46730 width: 14152 height: 46849
[INFO 2023-08-03 17:26:38,900 client-main TerminalClient.cpp:326] Window size changed: row: 28811 column: 46730 width: 50832 height: 46848

What can I do to fix it please?

[MisterTea]

Those prints are saying that your window size is changing. The window size is measured in CHARACTERS, so a window size of 50k by 46k is bonkers and will definitely blow up your CPU. You need to find out why the window size is so large.

[rahulraj80]

Those prints are saying that your window size is changing. The window size is measured in CHARACTERS, so a window size of 50k by 46k is bonkers and will definitely blow up your CPU. You need to find out why the window size is so large.

So - this is running from a cron job, and what it does is setup a reverse tunnel to forward a port (say) A from the device to a remote server on port B. It is a non-standard port, so unless I visit the https://server:B there should be no activity.

Where does the window thing come (and its size)? Also, the size is alternating for some reason. Are there two sessions running simultaneously? I am using the -x option which I understood from the docs that it will kill the other sessions, so there should be only one active at any time.

[MisterTea]

How often does the job run? The window size is based on the size of the shell that called et. If it's done non-interactively, maybe the size is arbitrary?

[rahulraj80]

It is run from cron. This is the line in my crontab :

#@reboot 0 * * * * /usr/bin/et -x xxxxxx@yyyyyyyy:pppp -r 8xxx:10xxxx &
0 * * * * /usr/bin/et -x xxxxxx@yyyyyyyy:pppp -r 8xxx:10xxxx &

I am not sure how window size is defined when a job is run by the cron daemon.