My Minisign public key:
RWQLYkPbRQ8b56zEe8QdbjLFqC9UrjOaYxW5JxwsWV7v0ct/F/XfJlel
A sophisticated low memory handler for Linux
License: MIT License
My Minisign public key:
RWQLYkPbRQ8b56zEe8QdbjLFqC9UrjOaYxW5JxwsWV7v0ct/F/XfJlel
Jun 12 01:11:35 user-pc nohang[447]: ##################################################################
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.589
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.0
Jun 12 01:11:35 user-pc nohang[447]: PSI avg: 92.97 | MemAvail: 2039 M, 90.8 % | SwapFree: 4586 M, 92.8 % | dMem: 0 M/s
Jun 12 01:11:35 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.0 seconds
Jun 12 01:11:35 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:35 user-pc nohang[447]: Process with highest badness (found in 6 ms):
Jun 12 01:11:35 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:35 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.596
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.0
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.0 seconds
Jun 12 01:11:35 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 7 ms
Jun 12 01:11:35 user-pc nohang[447]: Total stat (what happened in the last 31 min 59 sec):
Jun 12 01:11:35 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:35 user-pc nohang[447]: victim badness < min_badness: 52
Jun 12 01:11:35 user-pc nohang[447]: ##################################################################
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.698
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.1
Jun 12 01:11:35 user-pc nohang[447]: PSI avg: 92.97 | MemAvail: 2039 M, 90.8 % | SwapFree: 4586 M, 92.8 % | dMem: 0 M/s
Jun 12 01:11:35 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.1 seconds
Jun 12 01:11:35 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:35 user-pc nohang[447]: Process with highest badness (found in 7 ms):
Jun 12 01:11:35 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:35 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.707
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.1
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.1 seconds
Jun 12 01:11:35 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 9 ms
Jun 12 01:11:35 user-pc nohang[447]: Total stat (what happened in the last 31 min 59 sec):
Jun 12 01:11:35 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:35 user-pc nohang[447]: victim badness < min_badness: 53
Jun 12 01:11:35 user-pc nohang[447]: ##################################################################
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.809
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.2
Jun 12 01:11:35 user-pc nohang[447]: PSI avg: 92.97 | MemAvail: 2039 M, 90.7 % | SwapFree: 4586 M, 92.8 % | dMem: -2 M/s
Jun 12 01:11:35 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.2 seconds
Jun 12 01:11:35 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:35 user-pc nohang[447]: Process with highest badness (found in 6 ms):
Jun 12 01:11:35 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:35 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.816
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.2
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.2 seconds
Jun 12 01:11:35 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 7 ms
Jun 12 01:11:35 user-pc nohang[447]: Total stat (what happened in the last 31 min 59 sec):
Jun 12 01:11:35 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:35 user-pc nohang[447]: victim badness < min_badness: 54
Jun 12 01:11:35 user-pc nohang[447]: ##################################################################
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.917
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.3
Jun 12 01:11:35 user-pc nohang[447]: PSI avg: 92.97 | MemAvail: 2039 M, 90.7 % | SwapFree: 4586 M, 92.8 % | dMem: 0 M/s
Jun 12 01:11:35 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.3 seconds
Jun 12 01:11:35 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:35 user-pc nohang[447]: Process with highest badness (found in 6 ms):
Jun 12 01:11:35 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:35 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 25.924
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.3
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.3 seconds
Jun 12 01:11:35 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 7 ms
Jun 12 01:11:35 user-pc nohang[447]: Total stat (what happened in the last 31 min 59 sec):
Jun 12 01:11:35 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:35 user-pc nohang[447]: victim badness < min_badness: 55
Jun 12 01:11:35 user-pc nohang[447]: ##################################################################
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 26.026
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.4
Jun 12 01:11:35 user-pc nohang[447]: PSI avg: 92.97 | MemAvail: 2039 M, 90.7 % | SwapFree: 4586 M, 92.8 % | dMem: 2 M/s
Jun 12 01:11:35 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.4 seconds
Jun 12 01:11:35 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:35 user-pc nohang[447]: Process with highest badness (found in 5 ms):
Jun 12 01:11:35 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:35 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 26.032
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.4
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.97) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.4 seconds
Jun 12 01:11:35 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 6 ms
Jun 12 01:11:35 user-pc nohang[447]: Total stat (what happened in the last 31 min 59 sec):
Jun 12 01:11:35 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:35 user-pc nohang[447]: victim badness < min_badness: 56
Jun 12 01:11:35 user-pc nohang[447]: ##################################################################
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_timer: 26.133
Jun 12 01:11:35 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:35 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:35 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:35 user-pc nohang[447]: psi_term_exceeded_timer: 66.5
Jun 12 01:11:35 user-pc nohang[447]: PSI avg: 92.25 | MemAvail: 2039 M, 90.7 % | SwapFree: 4586 M, 92.8 % | dMem: -2 M/s
Jun 12 01:11:35 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:35 user-pc nohang[447]: PSI avg (92.25) > sigterm_psi_threshold (90.0)
Jun 12 01:11:35 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.5 seconds
Jun 12 01:11:35 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:35 user-pc nohang[447]: Process with highest badness (found in 4 ms):
Jun 12 01:11:36 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:36 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_timer: 26.138
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:36 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:36 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: psi_term_exceeded_timer: 66.5
Jun 12 01:11:36 user-pc nohang[447]: PSI avg (92.25) > sigterm_psi_threshold (90.0)
Jun 12 01:11:36 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.5 seconds
Jun 12 01:11:36 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 5 ms
Jun 12 01:11:36 user-pc nohang[447]: Total stat (what happened in the last 32 min 0 sec):
Jun 12 01:11:36 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:36 user-pc nohang[447]: victim badness < min_badness: 57
Jun 12 01:11:36 user-pc nohang[447]: ##################################################################
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_timer: 26.239
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:36 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:36 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: psi_term_exceeded_timer: 66.6
Jun 12 01:11:36 user-pc nohang[447]: PSI avg: 92.25 | MemAvail: 2038 M, 90.7 % | SwapFree: 4586 M, 92.8 % | dMem: -1 M/s
Jun 12 01:11:36 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:36 user-pc nohang[447]: PSI avg (92.25) > sigterm_psi_threshold (90.0)
Jun 12 01:11:36 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.6 seconds
Jun 12 01:11:36 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:36 user-pc nohang[447]: Process with highest badness (found in 5 ms):
Jun 12 01:11:36 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:36 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_timer: 26.245
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:36 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:36 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: psi_term_exceeded_timer: 66.6
Jun 12 01:11:36 user-pc nohang[447]: PSI avg (92.25) > sigterm_psi_threshold (90.0)
Jun 12 01:11:36 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.6 seconds
Jun 12 01:11:36 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 5 ms
Jun 12 01:11:36 user-pc nohang[447]: Total stat (what happened in the last 32 min 0 sec):
Jun 12 01:11:36 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:36 user-pc nohang[447]: victim badness < min_badness: 58
Jun 12 01:11:36 user-pc nohang[447]: ##################################################################
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_timer: 26.346
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:36 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:36 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: psi_term_exceeded_timer: 66.7
Jun 12 01:11:36 user-pc nohang[447]: PSI avg: 92.25 | MemAvail: 2038 M, 90.7 % | SwapFree: 4586 M, 92.8 % | dMem: 1 M/s
Jun 12 01:11:36 user-pc nohang[447]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 12 01:11:36 user-pc nohang[447]: PSI avg (92.25) > sigterm_psi_threshold (90.0)
Jun 12 01:11:36 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.7 seconds
Jun 12 01:11:36 user-pc nohang[447]: Found 70 processes with existing /proc/[pid]/exe
Jun 12 01:11:36 user-pc nohang[447]: Process with highest badness (found in 5 ms):
Jun 12 01:11:36 user-pc nohang[447]: PID: 477, Name: Xorg, badness: 12
Jun 12 01:11:36 user-pc nohang[447]: Recheck memory levels...
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_timer: 26.353
Jun 12 01:11:36 user-pc nohang[447]: psi_post_action_delay_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: sigkill_psi_exceeded: False
Jun 12 01:11:36 user-pc nohang[447]: psi_kill_exceeded_timer: 0
Jun 12 01:11:36 user-pc nohang[447]: sigterm_psi_exceeded: True
Jun 12 01:11:36 user-pc nohang[447]: psi_term_exceeded_timer: 66.7
Jun 12 01:11:36 user-pc nohang[447]: PSI avg (92.25) > sigterm_psi_threshold (90.0)
Jun 12 01:11:36 user-pc nohang[447]: PSI avg exceeded psi_excess_duration (value = 40.0 sec) for 66.7 seconds
Jun 12 01:11:36 user-pc nohang[447]: victim badness 12 < min_badness 20; nothing to do; response time: 6 ms
Jun 12 01:11:36 user-pc nohang[447]: Total stat (what happened in the last 32 min 0 sec):
Jun 12 01:11:36 user-pc nohang[447]: Send SIGTERM to tail: 1
Jun 12 01:11:36 user-pc nohang[447]: victim badness < min_badness: 59
Jun 12 01:11:36 user-pc nohang[447]: ##################################################################
In my case there's a lot of memory-eager applications which sets themselves oom_score_adj=-1000 and therefore I keep getting xfce4-notifyd (with 25M VmRSS) killed first which occasionally gets an oom_score value of 1 instead of the process that actually ate all of my ram and triggered -m limits (but keeps its oom_score at 0).
I don't really know is that a common problem but anyway, in my opinion it'll be good to have an option to decide which process to kill based exclusively on VmRSS value
subj
subj
июл 03 23:16:44 PC nohang[6885]: Regexp '' matches with name 'tail'
июл 03 23:16:44 PC nohang[6885]: Execute the command: kill -SEGV 7026
июл 03 23:16:44 PC nohang[6885]: Exit status: 0; exe duration: 0.009 sec
июл 03 23:16:44 PC nohang[6885]: Implement a corrective action:
июл 03 23:16:44 PC nohang[6885]: Run the command: kill -SEGV 7026
июл 03 23:16:44 PC nohang[6885]: Exit status: 0; total response time: 77 ms
июл 03 23:16:44 PC nohang[6885]: Total stat (what happened in the last 9 min 22 sec):
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6895': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6925': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6971': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6997': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 7026': 1
июл 03 23:16:44 PC nohang[6885]: Execute the command: /usr/sbin/nohang_notify_helper --uid 0 --time 1562163404.0337765 &
июл 03 23:16:44 PC nohang[6885]: Exit status: 0; exe duration: 0.026 sec
июл 03 23:16:44 PC nohang[6885]: Implement a corrective action:
июл 03 23:16:44 PC nohang[6885]: Run the command: kill -SEGV 7026
июл 03 23:16:44 PC nohang[6885]: Exit status: 0; total response time: 77 ms
июл 03 23:16:44 PC nohang[6885]: success: True
июл 03 23:16:44 PC nohang[6885]: victim will die: None
июл 03 23:16:44 PC nohang[6885]: response_time: 0.10454392433166504 sec
июл 03 23:16:44 PC nohang[6885]: Process exited (VmRSS = 0) in 0.00046 sec
июл 03 23:16:44 PC nohang[6885]: The victim died in 0.58 sec
июл 03 23:16:44 PC nohang[6885]: Memory status after implementing a corrective action:
июл 03 23:16:44 PC nohang[6885]: MemAvailable: 3097.5 MiB, SwapFree: 7390.2 MiB
июл 03 23:16:44 PC nohang[6885]: Total stat (what happened in the last 9 min 23 sec):
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6895': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6925': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6971': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 6997': 2
июл 03 23:16:44 PC nohang[6885]: Run the command 'kill -SEGV 7026': 2
2019-05-26 23:02:21,140: Process with highest badness (found in 13245 ms):
TODO: Add in the documentation the conditions for the emergence of such behavior.
Fix it:
апр 25 23:11:06 PC nohang[31850]: The victim died in the search process: ValueError: 10
апр 25 23:11:06 PC nohang[31850]: The victim died in the search process: FileNotFoundError: 10
апр 25 23:11:06 PC nohang[31850]: ProcessLookupError (the victim died in the search process): : 10
A new detector.
Memory errors if vm.overcommit_memory=2: https://imgur.com/a/p9j67KA
Maybe useful if vm.overcommit_memory=2 to prevent MemoryError in innocent processes.
See also: https://www.kernel.org/doc/html/latest/vm/overcommit-accounting.html
overcommit_checking_enabled = False
soft_threshold_overcommit = 5%
hard_threshold_overcommit = 10%
/proc/meminfo:
CommitLimit: 25149116 kB
Committed_AS: 1909064 kB
vm.overcommit_memory=2, Committed_AS/CommitLimit = 50.0 %
/proc/sys/vm/overcommit_memory
GUI nonifications is broken in nohang-dev-desktop
re_processname -> BADNESS_ADJ_RE_NAME
cgroup_v1 -> cgroup_name_systemd
:name= -> :name=systemd
find_cgroup_mountpoints()
return c1_systemd_mountpoint, c2_mountpoint
improve pid_to_cgroup_v2
rename and improve pid_to_cgroup_v1
find_cgroup_mountpoints()
return c1_systemd_mountpoint, c2_mountpoint
pid_to_cgroup_v1_systemd_pid_list(pid)
return []
pid_to_cgroup_v2_pid_list(pid)
return []
' cgroup2 rw,'
',name=systemd 0 0'
nohang --cgroup-info
mountpoint + /system.slice/foo.service + /cgroup.procs
This piece of code is using undefined variables:
nohang:1439:54: F821 undefined name 'prefer_regex'
nohang:1440:54: F821 undefined name 'prefer_factor'
nohang:1441:54: F821 undefined name 'avoid_regex'
nohang:1442:54: F821 undefined name 'avoid_factor'
nohang:1447:54: F821 undefined name 'prefer_re_cmdline'
nohang:1448:54: F821 undefined name 'prefer_cmd_factor'
nohang:1449:54: F821 undefined name 'avoid_re_cmdline'
nohang:1450:54: F821 undefined name 'avoid_cmd_factor'
nohang:1455:54: F821 undefined name 'prefer_re_uid'
nohang:1456:54: F821 undefined name 'prefer_uid_factor'
nohang:1457:54: F821 undefined name 'avoid_re_uid'
nohang:1458:54: F821 undefined name 'avoid_uid_factor'
Also this import is not used anywhere:
nohang:1033:5: F401 'sre_constants' imported but unused
subj
The maximum PID that can be used on 64-bit linux systems is 2^22, can the padding in oom-sort
be increased to 7 chars for PID or determined from /proc/sys/kernel/pid_max
? Happy to submit a PR.
Нигде не нашел указание версии скрипта. Это может понадобится при создании пакета для пакетного менеджера. Можно сделать через git tag.
subj
nohang always try to run command instead of send SIGTERM even if execute_the_command = False
allow_fallback = False
fallback_mem = 1 M
fallback_swap = 1 M
fallback_zram = 90 %
In a critical situation: disable customization, quickly find a victim by oom_score
and send SIGKILL
.
subj
subj
TODO: add shared, buffers, cache, PSI status, zram status.
ChromiumOS kernel patches include /dev/chromeos-low-mem
pseudo-device
https://gitlab.freedesktop.org/seanpaul/dpu-staging/commit/0b992f2dbb044896c3584e10bd5b97cf41e2ec6d
(as far as I understood from the changelog, it was previously named /dev/low-mem
)
That file can be used to detect if system is in low memory condition or not.
Maybe it will be interesting to you. I've never used nohang frankly speaking, just wanted to give an idea...
Monitoring has started!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Memory status that requires corrective actions:
MemAvailable [0 MiB, 0.0 %] <= soft_threshold_min_mem [982 MiB, 10.0 %]
SwapFree [2168 MiB, 14.7 %] <= soft_threshold_min_swap [2210 MiB, 15.0 %]
Found 92 processes with existing /proc/[pid]/exe realpath
Process with highest badness (found in 6 ms):
PID: 12297, Name: tail, badness: 727
Recheck memory levels...
Memory status that requires corrective actions:
MemAvailable [0 MiB, 0.0 %] <= soft_threshold_min_mem [982 MiB, 10.0 %]
SwapFree [2163 MiB, 14.7 %] <= soft_threshold_min_swap [2210 MiB, 15.0 %]
Regexp '^tail$' matches with name 'tail'
Execute the command(1) in Thread-1: kill -TERM 12297
Implement a corrective action:
Run the command: kill -TERM 12297
Exit status: None; total response time: 7 ms
The victim doesn't respond on corrective action in 0.027 sec
Memory status after implementing a corrective action:
MemAvailable: 0.0 MiB, SwapFree: 2130.9 MiB
Total stat (what happened in the last 26 sec):
Run the command 'kill -TERM $PID': 1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
victim_cache_time is not exceeded for 4579503_pid12297 (0.082 < 10.0)
Memory status that requires corrective actions:
MemAvailable [0 MiB, 0.0 %] <= soft_threshold_min_mem [982 MiB, 10.0 %]
SwapFree [2073 MiB, 14.1 %] <= soft_threshold_min_swap [2210 MiB, 15.0 %]
New victim is cached victim 12297 (tail)
Recheck memory levels...
Memory status that requires corrective actions:
MemAvailable [0 MiB, 0.0 %] <= soft_threshold_min_mem [982 MiB, 10.0 %]
SwapFree [2073 MiB, 14.1 %] <= soft_threshold_min_swap [2210 MiB, 15.0 %]
max_soft_exit_time is not exceeded (0.1 < 10.0) for the victim
Command(1) execution completed in 0.128 sec; exit status: 0
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
victim_cache_time is not exceeded for 4579503_pid12297 (0.132 < 10.0)
Memory status that requires corrective actions:
MemAvailable [0 MiB, 0.0 %] <= soft_threshold_min_mem [982 MiB, 10.0 %]
SwapFree [2099 MiB, 14.2 %] <= soft_threshold_min_swap [2210 MiB, 15.0 %]
New victim is cached victim 12297 (tail)
Recheck memory levels...
Memory status that requires corrective actions:
MemAvailable [0 MiB, 0.0 %] <= soft_threshold_min_mem [982 MiB, 10.0 %]
SwapFree [2104 MiB, 14.3 %] <= soft_threshold_min_swap [2210 MiB, 15.0 %]
victim badness (0) < min_badness (10); nothing to do; response time: 0 ms
Total stat (what happened in the last 26 sec):
Run the command 'kill -TERM $PID': 1
victim badness < min_badness: 1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
Some rpmlint errors:
E: zero-length /etc/nohang/version
This empty file really necessary or it could be deleted safely?
/dev/shm/nohang/notify_cache
and remove it at exit
мая 26 23:02:22 user-VirtualBox nohang[530]: Memory status before implementing a corrective action:
мая 26 23:02:22 user-VirtualBox nohang[530]: MemAvailable: 1574.5 MiB, SwapFree: 546.0 MiB
мая 26 23:02:22 user-VirtualBox nohang[530]: Victim VmRSS: 3268608 KiB
мая 26 23:02:22 user-VirtualBox nohang[530]: Timer (value = 0.01 sec) expired; seems like the victim handles signal
мая 26 23:02:22 user-VirtualBox nohang[530]: Implement a corrective action:
мая 26 23:02:22 user-VirtualBox nohang[530]: Send SIGTERM to the victim; total response time: 13246 ms
todo: recheck thresholds before implementing corrective action.
@KILL_SESSION_AS_A_SINGLE_UNIT_RE_SID_NAME ^(sh|bash|zsh)$
TODO: Improve documentation
июн 19 22:06:22 pc systemd[1]: nohang.service: Service RestartSec=100ms expired, scheduling restart.
июн 19 22:06:22 pc systemd[1]: nohang.service: Scheduled restart job, restart counter is at 582.
июн 19 22:06:22 pc systemd[1]: Stopped Highly configurable OOM prevention daemon.
июн 19 22:06:22 pc systemd[1]: Started Highly configurable OOM prevention daemon.
июн 19 22:06:22 pc nohang[3204]: Config: /etc/nohang/nohang.conf
июн 19 22:06:22 pc nohang[3204]: Monitoring has started!
июн 19 22:06:22 pc nohang[3204]: Traceback (most recent call last):
июн 19 22:06:22 pc nohang[3204]: File "/usr/sbin/nohang", line 3075, in <module>
июн 19 22:06:22 pc nohang[3204]: swap_free, swap_total) = check_mem_swap_ex()
июн 19 22:06:22 pc nohang[3204]: File "/usr/sbin/nohang", line 1298, in check_mem_swap_ex
июн 19 22:06:22 pc nohang[3204]: if swap_total > swap_min_sigkill_kb:
июн 19 22:06:22 pc nohang[3204]: UnboundLocalError: local variable 'swap_min_sigkill_kb' referenced before assignment
июн 19 22:06:22 pc systemd[1]: nohang.service: Main process exited, code=exited, status=1/FAILURE
июн 19 22:06:22 pc systemd[1]: nohang.service: Failed with result 'exit-code'.
Positive @BANNESS_ADJ
settings should be ignored for processes with oom_score_adj
< 0.
subj
decrease_oom_score_adj = False
oom_score_adj_max = 0
->
ignore_positive_oom_score_adj = False
$respect_memory.oom.group = False
@kill_cgroup_v2_group_re ^/workload\.slice/
@kill_cgroup_v1_group_re ^/workload\.slice/
@kill_cgroup_v1_group_re ^/system\.slice/foo\.service$
If the process cgroup matches the specified one, then all processes with the same сgroup will be killed.
See https://t.me/devops_ru/580758 and further.
descendants_badness_adj = 300
It means:
The choice of the victim.
Search its descendants.
Descendants get increased badness.
Re-selection of the victim from the new pool - the first victim and her children with a new badness.
subj
TODO: new options
--target /system.slice
--period 2
--log-file ./psi-monitor.log
TODO: new output
2019-08-03 16:47:37,629: Starting psi-monitor, target: SYSTEM_WIDE, period: 2, log-file: ./psi-monitor.log
2019-08-03 16:47:37,629: ----------------------------------------------------------------------------------------------------------------
2019-08-03 16:47:37,629: some cpu pressure || some memory pressure| full memory pressure || some io pressure | full io pressure
2019-08-03 16:47:37,629: ---------------------||---------------------|----------------------||---------------------|---------------------
2019-08-03 16:47:37,629: avg10 avg60 avg300 || avg10 avg60 avg300 | avg10 avg60 avg300 || avg10 avg60 avg300 | avg10 avg60 avg300
2019-08-03 16:47:37,629: ------ ------ ------ ||------ ------ ------ | ------ ------ ------ ||------ ------ ------ | ------ ------ ------
2019-08-03 16:47:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:49:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:47:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:49:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:47:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:49:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:47:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:49:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:47:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:49:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:47:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
2019-08-03 16:49:37,629: 44.29 28.54 20.08 || 44.29 28.54 20.08 | 43.42 27.88 19.43 || 44.29 28.54 20.08 | 43.42 27.88 19.43
Installing latest git master (c4f6a66) fails for me (Ubuntu 18.04)
$ sudo make install
install -d /usr/local/bin
install -m0755 nohang /usr/local/bin/nohang
install -m0755 nohang_notify_helper /usr/local/bin/nohang_notify_helper
install -m0755 oom-sort /usr/local/bin/oom-sort
install -m0755 psi-top /usr/local/bin/psi-top
install -m0755 psi-monitor /usr/local/bin/psi-monitor
install -d /etc/nohang
git describe --tags --long --dirty > version
install -m0644 version /etc/nohang/version
rm -fv version
removed 'version'
install -m0644 nohang.conf /etc/nohang/nohang.conf
install -m0644 nohang.conf /etc/nohang/nohang.conf.default
install -d /etc/logrotate.d
install -m0644 nohang.logrotate /etc/logrotate.d/nohang
install -d /usr/share/man/man1
gzip -c nohang.1 > /usr/share/man/man1/nohang.1.gz
gzip -c oom-sort.1 > /usr/share/man/man1/oom-sort.1.gz
install -d /etc/systemd/system
sed "s|:TARGET_BIN:|/usr/local/bin|g;s|:TARGET_CONF:|/etc|g" nohang.service.in > nohang.service
install -m0644 nohang.service /etc/systemd/system/nohang.service
rm -fv nohang.service
removed 'nohang.service'
chcon -t systemd_unit_file_t /etc/systemd/system/nohang.service
chcon: can't apply partial context to unlabeled file '/etc/systemd/system/nohang.service'
Makefile:12: recipe for target 'install' failed
make: [install] Error 1 (ignored)
I guess the problem is that
File context can be temporarily modified with the chcon command. If you want to permanantly change the file context you need to use the semanage fcontext command. This will modify the SELinux labeling database. You will need to use restorecon to apply the labels.
-- https://www.systutorials.com/docs/linux/man/8-system_selinux/
Jun 07 05:54:28 user-pc nohang[3047]: PSI avg: 85.81 | MemAvail: 2035 M, 90.6 % | SwapFree: 4636 M, 93.8 % | dMem: -1 M/s
Jun 07 05:54:28 user-pc nohang[3047]: psi_post_action_delay_exceeded: True
Jun 07 05:54:28 user-pc nohang[3047]: sigkill_psi_exceeded: False
Jun 07 05:54:28 user-pc nohang[3047]: psi_kill_exceeded_timer: 0
Jun 07 05:54:28 user-pc nohang[3047]: sigterm_psi_exceeded: True
Jun 07 05:54:28 user-pc nohang[3047]: psi_term_exceeded_timer: 159.4
Jun 07 05:54:28 user-pc nohang[3047]: min_delay_after_sigterm IS EXCEEDED, it is time to action
Jun 07 05:54:28 user-pc nohang[3047]: PSI avg (85.81) > sigterm_psi_threshold (60.0)
Jun 07 05:54:28 user-pc nohang[3047]: PSI avg exceeded psi_excess_duration (value = 30.0 sec) for 159.4 seconds
Jun 07 05:54:28 user-pc nohang[3047]: Found 66 processes with existing /proc/[pid]/exe
Jun 07 05:54:28 user-pc nohang[3047]: Process with highest badness (found in 7 ms):
Jun 07 05:54:28 user-pc nohang[3047]: PID: 482, Name: Xorg, badness: 11
Jun 07 05:54:28 user-pc nohang[3047]: Thresholds is not exceeded now
Jun 07 05:54:28 user-pc nohang[3047]: psi_post_action_delay_exceeded: False
Jun 07 05:54:28 user-pc nohang[3047]: sigkill_psi_exceeded: False
Jun 07 05:54:28 user-pc nohang[3047]: psi_kill_exceeded_timer: 0
Jun 07 05:54:28 user-pc nohang[3047]: sigterm_psi_exceeded: True
Jun 07 05:54:28 user-pc nohang[3047]: psi_term_exceeded_timer: 159.4
For me as casual user, is more conveniently to choose process which should be killed. It is difficult to constantly update the priorities of processes in the config, especially since various situations may arise that will be decided not in favor of the user.
I think it's better to show list of worst processes and possibility to pass SIGTERM, SIGKILL to one or many of them. If the indicator of the load on the subsystem reaches critical values, then the process is automatically killed, as it works now.
июн 15 03:27:39 pc nohang[5282]: MemAvail: 9335 M, 95.2 % | SwapFree: 1682 M, 82.1 % | dMem: -2 M/s
июн 15 03:27:42 pc nohang[5282]: MemAvail: 9327 M, 95.1 % | SwapFree: 1686 M, 82.3 % | dMem: -1 M/s
июн 15 03:27:45 pc nohang[5282]: MemAvail: 9321 M, 95.0 % | SwapFree: 1690 M, 82.5 % | dMem: -1 M/s
июн 15 03:27:46 pc nohang[5282]: nohang_notify_helper: wait_time: 8
июн 15 03:27:46 pc nohang[5282]: Send GUI notification: Low memory MemAvail: 0%
июн 15 03:27:46 pc nohang[5282]: SwapFree: 50% ('user', 'DISPLAY=:0', 'DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus')
июн 15 03:27:46 pc nohang[5282]: Send GUI notification: Low memory MemAvail: 0%
июн 15 03:27:46 pc nohang[5282]: SwapFree: 50% ('root', 'DISPLAY=:0', 'DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-7XOX46CH5b,guid=c22809dd399138da090795805d03d826')
июн 15 03:27:46 pc nohang[5282]: TimeoutExpired: notify user: user
июн 15 03:27:46 pc nohang[5282]: TimeoutExpired: notify user: root
июн 15 03:27:46 pc nohang[5282]: nohang_notify_helper: wait_time: 8
июн 15 03:27:46 pc nohang[5282]: Send GUI notification: Low memory MemAvail: 0%
июн 15 03:27:46 pc nohang[5282]: SwapFree: 17% ('root', 'DISPLAY=:0', 'DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-7XOX46CH5b,guid=c22809dd399138da090795805d03d826')
июн 15 03:27:46 pc nohang[5282]: Send GUI notification: Low memory MemAvail: 0%
июн 15 03:27:46 pc nohang[5282]: SwapFree: 17% ('user', 'DISPLAY=:0', 'DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus,guid=7625e2f7cd8770020fae5d565d03d64d')
июн 15 03:27:46 pc nohang[5282]: TimeoutExpired: notify user: root
июн 15 03:27:46 pc nohang[5282]: TimeoutExpired: notify user: user
TODO: exclude root
incorrect badness value in log, will be fixed later
~# nohang -v hh
Unknown option: -v
У вас в конфиге https://github.com/hakavlad/nohang/blob/master/nohang.conf#L217 задается $DISPLAY
, но вот здесь b1a6565 вроде бы DISPLAY берется из ps ae
.
I'd rather have nohang
kill a single Firefox tab instead of the whole browser. Sadly, both processes have the same name
, so I can't use prefer_regex
for that.
If nohang
would give me some way to also match the cmdline-params (can get those from /proc/$pid/cmdline
), I would be able to give processes with an -childID
"preferred" treatment :)
Not sure if this should require extra config-options (enable_cmdline
, prefer_cmdline_regex
,...) are just be included in the standard behaviour.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.