akkana / scripts Goto Github PK

(this is not really an issue)

First, thanks for the blogpost about termsize, it got me started on some yak shaving I didn't intend to do today... :-)

While reading your post I figured there should be a way to do it without python at all.

Here's a way to do it, I thought you might be interested.

#!/bin/bash
set -eu

# Reset term to initial state on exit
stty_ori=$(stty -g)
function atexit {
    stty $stty_ori
}
trap atexit EXIT

# Turn echo off to avoid writing term size on the screen
stty -echo

# Use bash's "read" to send the control seq to the term and read its answer
# 7: save cursor pos
# [r: scroll screen
# [x;yH: set cursor pos
# [6n: get cursor pos
read -p $'\e7\e[r\e[9999;9999H\e[6n' -d R termsize

# Strip leading control char + [ (e.g. \e[x;yH)
termsize=${termsize#??}

# Reset cursor pos
echo -en '\e8'

# Actually resize the terminal
stty $(echo $termsize  | sed 's/^/rows /; s/;/ cols /')

Also, busybox has resize(1). (which I discovered after writing the above...)
https://git.busybox.net/busybox/tree/console-tools/resize.c#n85

No filename; making one up                                                                                                                                                                                                         
/home/yoh/.tmp/tmprz_ti1ww/part-001.txt doesn't have a Content-Id, not saving
No filename; making one up
/home/yoh/.tmp/tmprz_ti1ww/part-001.html doesn't have a Content-Id, not saving
Traceback (most recent call last):
  File "/home/yoh/.mutt/soft/viewhtmlmail", line 183, in <module>
    view_html_message(stdin, tmpdir)
  File "/home/yoh/.mutt/soft/viewhtmlmail", line 150, in view_html_message
    htmlsrc = html_part.get_payload(decode=True).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 28462: invalid continuation byte

not sure if that email doesn't have any sensitive information, so will now send it via email to the address in git commits.

PS THANK YOU for the viewhtml -- I had been using it for years!

Firstly, thank you so much for maintaining viewhtmlmail. This is an essential part of using (neo)mutt for me! Very much appreciated.

I've noticed for certain characters, the script decodes the unicode. That is, the output contain strings such as \u201c. I compared these emails to the original raw html from these emails. Everything is identical (including headers), except these characters are modified. I have replicated this issue with the provided test file htmlmail.eml, adding a few characters as per the following.

--- /tmp/htmlmail1.eml	2022-10-17 13:42:30.306089285 +1100
+++ /tmp/htmlmail2.eml	2022-10-17 13:45:40.435774500 +1100
@@ -72,7 +72,7 @@
 hing about pi=C3=B1on trees=E2=80=94and em and en=E2=80=93dashes.</div><div=
 ><br></div><div>And an image!<br><div><img src=3D"cid:ii_kh9ezunp0" alt=3D"=
 tuxnetwork.jpg" width=3D"256" height=3D"187"><br></div><div>and some text a=
-fter the image. And an emoji! =F0=9F=98=80<br></div></div><div><br>-- <br><=
+fter the image. And an emoji! =F0=9F=98=80 And “quotes” at 37°C<br></div></div><div><br>-- <br><=
 div dir=3D"ltr" class=3D"gmail_signature" data-smartmail=3D"gmail_signature=
 ">=C2=A0 =C2=A0 ...Akkana</div></div></div>

The original html has

And “quotes” at 37°C

and the version that viewhtmlmail produces has

And \u201cquotes\u201d at 37�C

The quotes have decoded, and the ° turns into unicode U+FFFD REPLACEMENT CHARACTER.

Thank you for the updated version of viewhtmlmail.py. It's working much more smoothly now!

I just had some questions about the options sent to Firefox. Previously, viewhtmlmail.py would open the email in a new tab in Firefox, but now it opens the Firefox profile selector. I assume that's because it's attempting to run firefox -P Default -private-window, but I don't have a profile named Default. Instead, if I change the arguments to [ "-new-tab" ], this seems to mimic the old behaviour.

Also, I can see that the script tests a few other browsers as well. I get the following errors on the CLI

**** Couldn't run qutebrowser ! [Errno 2] No such file or directory: 'qutebrowser'
**** Couldn't run quickbrowse ! [Errno 2] No such file or directory: 'quickbrowse'

Do you think it might be good to suppressing these errors, since the script is merely testing for their presence? IMO they aren't really "errors" per se.

Currently these are the changes I've made locally to fix these two issues for me.

--- a/viewhtmlmail.py	2022-12-05 10:27:32.924437318 +1100
+++ b/viewhtmlmail.py	2022-11-25 13:56:05.791325129 +1100
@@ -86,15 +86,14 @@
         'CONVERT_PDF_TO_HTML': True
     }),
     ('firefox', {
-        'ARGS_FIRST': [ "-P", "Default", "-private-window",
-                        "-new-instance" ],
+        'ARGS_FIRST': [ "-new-tab" ],
         'ARGS': [  "-P", "Default", "-private-window" ],
         'BACKGROUND': True,
         'CONVERT_PDF_TO_HTML': False,
     })
 ])
 
-WORKING_BROWSER = None
+WORKING_BROWSER = 'firefox'
 
 # First call to a browser?
 first_browser = True

Your wonderful viewhtmlmail script chokes if mutt is using Maildir-formatted mailboxes. The fix is straightforward. If "fp = open(mbox)" fails because the mbox is actually a directory, then just do the following to read (and delete) the message:

maildir = mailbox.Maildir(mbox, factory=None)
key, msg = maildir.popitem()

Not an issue, I just wanted to thank you for the awesome code, and also wanted to let you know that I ported your code as an interactive iPython notebook that users can now run online:

https://github.com/Christophe-Gauge/NEOWISE/blob/master/comet.ipynb

I changed the start and end dates and it crashes almost immediately:

$ python3 conjunctions.py
Looking for planetary events between 2018-01-01 and 2021-01-01:

Traceback (most recent call last):
  File "conjunctions.py", line 658, in <module>
    run(start, end, observer, toolate, output_format)
  File "conjunctions.py", line 556, in run
    observer.date, sep)
  File "conjunctions.py", line 298, in add
    self.clist.delete(cc)
AttributeError: 'list' object has no attribute 'delete'

Yes, python3 has a "get_terminal_size" feature... But it relies on the exact same stty settings that are usually wrong for a serial console, as it just checks stty settings.

The premise of this script is that you suspect the stty settings of being wrong, because you're using a terminal which implicitly sets terminal size to 24x80, and you want to override that.

Suggested add, to remove blank line before 'Reset...' text...

ttyfd.write(b'\033[1A')
print("\nReset Terminal, Rows %d, Columns %d" % (rows, cols))

This is just an escape sequence to move the cursor up 1 line. Saving the cursor location and restoring it would also work. The real issue is that the write to screen (buffer) even with the echo turned off, is still changing the cursor location.

If you run termsize in urxvt, you see:

$ ./termsize

eset the terminal to 32 rows, 80 cols
^[[32;80R%
$ ;80R
zsh: command not found: 80R

The R in Reset is missing, and the escape sequence following it is not doing whatever it's supposed to be doing and the last part of it gets inserted into the next command as if it were typed.

Hey, I'd like to use your class for python 3 and with block=false. However, if I change the shebang to
#!/usr/bin/env python3
and the main initializer to
keyreader = KeyReader(echo=False, block=False)
upon calling the Keyreader.py from command prompt I receive an error

johndoe@raspberrypi:~/git/raspberry/rolladen$ KeyReader.py
Traceback (most recent call last):
File "./KeyReader.py", line 104, in
main()
File "./KeyReader.py", line 93, in main
key = keyreader.getch()
File "./KeyReader.py", line 84, in getch
return sys.stdin.read()
File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
TypeError: a bytes-like object is required, not 'NoneType'

Hi!
Could you give me an advice how to install a package 'python-poppler-qt5' in python3.65? It says: 'Failed to determine Qt version ([WinError 2] The system cannot find the file specified).' I have Qt 5.12 in Windows installed, PyQt5.11.3 and PyQt5-sip 4.19.13.

latest version does not work for me, tried both debian 11.3 and RHEL 8.6, serial console

for some reason getmaxyx() always returns 80x25

if it is working for you maybe rewrite it so if you get 80x25 from getmaxyx() fall back to non curses version as if someone had 80x25 terminal, he wouldn't be using termsize in the first place ;)

As pointed out in blog post https://shallowsky.com/blog/tech/web/finding-web-orphans.html orphaned files on a website are a common issue with very few solutions available.

Therefore I tried to run the script with python 3.10.0 on WSL 1.0.4 Debian 5.15.79.1-microsoft-standard-WSL2
After a few minor changes it ran up to below error.
I'm a beginner with python therefore it's highly likely I did something wrong or is it maybe a bug?

/mnt/c/Users/mholzer/Documents/git/rusefi_documentation$ python3.10 -d ./wiki-tools/weborphans.py /mnt/c/Users/mholzer/Documents/git/rusefi_documentation/ https://wiki.rusefi.com/ >./wiki-tools/weborphans_wiki.rusefi.com
.log
Traceback (most recent call last):
  File "/mnt/c/Users/mholzer/Documents/git/rusefi_documentation/./wiki-tools/weborphans.py", line 355, in <module>
    spider.spide()
  File "/mnt/c/Users/mholzer/Documents/git/rusefi_documentation/./wiki-tools/weborphans.py", line 106, in spide
    self.check_url(self.starturl)
  File "/mnt/c/Users/mholzer/Documents/git/rusefi_documentation/./wiki-tools/weborphans.py", line 239, in check_url
    url = urlparse.urlunparse(lurl)
AttributeError: 'function' object has no attribute 'urlunparse'

the modified code & output:
weborphans.zip