selectel / pyte

Simple VTXXX-compatible linux terminal emulator

               _
              | |
 _ __   _   _ | |_  ___
| '_ \ | | | || __|/ _ \
| |_) || |_| || |_|  __/
| .__/  \__, | \__|\___|
| |      __/ |
|_|     |___/      0.8.2

What is pyte?

It's an in memory VTXXX-compatible terminal emulator. XXX stands for a series of video terminals, developed by DEC between 1970 and 1995. The first, and probably the most famous one, was VT100 terminal, which is now a de-facto standard for all virtual terminal emulators. pyte follows the suit.

So, why would one need a terminal emulator library?

• To screen scrape terminal apps, for example htop or aptitude.
• To write cross platform terminal emulators; either with a graphical (xterm, rxvt) or a web interface, like AjaxTerm.
• To have fun, hacking on the ancient, poorly documented technologies.

Note: pyte started as a fork of vt102, which is an incomplete pure Python implementation of VT100 terminal.

Installation

If you have pip you can do the usual:

pip install pyte

Otherwise, download the source from GitHub and run:

python setup.py install

Similar projects

pyte is not alone in the weird world of terminal emulator libraries, here's a few other options worth checking out: Termemulator, pyqonsole, webtty, AjaxTerm and of course vt102.

pyte's Issues

Handling of backspace/delete

After some testing, I have noticed that handling of backspace/delete in Stream does not seem to be perfect.

My terminal emulators generally support these 4 options for backspace and delete:

• CTRL-H - \x08
• ASCII DEL - \x7f
• ESCAPE - \x1b[3~
• TTY ERASE - becomes ASCII DEL

Many terminals only have settings to go from ASCII DEL to CTRL-H for compatibility reasons. This option makes Emacs sad, but is sometimes required. The escape sequence appears to be the only sequence resulting in forward-delete behaviour in my shell (Although it appears broken when I ssh through tmux). Neither the ASCII del char, nor the escape sequence is handled by pyte's streams. The del char never seems to raise any events, with the escape sequence raising a debug event.

I will most likely fork it later today to add DEL to control and escape and submit a pull request, unless I forget or someone else manage to do it before me.

UPDATE:
The issue lies with ASCII DEL (\x7f) being explicitly ignored. The forward delete thing is under another issue, and closed for now. As per a comment from @superbobry this was likely to fix Midnight Commander, but I believe that it might be due to incorrect handling. \x7f is just a whitespace rendition-wise, cursor-increment and all. It's only magic when it arrives on stdin from a terminal emulator, which it tends to do quite often.

Control characters inside ESC sequences

According to linux/drivers/tty/vt.c: «Control characters can be used in the _middle_of an escape sequence.» -- since we claim to support TERM=linux we should allow that as well.

Current status: works for CSI sequences but still doesn't for ESC # ....

DiffScreen.draw() marks wrong line as dirty when wrapping occurs

The current implementation of DiffScreen.draw() in pyte/screens.py as at 12276a8 is that it first marks the row containing the cursor as dirty and then calls Screen.draw().

However, the latter method will, if wrapping is enabled (if mo.DECAWM in self.mode), move to the next line before actually writing the character to the virtual terminal. This means that the wrong line might have been marked as dirty.

It seems to me that simply switching the order of the statements in DiffScreen.draw() would fix this issue.

Weird wrapping

If I write 90 characters to a 80 columns width screen:

mport pyte
stream = pyte.Stream();screen = pyte.Screen(80, 24);stream.attach(screen)
stream.feed('a' * 90)                                                                                                                               
for idx, line in enumerate(screen.display, 1): print("{0:2d} {1} ¶".format(idx, line))

The wrapping becomes really weird. I don't get it from the code...

 1 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ¶
 2                                                                                a ¶
 3                                                                                a ¶
 4                                                                                a ¶
 5                                                                                a ¶
 6                                                                                a ¶
 7                                                                                a ¶
 8                                                                                a ¶
 9                                                                                a ¶
10                                                                                a ¶
11                                                                                a ¶
12                                                                                  ¶
13                                                                                  ¶
14                                                                                  ¶
15                                                                                  ¶
16                                                                                  ¶
17                                                                                  ¶
18                                                                                  ¶
19                                                                                  ¶
20                                                                                  ¶
21                                                                                  ¶
22                                                                                  ¶
23                                                                                  ¶
24                                                                                  ¶ 

Thank you.

Not an issue, but a thank you. (can't find a contact button ;) )

Pyte is wonderful! I started this weekend a tmux clone in pure Python. Pyte actually already did half the amount of my work.

I have a working prototype here which opens a bash shell and vim editor next to each other: https://github.com/jonathanslenders/python-vterm/blob/master/pymux.py

So, maybe some pull requests from my side will follow when the project goes on. If not, that means that pyte is just perfect.

Cheers,
Jonathan

p.s.: you can close this 'issue'.

Can Pyte be used as dropin to communicate with another tool that sometimes sends terminal comamnd characters?

Hi,

I've got the following situation and after reading the docs I'm not sure if or if not pyte could be of help. I'd appreciate to get your opinion about it.

The situation is that I use pexpect to communicate via ssh remotely to another computer. To implement the ssh connection pexpect interally uses a pty for the interaction. That design decision leads to the user (me) to receiving terminal command chars in some situations. But actually I don't care about the command characters at all. All I want is to receive the text that the corresponding command prints to stdout on the remote device. In the linked example above all I want is what ls is telling me (and sometimes its returncode is also interesting).

Now I wonder if it would be possible to plug something like pyte on top of the pexpect API with the intention to simply let pyte ignore all these characters and filter them out of my output streams. I've literally spend months on manual fitlering already but I get so many different characters by now, that I can't handle it by myself any more (see linked issue in pexpect for an example).

I'm really really curious about your opinion. Also if you can suggest some websites or books that help me become more skilled in acting like a terminal, that would be great. I bet a real solution that is not a workaround can only be possible if I learn some more about terminals. But everything I've found was only part of the puzzle. It seems terminals do lots of things because of historical reasons that are not all documented in some place (or at least some place that can be found via google by the uneducated).

Broken Help

Я установил вашу билиотеку через easy_install.
Вот что случается при попытке получить help:

$ python
Python 2.7.3 (default, Aug  1 2012, 05:14:39) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyte
>>> help(pyte)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site.py", line 469, in __call__
    return pydoc.help(*args, **kwds)
  File "/usr/lib/python2.7/pydoc.py", line 1747, in __call__
    self.help(request)
  File "/usr/lib/python2.7/pydoc.py", line 1794, in help
    else: doc(request, 'Help on %s:')
  File "/usr/lib/python2.7/pydoc.py", line 1531, in doc
    pager(render_doc(thing, title, forceload))
  File "/usr/lib/python2.7/pydoc.py", line 1526, in render_doc
    return title % desc + '\n\n' + text.document(object, name)
  File "/usr/lib/python2.7/pydoc.py", line 329, in document
    if inspect.ismodule(object): return self.docmodule(*args)
  File "/usr/lib/python2.7/pydoc.py", line 1104, in docmodule
    result = result + self.section('CLASSES', join(contents, '\n'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 623: ordinal not in range(128)

Это баг или просто хелпа нет?

Incorrect Mapping for Char class to _Char NamedTuple

I have run into a problem with the mapping of the Char class to the _Char named tuple. Reverse and Strike-through end up reversed.

Here is a diff with the patch and tests:

diff --git a/pyte/screens.py b/pyte/screens.py
index eb69b4f..ede2a93 100644
--- a/pyte/screens.py
+++ b/pyte/screens.py
@@ -84,7 +84,7 @@ class Char(_Char):
                 italics=False, underscore=False, reverse=False,
                 strikethrough=False):
         return _Char.__new__(cls, data, fg, bg, bold, italics, underscore,
-                             reverse, strikethrough)
+                             strikethrough, reverse)


 class Cursor(object):
diff --git a/tests/test_screen.py b/tests/test_screen.py
index 4cd7ddc..b21144d 100644
--- a/tests/test_screen.py
+++ b/tests/test_screen.py
@@ -34,6 +34,23 @@ def update(screen, lines, colored=[]):

 # Tests.

+def test_char_class_flags():
+    char = Char("B", bold=True)
+    assert char.bold == True
+
+    char = Char("I", italics = True)
+    assert char.italics == True
+
+    char = Char("U", underscore= True)
+    assert char.underscore == True
+
+    char = Char("S", strikethrough = True)
+    assert char.strikethrough == True
+
+    char = Char("R", reverse = True)
+    assert char.reverse == True
+
+
 def test_remove_non_existant_attribute():
     screen = Screen(2, 2)
     assert screen.buffer == [[screen.default_char, screen.default_char]] * 2

Crashes when parsing data which is not valid UTF-8

janus@zeus ~/pyte/examples % PYTHONPATH=.. python3 capture.py cat /dev/urandom
Traceback (most recent call last):
  File "capture.py", line 43, in <module>
    stream.feed(os.read(fd, 1024).decode())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9f in position 1: invalid start byte

This is a simple but contrived example. But you get the same problem with e.g. Shift-JIS content:

PYTHONPATH=.. python3 capture.py curl \
http://www.rikai.com/library/kanjitables/kanji_codes.sjis.shtml -H 'User-Agent: Mozilla/9'

Update the pyte distributed through pip

The pip version of pyte giving error.(0.5.2)
stream = pyte.Stream(screen)
TypeError: init() takes exactly 1 argument (2 given)

The repository version is working fine(0.6.0). Please update the pyte in pip.

set_margins() got an unexpected keyword argument 'private'

testing latest master branch and python 2.7

Using

		self.screen = pyte.Screen(self.term_cols, self.term_rows)
		self.stream = pyte.ByteStream()
		self.stream.attach(self.screen)

When I exit out of a midnight commander screen mc I get an error reported by stream.feed(buf) , where
buf is the below :

[4;2H
[37m
[44m/..                                    
[37m
[44m
[37m
[44mUP--DIR
[37m
[44m
[37m
[44mMar  1 05:06
[5;2H
[30m
[46m/.cache                                
     15
Mar  1 17:28
[26;2H
[37m
[44m/.cache
[29;17H
[5;2H
[37m
[44m/.cache                                
[37m
[44m
[37m
[44m     15
[37m
[44m
[37m
[44mMar  1 17:28
[6;2H
[30m
[46m/.config                               
     15
Mar  1 17:28
[26;5H
[37m
[44monfig
[29;17H
[6;2H
[37m
[44m/.config                               
[37m
[44m
[37m
[44m     15
[37m
[44m
[37m
[44mMar  1 17:28
[7;2H
[30m
[46m/.local                                
     18
Mar  1 17:28
[26;4H
[37m
[44mlocal 
[29;17H
[7;2H
[37m
[44m/.local                                
[37m
[44m
[37m
[44m     18
[37m
[44m
[37m
[44mMar  1 17:28
[8;2H
[30m
[46m .bash_history                         
    430
Mar  1 18:35
[26;2H
[37m
[44m .bash_history
[29;17H
[8;2H .bash_history                         
    430
Mar  1 18:35
[9;2H
[30m
[46m .bash_logout                          
     18
Mar  1 05:06
[26;9H
[37m
[44mlogout 
[29;17H
[9;2H .bash_logout                          
     18
Mar  1 05:06
[10;2H
[30m
[46m .bash_profile                         
    193
Mar  1 05:06
[26;9H
[37m
[44mprofile
[29;17H
[10;2H .bash_profile                         
    193
Mar  1 05:06
[12;2H
[30m
[46m afile                                 
     56
Mar  1 18:36
[26;3H
[37m
[44mafile        
[29;17H
[9;35H
[30m
[47m                                                        
[10;35H 
[34m The Midnight Commander 
[30m
[11;35H 
 Do you really want to quit the Midnight Commander? 
[12;35H 
[13;35H 
                  
[46m[ 
[34mY
[30mes ]
[47m  [ 
[34mN
[30mo ]                   
[14;35H 
[15;35H                                                        
[13;57H
[28;7H
[39m
[49mBring text back from the dead with C-y.
[28;124H
[1;1H
[1;124H
[?1015l
[?1002l
[?1001r
[?1l
[30;1H
[39;49m
[?1049l
[?47l
[39;49m
]0;anazmy@db1:~
[anazmy@db1 ~]$ ls -alSh

UTF-8 character incorrectly interpreted as OSC

When trying to upgrade my project to use the current git master, I needed to send data as bytes.
This works, however I can't decode UTF-8 data. For example, the following test script will fail:

import pyte
screen = pyte.Screen(10, 1)
stream = pyte.Stream(screen)
c = b"\xE2\x80\x9D" #u+201d
stream.feed(c)
print(screen.display)

The first character of the screen should be '”', but is blank.

It looks like pyte is interpreting the 9d as OSC, causing problems from there.
According to Wikipedia articles on ANSI escape codes and C0 and C1 control codes, in UTF-8 mode, OSC (also ST, CSI) should be encoded in UTF-8, so in this case 0xc2 0x9d.

Please create 0.4.10 release

I hope this isn't too much to ask, but would you mind creating a 0.4.10 release? I'd like to deploy some code and be able to use an official release that includes my pull requests. It's no big deal if I have to do a release from my fork but it's just not as nice. Thanks in advance!

Zero-width characters

The current behaviour regarding zero-width/combining characters is to append them to the char in the preceding cell. If there is no preceding cell -- nothing is printed (as reported by @chubin in #65). This sounds reasonable to me and I'd be happy to just leave that be, but other terminals handle these characters differently, e.g. iTerm simply prints the combining character on its own.

$ echo "\u0483"
 ҃

If you have an issue with pyte behaviour, please comment in this issue.

would it be possible to publish also a wheel on pip ?

python setup.py sdist bdist_wheel  --universal
twine upload dist/*

License

Thank you for this library! I'm using with https://github.com/wraithan/pytmux.

I assume it's your intention by making this LGPL to simply give pyte-related contributions back to the project. Most python projects are permissively licensed (ISC, MIT, BSD, Apache) since it's not a practice distribute binary blobs. The language used in LGPL however doesn't protect edge cases that exist in interpreted scripting languages, like subclassing, monkey-patching, mixing in, vendorizing (copying the code into a project), copying and pasting snippets.

If you could put it under MIT/BSD/Apache that would be a lot nicer downstream.

Behaviour of linefeed above scrolling region

Assume the scrolling region is set from line 3 to 27 (VT-100 coordinates).
If I send CSI H to bring the cursor to 1, 1, then send a linefeed, I think the cursor should be at 2, 1.
However, it moves to 3, 1.

If I run the following in the shell, with screen:

clear;echo -ne '\x1b[3;27r\x1b[H\n';read

Then press CTRL-A i to check the cursor position, it shows column 2, line 1 as I expected.

However, the same sequence with pyte:

stream.feed(b'\x1b[3;27r\x1b[H\n')
print(screen.cursor.y, screen.cursor.x)

returns (2, 0) -- line 3 column 1 accounting for the 0-based offsets.

I'm not sure who's correct here, except it looks like this is corrupting my nano display if I run ssh -> screen -> nano.

Put buffer manipulations in screens.py in utility functions.

Hi,

For my project, I need to create a custom Screen class that doesn't use the current "buffer" as a storage. More specific, I need a data structure that is able to do line reflow and such, and per-character diffs (instead of per lines).

The problem is that I still want to inherit from pyte.Screen because a lot of functionality could be reused. But for some functions, I have to copy paste too much, because there's one line in there that accesses the self.buffer variable.

Changes that I'd currently like to see:

• In set_mode and reset_mode, use a utility function self._mark_all_reverse and self._unmark_all_reverse.
• In draw, the line where we access buffer. Use self._set_char(x, y, char).
• Mabe a few other minor places.

If you'd like to, I can always create a pull request.
I think this can contribute to better code reusability.

Cheers,
Jonathan

Revise ByteStream API

The current implementation of ByteStream is broken.

Firstly, it maintains a list of decoders each with its own state. I doubt that intermixed outputs (each with a different encoding) is a realistic scenario. Secondly, it never finalizes the decoders (as discussed in #48).

(FYI) pymux

Hi all,

Thanks a lot for making Pyte! It allowed me to create pymux: https://github.com/jonathanslenders/pymux

There are a few changes I had to do to the "stream" class. For some of them, I think it make sense to merge them back into Pyte. Please, have a look to this file:
https://github.com/jonathanslenders/pymux/blob/master/pymux/stream.py
I needed support for CPR, device attributes, and terminal title support. Further, the hasattr calls in "dispatch" were not acceptable.

Probably, in the future, I'll do some more changes in order to improve the performance. Let me know whether you're interested.

The screen class is completely custom made: https://github.com/jonathanslenders/pymux/blob/master/pymux/screen.py

Jonathan

Disappearing chrs

Was doing a small test for an app I'm working on

# -*- coding: utf-8 -*-
from __future__ import print_function, unicode_literals

from pyte import ByteStream,Screen,modes as mo


if __name__ == "__main__":
    screen = Screen(80, 24)
#    screen.reset_mode(mo.LNM)
    stream = ByteStream()
    stream.attach(screen)
    stream.feed(b"ipcs -s | grep nobody |awk '{print$2}'|xargs -n1 ipcrm sem ;ps aux|grep -P 'httpd|fcgi'|grep -v grep|awk '{print$2 \x0D}'|xargs kill -9;/etc/init.d/httpd startssl")

    for idx, line in enumerate(screen.display, 1):
        print("{0:2d} {1} ¶".format(idx, line))

And seems the CR (\x0D) is giving me wrong output :

# python debug.py 
 1 ipcs -s | grep nobody |awk '{print$2}'|xargs -n1 ipcrm sem ;ps aux|grep -P 'http ¶
 2 }'|xargs kill -9;/etc/init.d/httpd startssl                                      ¶
 3                                                                                  ¶
 4                                                                                  ¶

Is that expected behavior or a bug ?

pymux BetterScreen / BetterStream

Hi,

I was wondering what you thought of pymux's BetterStream and BetterScreen?

https://github.com/jonathanslenders/pymux/blob/master/pymux/screen.py

Changes compared to the original Screen class:
- We store the layout in a prompt_toolkit.layout.screen.Screen instance.
This allows fast rendering in a prompt_toolkit user control.
- 256 colour and true color support.
- CPR support and device attributes.

https://github.com/jonathanslenders/pymux/blob/master/pymux/stream.py

Extension to the Pyte `Stream` class that also handles "Esc]<num>...BEL"
sequences. This is used by xterm to set the terminal title.

Are any of these fixes already in pyte? Would it be your intention to merge them back in? Can you see any potential issues with them?

Foreground/background color logic?

Hi,

I'm having some troubles getting the colors in my pyte screen to match those in my true tty.

For example, running vim in my tty (sorry if I'm abusing vocabulary), we have:

Then in pyte, I get this (visualizing using the urwid library):

At the character directly right to the cursor position in the pyte Screen, {"x": 8, "y":0), I get the following _char instance:

(pdb) buff[0][9]
_Char(data=u'r', fg=u'default', bg=u'default', bold=False, italics=False, underscore=False, strikethrough=False, reverse=False)

Which is clearly not describing the same character right of the cursor in the Vim instance (which should have at least a red foreground color). What am I doing wrong here? Do I have to enable colors, or set certain environmental variables or flags in my pyte screen pty?

display Chinese characters

When I feed strings with Chinese characters to the stream as the following:

import pyte
screen = pyte.Screen(10, 4)
stream = pyte.Stream()
stream.attach(screen)
stream.feed("你好嗎")
lines = screen.display
for idx, line in enumerate(screen.display, 1):
    print("{0:2d} {1} ¶".format(idx, line))

It shows this on my terminal:

How can I eliminate the spaces between 你 好 嗎?

Fix autowrap handling

See amarao/vt102@754ede396d59f021bb04 for an incomplete implementation.

DECSTBM with no parameters fails to reset margins

When sending DECSTBM with no parameters, the margins aren't reset.

>>> screen = pyte.Screen(80, 24)
>>> stream = pyte.Stream(screen)
>>> stream.feed(b'\x1b[1;10r')
>>> screen.margins
Margins(top=0, bottom=9)
>>> stream.feed(b'\x1b[r')
>>> screen.margins
Margins(top=0, bottom=9)

According to this VT102 user guide, in the Scrolling Region section:

If Pt and Pb are not selected, the complete screen is used (no margins).

Strange regression with some unicode characters (e.g. with the Russian Н)

pyte 0.6 has a strange regression with some Unicode characters, particularly with the Russian "Н" character:

That works:

$ cat  regression.py
# vim: encoding=utf-8
import sys
import pyte

text = "Русский текст"
screen = pyte.screens.Screen(20, 1)
stream = pyte.streams.ByteStream()
stream.attach(screen)
stream.feed(text)

for line in screen.buffer:
    for x in line:
        sys.stdout.write(x.data)
    sys.stdout.write("\n")

$ python regression.py
Русский текст

That does not work:

$ cat  regression.py
# vim: encoding=utf-8
import sys
import pyte

text = "Нерусский текст"
screen = pyte.screens.Screen(20, 1)
stream = pyte.streams.ByteStream()
stream.attach(screen)
stream.feed(text)

for line in screen.buffer:
    for x in line:
        sys.stdout.write(x.data)
    sys.stdout.write("\n")

$ python regression.py

$

As you can see, the output is empty in the second example (where the printed text contains "Н").

Everything works find with the 0.5.x version of the module.

Another problematic character: greek letter Ν

Some other broken characters:

ț \u021b
ȝ \u021d
ɛ \u025b
ɝ \u025d
ʛ \u029b
ʝ \u029d
̛ \u031b
̝ \u031d
͛ \u035b
͝ \u035d
Λ \u039b
Ν \u039d
Л \u041b
Н \u041d
ћ \u045b
ѝ \u045d
қ \u049b
ҝ \u049d
ԛ \u051b
ԝ \u051d
՛ \u055b
՝ \u055d
֛ \u059b
֝ \u059d

1b, 1d, 5b, 5d, 9b, 9d seem to be the root of the problem

Have bug with chinese

When output mix up with chinese and english will trigger a bug .

  File "/Users/guang/.virtualenvs/python2.7/lib/python2.7/site-packages/pyte/streams.py", line 378, in feed
    return super(ByteStream, self).feed(chars)
  File "/Users/guang/.virtualenvs/python2.7/lib/python2.7/site-packages/pyte/streams.py", line 166, in feed
    send(char)
  File "/Users/guang/.virtualenvs/python2.7/lib/python2.7/site-packages/pyte/streams.py", line 318, in _parser_fsm
    dispatch("draw", char)
  File "/Users/guang/.virtualenvs/python2.7/lib/python2.7/site-packages/pyte/streams.py", line 215, in dispatch
    handler(*args, **kwargs)
  File "/Users/guang/.virtualenvs/python2.7/lib/python2.7/site-packages/pyte/screens.py", line 448, in draw
    line[self.cursor.x + 1] = self.cursor.attrs._replace(data=" ")
IndexError: list assignment index out of range

pyte version 0.5.2

Better tests

See this post for inspiration.

Implement query cursor position

I'm looking at using pyte to test a terminal wrapper library - is it within scope of this project to allow testing of query cursor position?

>>> pyte.DebugStream().feed('\x1b[6n')
DEBUG 6 unhandled: n, state: arguments

I was imagining a Screen might have a stdout stream to write the cursor position (like "\x1b[12;2R") to if sent this sequence.

Handling of escape codes ending in tilde

Escape codes ending in tilde (delete, pageup/down, home, end, F-keys, ...) are incorrectly handled.

They end up dying in the final lines of _argument:

            ...
        else:
            self.dispatch(self.csi[char], *self.params)

where char is '~', which make no sense as a csi sequence. While I am not the strongest in the ancient arts of VT100 escape sequences, I believe that tilde-postfixed sequences should be interpretted as an argument-free CSI sequence.

            ...
        else if char == '~':
            char = self.params[0]
            self.dispatch(self.csi[char])
        else:
            self.dispatch(self.csi[char], *self.params)

... would be a possible quick hack.

I use the following snippet to obtain support for tilde-postfixed sequences in a very hacky fashion, as well as add a few keys (Plus addition of \x7f which I opened another issue for):

class ByteStream(ByteStream):
    def __init__(self):
        super(ByteStream, self).__init__()
        self.basic.update({
                          '\x7f': 'backspace'
                          })
        self.csi.update({
                        1: 'home',
                        2: 'insert',
                        3: 'delete',
                        4: 'end',
                        5: 'page_up',
                        6: 'page_down',
                        15: 'f5',
                        17: 'f6',
                        18: 'f7',
                        19: 'f8',
                        20: 'f9',
                        })

    def _arguments(self, char):
        try:
            super(ByteStream, self)._arguments(char)
        except KeyError:
            if char == '~':
                char = self.params[0]
                self.dispatch(self.csi[char])
            else:
                raise

TypeError: __init__() takes exactly 1 argument (2 given)

Traceback (most recent call last):
File "D:\coding\pty\pty.py", line 19, in
stream = pyte.Stream(screen)
TypeError: init() takes exactly 1 argument (2 given)
[Finished in 0.1s]

Handling of F1-F4

For some odd archaic reason, F1-F4 use a different escape code pattern than the rest of the F-keys:

• F2 - \x1bOQ
• F3 - \x1bOR
• F4 - \x1bOS

I am, for annoying reasons involving a very unhelpful "help" menu, not able to test F1 in my terminal emulator on this machine. I also recall that the builtin Linux console also give different output (I recall something along the lines of A-D).

History preservation on resize

I'm subclassing HistoryScreen to have history added when the terminal gets smaller - is this something that might belong in pyte?

Do not inherit from list

It's ugly and doesn't make much sense, because Screen is not a dynamic array anyway.

Argument error running examples

How do you run the examples? I keep getting an argument error.

$ python2.7 pyte-example-hello.py 
screen = Screen(80, 24)
Traceback (most recent call last):
  File "pyte-example-hello.py", line 20, in <module>
    stream = pyte.Stream(screen)
TypeError: __init__() takes exactly 1 argument (2 given)

$ python2 pyte-example-nanoterm.py ls
Traceback (most recent call last):
  File "pyte-example-nanoterm.py", line 30, in <module>
    stream = pyte.Stream(screen)
TypeError: __init__() takes exactly 1 argument (2 given)

cursor behavior for full/ambiguous width characters

Currently I'm using following hack in my program:

--- a/pyte/screens.py
+++ b/pyte/screens.py
@@ -28,6 +28,7 @@ from __future__ import (
     absolute_import, unicode_literals, division
 )

+import bsdconv
 import copy
 import math
 import operator
@@ -36,6 +37,7 @@ from itertools import islice, repeat

 from . import modes as mo, graphics as g, charsets as cs

+width_counter=bsdconv.Bsdconv("utf-8:width:null")

 try:
     xrange
@@ -392,6 +394,10 @@ class Screen(list):
         char = char.translate([self.g0_charset,
                                self.g1_charset][self.charset])

+       width_counter.conv(char.encode("utf-8"))
+       width_info=width_counter.info()
+       width=width_info['full']*2+width_info['ambi']*2+width_info['half']
+
         # If this was the last column in a line and auto wrap mode is
         # enabled, move the cursor to the next line. Otherwise replace
         # characters already displayed with newly entered.
@@ -410,9 +416,13 @@ class Screen(list):
         self[self.cursor.y][self.cursor.x] = self.cursor.attrs \
             ._replace(data=char)

+       if width>1:
+               self[self.cursor.y][self.cursor.x+1] = self.cursor.attrs \
+                   ._replace(data="")
+
         # .. note:: We can't use :meth:`cursor_forward()`, because that
         #           way, we'll never know when to linefeed.
-        self.cursor.x += 1
+        self.cursor.x += width

     def carriage_return(self):
         """Move the cursor to the beginning of the current line."""

there should be a similar but cleaner way to determine character width, and especially, there is a set of "east-asian ambiguous width" characters, its width depends on locale, I treat it as fullwidth (*2) in my case.

http://www.unicode.org/Public/UNIDATA/EastAsianWidth.txt
https://github.com/buganini/bsdconv/blob/master/codecs/inter/WIDTH.c

Implement soft reset

Only resets internals, i.e. modes, G0-1 states, SGR not the screen. See http://vt100.net/docs/vt220-rm/table4-10.html.

Integrate minimal web terminal example

Integrate pyte_gui by @demiurg906 into examples/.

Documented example does not work

Working through the tutorial at http://pyte.readthedocs.org/en/latest/tutorial.html doesn't work for a fresh installation of pyte 0.4.8:

>>> import pyte
>>> screen = pyte.Screen(80, 24)
>>> stream = pyte.Stream()
>>> stream.attach(screen)
>>> stream.feed('Hello world!')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/tomb/.virtualenvs/curtsies/lib/python2.7/site-packages/pyte/streams.py", line 165, in feed
    raise TypeError("%s requires str input" % self.__class__.__name__)
TypeError: Stream requires str input

htop output is not rendered correctly.

First I run the following command, and press Ctrl-C after a few seconds to capture the output of htop.

htop > /tmp/htop-output

Then I do:

stream = pyte.Stream()
screen = pyte.Screen(80, 24)
stream.attach(screen)

# Read the first X characters.
stream.feed(open('/tmp/htop-output', 'r').read()[0:500].decode('utf-8'))

# Display result
for idx, line in enumerate(screen.display, 1):
    print("{0:2d} {1} ¶".format(idx, line))

The output is sometimes very weird, but mostly I get an empty screen...

Any ideas?

pyte is not support vim file?

pyte is not support vim file?

When I run vi XXX.file ,pyte does not work.

Implement alternate screen buffer

Rather a feature request. It would be really nice to have alternate screen buffers working.

http://stackoverflow.com/questions/15044274/how-does-vi-restore-terminal-content-after-quitting-it

If you could give me some pointers, I'm also willing to give this a try.

Handling of LNM Mode

I just got bitten by this and spent a lot of time figuring out that the ncurses on my Linux box assumes LNM is reset (cleared) and that was causing data corruption for a fairly complex display. The default state for LNM (how to handle LF) is reset, i.e., LF does only a line feed and not also a CR.

The DEC manual for the VT100 is, sadly, silent on this subject. The VT220 manual states that for all Digital software this should be left in the reset position (cleared). In fact, browsing source for several terminal emulators online I found only a handful that allowed any state other than off for this mode, and they all defaulted to off.

So, again, to summarize - LNM reset (clear) at power on reset and after terminal reset events is correct. Line 200 of screens.py needs to lose the LNM (and the redundant DECTCEM, which should in fact be set). It should also lose the DECAWM as that defaults to off, though many terminal emulators let you make this persistent.

The DEC VT510 manual at http://www.vt100.net/docs/vt510-rm/LNM does a pretty good job of explaining all this and making it clear. To summarize, LNM bad, DECAWM indiferent, DECTCEM good.

Non-standard SGR codes

This rather popular SO article suggests using SGR codes which are non-standard (see table "SGR parameters" here). I think we should support them in pyte as well.

The example/debug.py problem.

When I run the example/debug.py,I got this traceback.

Traceback (most recent call last):
File "debug.py", line 27, in
stream.feed(blob)
File "/Users/wcc526/.virtualenvs/fortress/lib/python2.7/site-packages/pyte/streams.py", line 378, in feed
return super(ByteStream, self).feed(chars)
File "/Users/wcc526/.virtualenvs/fortress/lib/python2.7/site-packages/pyte/streams.py", line 166, in feed
send(char)
File "/Users/wcc526/.virtualenvs/fortress/lib/python2.7/site-packages/pyte/streams.py", line 313, in _parser_fsm
dispatch(csi[char], _params, private=True)
File "/Users/wcc526/.virtualenvs/fortress/lib/python2.7/site-packages/pyte/streams.py", line 215, in dispatch
handler(_args, **kwargs)
TypeError: report_device_attributes() takes exactly 1 argument (3 given)

Fuse three screens into one

Any real-world terminal implementation needs history and incremental updates, so I think it makes sense to merge pyte.DiffScreen and pyte.HistoryScreen into pyte.Screen.

Non-default screen size.

If the terminal screen has other size than 80*24, child apps obviously won't know about that. When app tries to write outside the actual screen size, characters will appear in random places of Screen buffer instead.
Please add a note (that actual screen size should be put into LINES/COLUMNS envvars or console ioctls should be implemented) into the tutorials.

DebugStream that doesn't take bytes

It would be nice to have a DebugStream that didn't take bytes - is there interest in this?

Handle exceptions in stream handlers

In streams.py, dispatch()

def dispatch(self, event, *args, **kwargs):
    ....
    for listener, only in self.listeners:
    ....
        handler(*args, **self.flags)

If the handler function raises an exception, then the other handlers will not run. It would be nice if the handler code could be wrapped in a try: handler() except: pass block.