Incompatible with PyDev 2.7.5. about live-py-plugin

[donkirkby]

Thanks for the thorough bug report, Coaldust. I built it with PyDev 2.7.1, so the most likely cause is that they changed their interface, and I need to update my code. Until I do that, you might be able to work around it by installing an older version of PyDev (there's usually a checkbox to show old versions in the install dialog).

Not sure why your comment got deleted from the blog, but this is a better place to report bugs, anyway.

If there is some way I can help you debug it, I will try to do so. While I know Python rather well, I don't know Java well, and I have very little experience even using Eclipse. I'm convinced I can learn, given time, though.

I want this to work rather badly.

I will try using PyDev 2.7.1.

As an aside, I second the "[enhancement] Show result of loop-condition or if-condition in output" request. While it's possible to move all conditions into functions or assign them to variables it makes the code longer, and more awkward, in most cases.

I'm really glad that you show past loop/recursion history, and this feature request has much the same effect. The few instant feedback implementations I have seen (e.g. Light Table, EMACS litable, Firefox's Live Scratchpad, ZeroBrane Studio (which /only/ has turtle graphics, no variable value display!)) do not show past 'passes', and it makes it much harder to understand what happened when a loop 'goes off the rails'.

If cluttered output is the issue, I'd be quite content with a option that was off by default.

I'm not having success with PyDev 2.7.1.

PyDev 2.7.1 itself works. However, attempting to use the "Live Coding" menu option now causes the expected panel to open, but nothing is displayed inside it. I've tried using "echo on" comments in both panels, as shown in your videos, but this doesn't help. I've also tried commenting out the guard that prevents the code from running outside instant feedback mode to no avail.

The sample code I've used is:

def main():
    # Put your code here.  Perform no I/O.
    x = 2 + 2
    return x

if __name__ == '__live_coding__':
    result = main()

There are no longer errors in the Error Log, only warnings.

Warnings (not errors) in Error Log:

"Keybinding conflicts occurred.  They may interfere with normal accelerator operation."

"A conflict occurred for CTRL+SHIFT+T:
Binding(CTRL+SHIFT+T,
    ParameterizedCommand(Command(org.eclipse.jdt.ui.navigate.open.type,Open Type,
        Open a type in a Java editor,
        ...
Binding(CTRL+SHIFT+T,
    ParameterizedCommand(Command(com.python.pydev.analysis.actions.pyGlobalsBrowserWorkbench,Python
        ..."

[donkirkby]

OK, I've reproduced the problem by upgrading to PyDev 2.7.5 on Eclipse 3.7.2, and using live-py-2.5. I will try to fix it along with a couple of other bugs, and release a new version soon. If you can't wait for the new release, you might try my version of Eclipse.
The echo on commands are not needed in live-py 2, you just need to display the Live Coding view. Does the Live Coding Canvas work for you?

Environment (the rest is the same):
Eclipse Classic 4.2.2
PyDev 2.7.1

The Live Coding window appears, but neither the value displaying nor turtle graphics (Live Coding Canvas?) work. I tested using examples from your blog entry. The Live Coding window remains completely blank. There is no error in the Error Log when I attempt it, only the warnings I mentioned before.

I'm guessing that your dropin has problems with both current Eclipse and current PyDev, so it'll probably have to be adjusted for both.

I will try testing with your version of Eclipse. I'm guessing from the instructions on your live-py-plugin installation page that you last had it working on Eclipse 3.7.0.

Environment (the rest is the same):
Eclipse IDE for Java Developers 3.7.2
PyDev 2.7.1

I was unable to download the 3.7.0 version of Eclipse, and the download link for the Classic version was broken. I got the closest available version (listed above).

Sadly, the behavior is the same. Unless there's some secret incantation I haven't been able to figure out from the blog post or videos, the Live Coding window never does anything. If something else is needed, it really needs to be documented.

This environment still has the key binding warnings I mentioned, but has a new error as well:

org.eclipse.swt.SWTException: Failed to execute runnable (org.eclipse.swt.SWTException: Widget is disposed)
...

I don't know what else to try. I started out using the latest stable version of everything, which is what I thought live-py-plugin would be tested on. It didn't say I had to use a specific older version of anything. In any case, I've given you all the information I can think of to help you find the problems by recreating the environment. The problems occur reliably. If you need me to test something new, please say so.

If you know something I might need to do (e.g. install, or type somewhere, or set in Preferences) to make the Live Coding window actually do something, let me know.

[donkirkby]

Thanks for all the testing you've done. I just published live-py v2.6, that I have successfully tested with PyDev 2.7.5 and Eclipse 3.7.2.
Please give it a quick test with your version of Eclipse, but don't fight with it if it doesn't work. Just tell me, and I'll install a newer version of Eclipse to track down the problem.

I wiped my workspace directory, wiped my Eclipse install, and installed the latest stable versions of Eclipse Classic and PyDev. This results in the same environment (versions) as I'd originally used, except for live-py.

There are no errors in Error Log.

The Live Coding window remains empty.

At this point, I'm going to have to assume I'm doing something wrong.

I'll try to walk you through my installation and configuration so you can figure out what it is.

1. unzip Eclipse into a directory on my Desktop (directory)
2. run Eclipse as an Administrator
3. go through the menus "Help: Install New Software"
4. paste "http://pydev.org/updates" into the textbox by "Work with:" and press the Enter key
5. check "PyDev" (leave the Mylyn Integration unchecked)
6. click "Next" repeatedly, selecting the defaults and accepting licenses and keys as I go
7. allow Eclipse to restart itself
8. close Eclipse
9. copy the live-py-plugin jar file into the "dropins" directory
10. run Eclipse again
11. click the close button ("[X]") on the "Welcome" display
12. go through the menus "File: New: Project", expand the "PyDev" subtree, and select "PyDev Project", and click "Next"
13. type in something (anything, I've tried changing it around) for the project name
14. change the grammar version to "3.0", the latest offered, though I'm really using 3.3.2
15. click the hyperlink that asks me to configure the interpreter
16. click "Auto Config" and accept the defaults
17. click "Apply" and wait
18. click "OK" to close that window
19. back on the PyDev project creation window, tell it to create the 'src' folder and add it to PYTHONPATH
20. click "Finish"
21. tell it that it's okay to open the PyDev perspective
22. expand my project tree and right click the 'src' directory and select "New: PyDev Package", then type in something (anything, I've tried changing it) for the name and click "Finish"
23. restart Eclipse, because otherwise the "Live Coding" menu doesn't appear
24. my init.py file will be selected already, and ready for me to type to, so I select "PyDev: Live Coding" from the menus, and start typing

That's it. I observe that the "Live Coding" window does nothing. I did go through this whole procedure as I wrote this to make sure I didn't miss anything.

I've tried several variations on the above procedure, including most of the walkthrough on the PyDev site to make sure it (PyDev) was working correctly. I've also just tried opening a Python source code file. I've tried typing "# echo on" comments in both windows. I've tried all different sorts of source. Nothing helps.

[donkirkby]

I tried to follow your steps as closely as could, but I still can't reproduce the problem. I'm running Eclipse 4.2.2 64-bit on Ubuntu 12.04 with PyDev 2.7.5 and live-py 2.6. I have Python 2.7 and 3.2 installed, so I had to manually select version 3.2.

I typed the following code into __init__.py:

x = 3

It displayed exactly the same thing in the Live Coding window.

If you're still willing to try something, create a test.py file in your src folder, and write the following code:

x = 3
print('x is %d' % x)

Then try actually running that using PyDev (Ctrl - F11). Do you see the message in a console window? If your Python code won't run that way, then it probably won't run under Live Coding either.

I did exactly as you said, and the "Live Coding" window remains empty for both "init.py" (I'm sorry, I don't know how to make this thing stop eating my underlines) and "test.py". However, I do see:

x is 3

in my console window.

[Edited]
I have, however, figured out what the problem is. See below.

I have great news! I know exactly what the problem is.

The problem is Python 3.3.2.

I have tested it myself with Python 3.2.5 and it works fine. It does not work at all when I select the 3.3.2 interpreter.

Also, should it comfort your nerves, your plugin works just fine with the release candidate of the next version of Eclipse, and Eclipse Classic is sufficient to run it.

So in the end your plugin had two problems. One was that it was incompatible with the latest stable PyDev, the other was that it was incompatible with the latest stable Python.

[Edited]
To be really specific, the problem seems to be the fact that "ast.TryExcept" has been renamed to "ast.Try" (and combined with TryFinally, hence the name) in Python 3.3. There may be other problems, but at least that is definitely a problem.

I don't understand how the code works well enough to help debug thoroughly. On that note, I would be /very/ interested in hearing exactly how you made this work, at a higher level than the code. I expect it to be rather difficult since I doubt the interpreter was intended to support this. I'm probably not the only one interested either. Maybe this is blog post material?

[donkirkby]

That is good news, I'm very relieved. I've split the Python 3.3 bug into issue #28, and I'll tackle it later.

A blog post is a good idea, but I can briefly tell you that I parse the user's Python code using the ast module, and then insert logging code around all the variable assignments, loops, function calls, and so on. When the execution ends, I format all the events into a report and display it to the user.

I posted some answers on Stack Overflow with a bit more detail about the parsing code, as well as the side-by-side display in Eclipse.

That's neat! I'll read up on what you've already posted, and keep an eye on the blog. I'm curious if and how you store the logging information before displaying it, or if you do it incrementally as it's running. I'm also interested in how you separated multiple passes through code with those vertical bars, the tricky part being making sure there's room for the longest value. Then there's how you handle noticing the text changed (a 'changed' event, I assume), and run the code without bogging down the editor (separate thread, aborted if it's still running while something changes?). I'll wait for the blog entry though.

I do hope you'll consider that branch/loop test option (even if it defaults to off, which is fine).

Thanks for working through this with me, and I look forward to it being fixed.

[donkirkby]

Yes, there's obviously a lot of detail to get it all working well. The
Python tests are a good place to see all the various scenarios it needs to
handle.
Sorry, but I don't remember what the branch/loop test option is. Was that
something I posted, or was it in Bret Victor's video? I would like to let
the user select a unit test to exercise the code instead of the current
method:

if __name__ == '__live_coding__':
    # Call your function or class method here

However, I think the next big adventure for this project is to attempt to
port it to Java development.

Branch/Loop Test Result Display

The "branch/loop test" option was a reference to this: #15

Your suggestion was to just work around its absence by always assigning the test result to a variable, which would cause it to be displayed because variables are already displayed. You closed it with "Won't Fix".

The disadvantage to this workaround is it makes procedures longer (each time you do it, it adds a line), and it's awkward/unnatural looking. While I agree that it's best to put complex expressions in a function, so they can be given a meaningful name, there's really no reason to give them a name in the common cases like "foo != bar". For simple tests it's often hard to even come up with a name that isn't just a restatement.

Honestly, the best use-case for the requested feature is probably in debugging long chains of nested if-then-else's. While it's all well and good to say that those should be avoided, they can't always be replaced with polymorphism or some other construct, at least without obfuscating the code. They're often the natural solution in things like efficient "direct coded" lexers.

In code like this it's easy to make a mistake in one of the tests and cause the wrong set of blocks to get executed. Being able to see which tests are true/false helps when you have a bug like this.

That doesn't mean I wouldn't ever want to see this information for loop tests. Sometimes when you write a loop and mess up the test, the loop body never runs, resulting in hard to find bugs. Then you have situations like the efficient lexers I mentioned, where you have nested and chained branches /inside/ a loop, with loops inside the branches. :P

Besides, adding this support for loops has nice symmetry with recursive functions (loops are just tail-recursive functions, after all), and your tool already shows the arguments for each pass through a recursive function. "for" loops also report their value (though, admittedly, not a boolean) already. It seems sort of inconsistent otherwise.

The trace part of the display is already going to take up a line for the test part of a branch or loop, does it really make the display any more cluttered to show a "True" or "False" instead of that blank line? Is adding an additional line to the trace (for the extra variable assignment workaround) really less cluttered?

Since you didn't like the idea, and at least two users (the person that proposed it, and myself) would, I suggested a compromise be made where it was an option, but was off by default.

Either way, it's your program, and I'm not trying to tell you what to do. I'm more begging you for it. ;) I'm quite appreciative of the work you've already done.

On Unit Testing

Here's how I'd do it (which doesn't mean it's the best way)...

Ask the user for a meaningful name for the unit test (e.g. "validating user input"), along with arguments for the procedure, and the expected return value.

Use a accordion/toolbox widget for the display: http://doc.qt.digia.com/qq/qq07-toolboxes.png

All the 'titles' of the accordion/toolbox can be viewed as a vertical column. To the left of each title would be a small icon that would be green when the test passed, and red when it failed. This would allow the user to view the 'blinkenlights' as they edited their code. They'd instantly know if they made a mistake.

Clicking the title associated with the unit test would show the trace information for that test. The appearance of the trace would be exactly as it is now

Remaining Issues:

• how to store the unit tests
• how to keep it easy to associate the source with the accordion titles, and their contents, the traces, appearing at different heights

Unit tests could be stored in a format very similar to, but not the same as, doctest. Advantages include being visible in the source, and showing in documentation dumps to the web or PDF. It's not possible to use doctest format because doctest has no provision for unit test names (which is weird, honestly). Perhaps a unit test name could end in a colon, and the rest would be the same as doctest format only indented under the colon-terminated line. That would be sort-of consistent with the rest of Python's syntax, and keep people used to doctest comfortable. I think it might even work with 'normal' doctest tools, but I'm not sure about that.

Unit tests could be stored externally... The only advantage I can think of is being less likely to confuse things trying to parse special comments in Python as documentation. It might also be easier to read the information, because you wouldn't have to fish it out of comments. Keeping it in sync with the source (e.g. if they delete a procedure in another editor, that had unit tests in live-py) would probably be more difficult, though.

My solution for the accordion/toolbox widget's variable height problem is the usual view would have the editor on the left and the titles/lights on the right (where the trace is right now), and clicking the title would actually cause the other titles to disappear showing the trace view for that unit test exactly the same as it is currently (same height as the source, same display). To get back to your 'list of titles/blinkenlights' view, you could click a [<] button to the right of the unit test title. I think this solves the UI problem pretty cleanly, even if it sort of is a new kind of widget (not really a accordion/toolbox). I can't think of anything bad about it.

I'm not particularly attached to my idea, though, and someone else may have a better one.