denosaurs / deno_python Goto Github PK

A recent issue I found while attempting to use this module is that it lacks pyenv support instead resolving to the default python installation on mac. This makes it so that imports can't properly resolve among other issues.

import { python } from "https://deno.land/x/[email protected]/mod.ts";

const gi = python.import("gi");
gi.require_version("Gtk", "4.0");
const Gtk = python.import("gi.repository.Gtk");

class MainWindow extends Gtk.ApplicationWindow {
  constructor() {
    super();
    this.a = 4; // error here
  }
}
console.log(new MainWindow());

error: Uncaught TypeError: 'set' on proxy: trap returned falsish for property 'a'
    this.a = 4;
          ^
    at new MainWindow (file:///home/mrcool/dev/deno/lab/b.ts:10:11)
    at file:///home/mrcool/dev/deno/lab/b.ts:14:13

more info here #53 (comment)

Hi.

I not sure about "support iterating over PyObject" PR but after the playground

PR: https://github.com/denosaurs/deno_python/pull/6/files

As you can see test method return a list of int but still throw error that list is not iterable.

Environment:

Deno 1.18.2 (release, x86_64-apple-darwin)
v8 9.8.177.6
Typescript 4.5.2
Python 3.9
OS: MacOS Big Sur

It's not clear in doc (README.md) whether "DENO_PYTHON_PATH" (in a Windows 10 context) should point to the python DIRECTORY, or to the python.exe FILE, or to anything else.

In my case pointing to python.dll solved the problem. (More precisely "C:\Python310\python310.dll").

It would be also worth to mention in docs that the flag "--unstable" IS ABSOLUTELY NECESSARY.

Thank you for this amazing piece of code you are working on!

value.toString in python for an exception loses the type information which is particularly handy for debugging

suggested alternative

  let errorMessage = (value ?? type).getAttr("__repr__").call().toString();

Trying to run this for the first time.

Code

import { pip } from "https://deno.land/x/python/ext/pip.ts";

const np = await pip.import("numpy");
const plt = await pip.import("matplotlib", "matplotlib.pyplot");

const xpoints = np.array([1, 8]);
const ypoints = np.array([3, 10]);

plt.plot(xpoints, ypoints);
plt.show();

Terminal

PS C:\Users\MyUser\Downloads> $env:DENO_PYTHON_PATH='C:\Users\MyUser\AppData\Local\Programs\Python\Python310\python310.dll'; deno run  -A --unstable --allow-net=deno.com .\deno1.js
Installing numpy
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\numpy already exists. Specify --upgrade to force replacement.
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\numpy-1.25.2.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\bin already exists. Specify --upgrade to force replacement.

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: deno.exe -m pip install --upgrade pip
error: Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'packages_distributions')
    const packages = importlib.metadata.packages_distributions();
                                        ^
    at Pip.import (https://deno.land/x/[email protected]/ext/pip.ts:130:41)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async file:///C:/Users/MyUser/Downloads/deno1.js:3:12
PS C:\Users\MyUser\Downloads> deno --version      
deno 1.36.1 (release, x86_64-pc-windows-msvc)
v8 11.6.189.12
typescript 5.1.6
PS C:\Users\MyUser\Downloads> 

In contrast to most JavaScript code, Python libraries often work synchronously and may even expect one's callback function to resolve synchronously, even if that doesn't make sense in JavaScript environment. One example might be the audible Python package that performs all HTTP requests synchronously and expects a synchronous callback to show an image to the user and get their response.

I have no experience with Deno's FFI yet, but it seems to me that calling synchronous Python functions asynchronously should be possible, as the Python interpreter runs in a different thread. Similarly, I would expect "converting" asynchronous JS functions to synchronous Python callbacks should be possible, if one blocks the Python interpreter until the JS promise is settled.

Is this something that sounds viable to you? And if so, is this something you'd be interested in adding to this library?

reproduction examples

> python.builtins.eval("'中文'#foo", {})
中文
> python.builtins.eval("'中文'", {})
Uncaught PythonError: ('EOL while scanning string literal', ('<string>', 1, 3, "'中文"))
> python.builtins.str("'中文'#foo")
'中文'
python.builtins.str("'中文'")
'中
> python.builtins.str("中文")
Uncaught PythonError: <class 'str'> returned a result with an error set

I tried to use deno_python with spacy (https://spacy.io/)

import { python } from "https://deno.land/x/[email protected]/mod.ts";

const spacy = python.import("spacy");
console.log(spacy);

But got a segmentation fault

 48182 segmentation fault  deno run -A --unstable spacy.ts

Working well with

 python test.py 

Environment:
deno 1.26.0 (release, x86_64-apple-darwin)
v8 10.7.193.3
typescript 4.8.3
Python 3.10.6

Do you have any idea about this issue @DjDeveloperr?

There's currently no way to use slices (:) or ellipsis (...).

Would something similar to NamedArgument be able to solve this?

import { python, Slice } from "https://deno.land/x/[email protected]/mod.ts";

const np = python.import("numpy");
const slice = np.array([1, 2, 3]).__getitem__(new Slice(1,3,null)); // equivalent to [1:3]
console.log(slice); // returns [2, 3]

import { python } from "https://deno.land/x/[email protected]/mod.ts";

const m = python.runModule(`
b = True
  `);

console.log(typeof m.b, m.b == true, /*works*/ m.b === true /*doesn' work*/);

function true false

I expected the type of b to be bool but any variables from python always have type function

Hi @eliassjogreen @DjDeveloperr

I'm trying to access TS/JS function by writing FFI in Rust and getting JS/TS function in v8 Context.

Unfortunately, rusty_v8 does not expose Isolate::Get_Current() method
denoland/rusty_v8#469.

Is there any way to call TS/JS function in Python land?

Hi @DjDeveloperr

I try this test but got 67010 segmentation fault in MacOS Big Sur.

const { Test } = python.runModule(`
class Test:
  def test(self, *args, **kwargs):
    return all([len(args) == 3, "name" in kwargs])
`);

const t = new Test();
const d = python.dict({ a: 1, b: 2 });
const v = t.test(1, 2, new NamedArgument("name", "vampire"), d);
assertEquals(v.valueOf(), true);

After investigating the issue. I saw that we have a segfault when we try to new tuple but not fill all items
https://github.com/denosaurs/deno_python/blob/main/src/python.ts#L568

Let's add an entry to the python wiki about this!

https://wiki.python.org/moin/IntegratingPythonWithOtherLanguages

/Users/elias/.pyenv/versions/3.11.2/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown

Every time my script shuts down i get a warning about a leaked semaphore. I am guessing we are forgetting to free some python memory somewhere somehow.

some warnings don't show up when using this library, for example:

a.py

import gi
gi.require_version('Gtk', '4.0')
from gi.repository import Gtk
from gi.repository import Gio

def on_activate(app):
    win = Gtk.ApplicationWindow(application=app)
    win.get_style_context()
    win.present()

app = Gtk.Application()
app.connect('activate', on_activate)

app.run(None)

a.ts

import {
  Gtk,
  NamedArgument,
  python,
} from "https://raw.githubusercontent.com/sigmaSd/deno-gtk-py/0.1.4/mod.ts";

const activ = python.callback((_, app) => {
  const win = new Gtk.ApplicationWindow(new NamedArgument("application", app));
  win.get_style_context();
  win.present();
});
const app = new Gtk.Application();
app.connect("activate", activ);
app.run();

this warning only appear on the python version

DeprecationWarning: Gtk.Widget.get_style_context is deprecated
  win.get_style_context()

I'm not really sure where does it come from , maybe the python binary itself is needed for such things ? or its lost in some return argument in ffi ?

https://docs.python.org/3/c-api/buffer.html

Haven’t found this in the examples.

If I wanted to pass linewidth=2 to the plot function of matplotlib, how can I do this from Deno code?

Not sure about other platforms, but findLib is pretty slow on Windows. It takes up 2s at startup on my machine. I think we should not use a command (where) to look for Python, instead try to load all supported Python versions in order and return the suitable library.

This is the reproduction I have, you need to keep moving the slider fast until it throws the above error or sefaults

import {
  Adw,
  Gtk,
  Gtk_,
  kw,
  NamedArgument,
  python,
} from "https://raw.githubusercontent.com/sigmaSd/deno-gtk-py/13f3da6c4890d62e09312747c905fa85263f5ca8/mod.ts";

class MainWindow extends Gtk.ApplicationWindow {
  #slider: Gtk_.Scale;
  constructor(kwArg: NamedArgument) {
    super(kwArg);
    this.set_default_size(600, 250);
    this.#slider = Gtk.Scale();
    this.#slider.set_range(0, 10);
    this.#slider.connect("value-changed", this.slider_changed);

    this.set_child(this.#slider);
  }

  slider_changed = python.callback(
    (_kwargs, slider: Gtk_.Scale): undefined => {
      slider.get_value(); // this line trigger the error/segfault
    },
  );
}

class App extends Adw.Application {
  #win: MainWindow | undefined;
  constructor(kwArg: NamedArgument) {
    super(kwArg);
    this.connect("activate", this.onActivate);
  }
  onActivate = python.callback((_kwarg, app: Gtk_.Application): undefined => {
    this.#win = new MainWindow(new NamedArgument("application", app));
    this.#win.present();
  });
}

const app = new App(kw`application_id=${"com.example.com"}`);
app.run(Deno.args);

The bunpy npm package is still on the v0.3.3 release; it would be good if it updates continued to be published to npm as releases are made (particularly as bunpy has stopped working for me since updating python).

CC @DjDeveloperr

import { pip } from 'https://deno.land/x/[email protected]/ext/pip.ts';
await pip.import('numpy');

Second and consecutive runs of this code give me the following warnings:

WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/numpy already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/numpy-1.26.0.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/bin already exists. Specify --upgrade to force replacement.

My version of deno is 1.37.1

https://docs.deno.com/runtime/manual/tools/jupyter#rich-content-output

Currently this python.callback((_, _) always gives a type error, becuase it seems the jscallback default to void which is not a PythonConvertible.

Currently the workaround is to always annotate the type, mostly like this python.callback((_, button: Gtk_.ToggleButton): undefined

If void can be add as a PythonConvertible this would remove this unnecessary typing.

Thanks for creating deno_python. I tried to run NumPy example but faced this issue.
Could you help me to resolve this issue?

error: Uncaught (in promise) PythonError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/usr/local/bin/deno"
  * The NumPy version is: "1.22.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): Symbol not found: _PyBaseObject_Type
  Referenced from: /usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so
  Expected in: flat namespace
 in /usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so

Traceback:
['  File "/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/__init__.py", line 144, in <module>\n    from . import core\n', '  File "/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/__init__.py", line 49, in <module>\n    raise ImportError(msg)\n']
  throw new PythonError(errorMessage);
        ^
    at maybeThrowError (https://deno.land/x/[email protected]/src/python.ts:641:9)
    at Python.importObject (https://deno.land/x/[email protected]/src/python.ts:717:7)
    at Python.import (https://deno.land/x/[email protected]/src/python.ts:727:17)
    at file:///Users/hoangpq/WebstormProjects/deno-working/server.ts:3:25

But working well on Python REPL

Environment:
OS: macOS
Python version: 3.8 (Homebrew)
Deno version: 1.18.0

Is possible to import deno module in python code ?