Advised way of importing libraries about belay

[roaldarbol]

The package manager is already dawning on me. I really really like it! However, I'm a bit confused by:

Belay assumes your project contains a python-package with the same name as tool.belay.name located in the root of your project.

Does that mean you cannot have just a pyproject.toml file, the python script and nothing else in a repo? (except for the autogenerated .belay-lib).

Edit: I'll make a separate issue for discussion of the package manager.

[BrianPugh]

so there are multiple ways, here are some thoughts in no particular order:

1. Belay now contains a package manager, so you can manage libraries externally from your script. This is (obviously) my preferred way for the reasons outlined in the documentation. This syncs files into the on-device lib folder.

2. It might be nice to programmatically run this in your script; should I expose a device.install() method that basically performs the belay install action? I should probably also add a device.update() method as well, but thats less critical since this should probably be manually ran anyway.

3. Currently, the best way of moving local files over is using the device.sync method, but that might be made (relatively) obsolete with the proposed device.install method mentioned in (2) for the common case of moving over your project's package.

4. As for actually importing libraries into the global scope, currently you have to do device("import my_library"). Running imports I could add a tiny bit of syntactical sugar to make it look like device.import("my_library"), but I don't think it's necessary to make the interface a little more complicated without making the importing process actually any easier. Importing libraries within a task will not make them available in the global scope, as its equivalent of the following (run in repl):

>>> def setup():
    import os
    
>>> 
>>> setup()
>>> dir(os)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'os' isn't defined

[roaldarbol]

I had obviously completely missed the package manager - that's really a neat way of doing it! I do think that a single-file setup would be good to have too, especially for sharing code (which happens frequently in within labs like mine). In R I really like the pacman package which handles library installation (and import). So in their case you'd use p_load("library") to 1) check if the package is installed, 2) install it if it isn't, and 3) load/import it. It's a bit trickier with python packages I think, as the installation name (e.g. belay install micropython-name) doesn't always correspond to the import name. So yeah, maybe it would be good to have an exposed device.install() method - it just needs to be clear that you need the installation name (i.e. micropython-name) whereas you'll use the name later for importing.

As for importing, good point about them only being available locally! It might be nice to have a way of doing it programatically as you suggest too. I think doing device.import("import library") is a good beginning. My opinion would be that the user should probably still write "import" themselves, as it makes other variations explicit (e.g. from library import function as fct). So it would look like:

device.import(
   "import logging",
   "from neopixel import NeoPixel",
   )

Just a thought, could you also load global variables in that way? (I'm not quite sure of the use case as I would write global variables in my python script and use them in the function calls). But if there's a use case and it would work, it could be called device.setup() instead, and would be kind like Arduino's void setup{}.

Wow. That was a lot of words. 😅

[BrianPugh]

as for your proposed device.import, thats actualy just the same as the normal device.__call__ method; i.e.:

device("import logging; from neopixel import Neopixel")

All device("python statement here") are executed on-device in the global context, so it's useful for imports and declaring global variables/objects. A possible workflow is instantiating hardware this way, and then using it inside tasks. An alternative is your setup, but no need to make it builtin, you could do something like:

@device.task
def setup():
    from mysensor import MySensor
    global my_sensor
    my_sensor = MySensor(Pin(15))
    
setup()

As a design philosophy, I'm trying to keep the number of device methods as minimal as possible (whole zen of python thing). I'd only feel inclined to add additional methods if it makes the user's life significantly easier (e.g. device.sync). Since both setup, library import, and global declarations can be done without much effort using the existing toolset, I don't feel inclined to add more methods.

Thinking about it more, there is one missing functionality here. Belay has the ability to replay commands, which is useful if a device disconnects/resets for whatever reason. Currently all task executions are not recorded. I should add an option to the task/thread decorator to allow recording, something like:

@device.task(record=True)
def setup():
    from mysensor import MySensor
    global my_sensor
    my_sensor = MySensor(Pin(15))
    
setup()

this way, your setup() will be re-executed upon re-connect.

[roaldarbol]

For some reason I have a great dislike for using global, simply because it adds extra lines. A way around it could be something suggested on StackOverflow, adding globals().update(locals()) to the end of the function to make everything available globally. Here I'd love a @device.global that adds that line to the call itself, rather than having to write it yourself at the end of a @device.task. I get what you mean about keeping methods minimal, but I'd also argue that pythonian philosopy also says to make things explicit rather than implicit. And using just device() is a very implicit way of going about things I'd argue.

[BrianPugh]

how about something like:

@device.task(record=True, global=True)
def setup():
    from mysensor import MySensor
    my_sensor = MySensor(Pin(15))
    
setup()

[BrianPugh]

so i was looking at how this would be implemented, and I think there are too many sharp corners for it to work. Example:

device("foo = 5")

@device.task(global=True)
def setup():
    foo += 1

setup()

This wouldn't work as expected. As far as implicitness, I think nothing is wrong using device(), the functionality is very explicit: execute code on device.

[roaldarbol]

Maybe this could be amended by solving #51? If the user can create a class, then a setup function would automatically update variables globally within the object. I.e. when this kind of work flow, we could recommend creating a class.

[BrianPugh]

@roaldarbol checkout #54 and provide feedback in the PR. I'm going to close this issue for now.