nrobinson2000 / po-util

Classic Edition of po-util: The Ultimate Local Particle Experience for Linux and macOS

This tool has been deprecated. Please upgrade to neopo.

Particle Offline Utility: The Ultimate Local Particle Experience for Linux and macOS

Particle Offline Utility, pronounced po-util, is a tool for installing and using the Particle Toolchain on Linux and macOS.

po-util makes it easy for developers to download the Particle Toolchain and install the required dependencies to quickly begin creating projects using the Particle Platform.

po-util features a responsive experience, giving developers the tools they need for seamless local development, providing understandable commands for simplifying complex tasks like properly compiling and flashing firmware.

Install

There are two versions of po-util. The first is po-util Classic, which is designed for Linux distributions. The second is po-util Homebrew Edition, which is designed for macOS.

The quickest and easiest way to install po-util on Linux or macOS is to run the following:

$ bash <(curl -sL get.po-util.com)

Note: Both versions of po-util have the same features. In December 2016, Homebrew Edition was forked from po-util in order to allow po-util to be installed with Homebrew. The original po-util was renamed to Classic, and was made Linux exclusive. Both Classic and Homebrew Edition continue to be developed and maintained in parallel.

Key Features

Library Manager

The library manager makes it easy to use Particle libraries when developing with po-util locally by using git to download Particle libraries hosted on GitHub and save them to a ~/.po-util/lib directory so that they make be linked inside of projects later. Read more about the Library Manager here.

Full dependency setup

po-util installs all of the dependencies required for local Particle Development, including the ARM toolchain, dfu-util, nodejs, particle-cli and the Particle firmware.

Local compilation and support for flashing over USB or OTA

Building firmware locally for Electrons, Photons and P1s, and automatically flashing over USB using dfu-util or Over The Air using particle-cli. Sequentially flashing multiple devices at once over the air is also supported.

Shortcuts for Building in Atom

po-util supports shortcuts for Atom, the popular open- source IDE by GitHub. The shourcuts provide quick access to commonly used commands for po-util, including: building firmware, flashing firmware over USB, and flashing firmware Over The Air. Read more about the shortcuts here.

Project Structure

The directory structure of a full po-util project is arranged like so:

firmware/
├ main.cpp
└ lib1/
  ├ lib1.cpp
  ├ lib1.h
  └ ...
bin/
├ firmware.bin
└ ...
 ci/
 devices.txt
 libs.txt
 .atom-build.yml
 .travis.yml
 .gitignore
 .git/
 README.md

• All user code is kept inside of firmware/.
• The compiled binary will be named firmware.bin, and it will be in bin/.
• You can keep track of which devices are in a project and list which ones to be flashed Over The Air in devices.txt.
• Libraries are kept track of in libs.txt.
• Atom shortcuts are configured in.atom-build.yml.
• Every po-util project is initialized as a repository with scripts in ci/ to use Travis CI for testing.

Project setup

When using po-util, your code is arranged into projects with the structure described above. This not only lets you easily work on many projects on one computer, but it provides a standardized way of building locally.

Creating a project with po-util is simple when you use po init to initialize a project by creating the necessary files. Below is an example:

$ po init DEVICE_TYPE someProject

You can set DEVICE_TYPE to either photon, P1, electron, core, pi, or duo. This is necessary for generating the Atom shortcuts file appropriately.

More Information

Here is some more information about using po-util.

Writing Firmware

po-util compiles any .cpp and .h files found in the firmware/ directory, but not .ino files, so #include "Particle.h" must be present in your main.cpp file. This is done for you when you run the po init command to create a project directory.

Building Firmware

To compile firmware, simply run po DEVICE_TYPE build, substituting DEVICE_TYPE for photon, P1, or electron. To compile and flash firmware to your device using dfu-util, simply run po DEVICE_TYPE flash. To clean the project, run po DEVICE_TYPE clean.

DFU Commands

To upload precompiled code over USB, run po DEVICE_TYPE dfu. To put your device into dfu mode, run po dfu-open. To get your device out of dfu mode, run po dfu-close.

Over The Air (OTA) Uploading

To upload precompiled code over the air using particle-cli, run po DEVICE_TYPE ota DEVICE_NAME, where DEVICE_NAME is the name of your device in the Particle cloud. You must be logged into particle-cli to use this feature. You can log into particle-cli with particle cloud login.

You can also flash code to multiple devices at once by passing the -m or --multi argument to ota. This would look like po DEVICE_TYPE ota -m. This relies on a file called devices.txt that you must create in your po-util project directory.

NOTE: This is different from the product firmware update feature in the Particle Console because it updates the firmware of devices one at a time and only if the devices are online when the command is run.

Testimonials

"I think you identified a common pain point for a lot of users, and the automated installation of the Particle toolchain is a problem that we've been focused on a lot recently ... you should already feel proud of yourself because you have made a meaningful contribution to the community. We are all very thankful for it - our community regularly impresses us, and you are certainly not an exception."

Will Hart, Particle General Manager

---

"I found that po-util works wonders by condensing the many disparate steps required to automate command-line interactions with Particle products into simple, clear commands. No need to futz with the weird serial stuff on Linux, and nicely full featured. So much better than pressing the buttons, even if you aren’t automating anything else. Highly recommend!"

Justice Reed, Head of Hardware, Amper Technologies

---

"It has been nearly flawless and a HUGE productivity booster... I cannot believe your steps worked installing so many utils and dependencies perfectly... The Particle world owes you a big debt. If there were Emmy awards for Particle contribs, you would get one."

Andrew Ward, Particle Community Member

---

"Nice! This is great stuff. Definitely helpful for the local dev / offline use case. Thanks for sharing."

Avidan Ross, Particle Investor

---

"Thanks a lot for this amazing tool. I finally managed to get everything as I wanted: to be able to work off-line."

Yannick, Particle Community Member

---

"Po-util is a very handy script - thanks for sharing."

H.S, Particle Community Member

---

"I can't heart this enough!"

Josh Fisher, Particle Community Member

po-util's Issues

Join my slack!

If you guys want, you can join my Slack. Invite yourselves at http://nrobinson2000.herokuapp.com/
@mrmowgli @monkbroc

Project structure compatibility?

I've created a similar project using po-util and the Particle CLI.

As per the documentation, po-util project structure is

while Particle's project structure is

How to make the project created with po-util compatible with Particle CLI?

Gcc-arm produces error when space in directory path

https://community.particle.io/t/po-util-a-toolchain-installer-helper-for-linux-and-osx/21015/38?u=nrobinson2000

Should po-util have a setup/config wizard?

I was thinking on creating an easy to use configuration wizard for when po-util is installed. The purpose of the wizard would be to help the user edit the ~/.po configuration file.

Possible features include:

• Selecting which particle firmware branch to use
• Selecting the Linux friendly DFU trigger baud rate (19200) or the default Particle rate (14400).

Device goes into DFU mode unintentionally when opening serial port on Linux

I did some testing for po-util on my Linux Mint desktop yesterday. Everything went smoothly, except for when I tried running po serial after using any of my DFU commands that make use of stty to change the baud rate to 19200.

• My DFU commands do stty -F /dev/ttyACM0 19200 to put my Particle devices into DFU mode.
• po serial simply does screen -S particle /dev/ttyACM0 to open up a serial monitor.

I can do po dfu-close, which gets device out of DFU mode by writing null bytes to it, but po serial still puts the device into DFU mode.

The only solution I have found is to unplug and plug in the USB cable several times, and then po serial behaves normally.

@mrmowgli, can you try this on your Linux machine? Update po-util and then try doing po dfu-open and po serial, using a device with the 19200 DFU baud rate patch.

Improved udev.rule to avoid uninstalling modemmanger on Linux.

po installs a udev rule, which currently has issues with the Linux ModemManager on Debian and Ubuntu flavors of Linux. We can change the rule to force ModemManager to ignore our device.

Append the following to the current rules:

, ENV{ID_MM_DEVICE_IGNORE}="1"

I need users to test if po-util installs on other Linux distributions.

I have made po-util compatible with RHEL and Arch based distros by making po-util detect what package manager a system has. I think it would be better to use one of the methods on this website http://www.cyberciti.biz/faq/find-linux-distribution-name-version-number/, perhaps using cat /etc/*-release and grep to find the names of distributions to choose the appropriate package manager.

I need to make sure that po-util will install properly on fresh RHEL distributions like Fedora and Arch distributions. There are no CI's that will provide RHEL or Arch, so it would be great if any users of these distributions could test if po-util installs correctly on their system.

oh-my-zsh compatibility

Could you please make this compatible with oh-my-zsh, I had difficulty understanding why this wouldnt work after being installed but discovered during the install that it couldnt find my .bashrc file

...
Shoud po-util automatically add and remove headers when using
    libraries?
(yes/no): yes

grep: /Users/username/.bashrc: No such file or directory
...

prompting me to use bash which worked fine

bash-3.2$ po init photon newProject

Directory initialized as a po-util project for photon

however when previously using oh-my-zsh, I was getting the following error:

po install
~/dev ~

po init photon newProject       
popd: too many arguments

po-util should only install missing components if possible.

I'm realizing there are some parts of the script which may need some tweaking, specifically the following cases:

Installation

• User has multiple ARM-gcc installs for different platforms
• User has a more recent version of Node.js around
• User already has brew installed (OSX)

Updating

• Updating the particle-cli util
• updating firmware sources
• supporting multiple branches or checkout latest by default

Misc

• User has a combination of Photon and Electron devices to support

Possible tasks:

• For ARM-gcc installs, check versions and see if it's necessary to install
• For Ubuntu builds (and possibly OSX) download gcc-arm-abi-none and run from ~/bin
• Have po-util set bin paths on run, and (possibly) export paths to put compiler first in path
• Check Node Versions and only install if neccesary
• Check for Brew existence
• Set the default firmware branch to latest

Support extra build flags

It would be really awesome to have the ability to pass build flags (such as -DBUILD_VERSION="nightly-20161121" or whatever) as part of the po build command. Unless I'm missing something, I haven't found where this is an option in the existing system.

Create a better logo for po-util

Hi guys, I created this simple logo in Photoshop today, but I think it is possible to make a better logo for po-util. If you have any suggestions please post them here.

Support for third party compounds

The official Particle Firmware only supports building for Raspberry Pi, Electron, P1, Photon and Core. Third party compounds like Bluz, and Oak are not supported in the Particle Firmware, and it is because of this that po-util can not be used with these compounds.

I apologize for any inconvenience.

dfu-util still requires sudo on linux to run

I believe this is a udev issue, I am in the plugdev and dialout groups.

without sudo, dfu-util can't identify the photon:

$ dfu-util -l

dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Cannot open DFU device 0489:e042
dfu-util: Device has DFU interface, but has no DFU functional descriptor
Deducing device DFU version from functional descriptor length
dfu-util: Cannot open DFU device 1235:800c
dfu-util: Device has DFU interface, but has no DFU functional descriptor
Deducing device DFU version from functional descriptor length
dfu-util: Cannot open DFU device 1235:800c

and with sudo:

$ sudo dfu-util -l
dfu-util 0.9

Copyright 2005-2009 Weston Schmidt, Harald Welte and OpenMoko Inc.
Copyright 2010-2016 Tormod Volden and Stefan Schmidt
This program is Free Software and has ABSOLUTELY NO WARRANTY
Please report bugs to http://sourceforge.net/p/dfu-util/tickets/

dfu-util: Device has DFU interface, but has no DFU functional descriptor
Deducing device DFU version from functional descriptor length
dfu-util: Device has DFU interface, but has no DFU functional descriptor
Deducing device DFU version from functional descriptor length
Found Runtime: [1235:800c] ver=027a, devnum=2, cfg=1, intf=5, path="3-3", alt=0, name="Scarlett 18i20 USB-DFU", serial="UNKNOWN"
Found Runtime: [1235:800c] ver=027a, devnum=2, cfg=1, intf=5, path="3-3", alt=0, name="Scarlett 18i20 USB-DFU", serial="UNKNOWN"
Found Runtime: [0489:e042] ver=0112, devnum=7, cfg=1, intf=3, path="1-1.6", alt=0, name="UNKNOWN", serial="08EDB9D980E0"

Missing curl command doesn't fail

When curl isn't detected on the system, ./po-util should error out early and give the reason.

Out of multiple devices, select the device to flash to?

We are in a situation where we are developing code for two separate photons, who each need a different firmware (think of a master and slave device). When we want to flash the master firmware to the master device, or the slave firmware to the slave device, we have to disconnect the other device from USB to flash it.

Ideally, we would like to select the device that needs to be flashed to, as an option for po commands, instead of autodetecting devices. For example, being able to use po photon flash --device /dev/cu.usbmodem14511 or po dfu-open -d /dev/cu.usbmodem14511 would be ideal.

Is this supported? We cannot seem to find something along these lines in the documentation.

Thank you,
Anthony,-

Missing path issue with using libraries

I often encounter the following issue with using libraries using po-util.

In some libraries, certain files within the library include other files of that library assuming that the path is known by the compiler. E.g. many of the libraries come with examples folders which include the parent library files assuming a certain path structure. However, when doing po <DEVICE> build, the build fails because of path issues saying that no such file exists. What I do currently is that I simply remove those example folders, but that's a temporary solution.

How do we change the include path of the compiler when using po-util in such cases? Or, is there some other solution?

Improve documentation and update man page etc.

Document #39

Update wiki and website, fix old links to go to https://po-util.com

po-util colliding with popd alias

Just finished the install of po-util and when I type po from a terminal prompt I get a message about popping the last directory. Decided to run which po and get the following:
po: aliased to popd

I'm running on a Mac and not sure how to get around this. Is there a way to re-alias po-util to something else?

po-util.com

I purchased po-util.com today from Namecheap using bitcoin. I am using it with my Digital Ocean server to host the same website as on github pages, and as my own link shortener because bit.ly links are not changeable.

To install my script, all you now need to do is to run:

curl -fsSL po-util.com/download | bash

Bountysource Donations

What to do if dfu-util can't be downloaded

If you get an error like this: https://travis-ci.org/nrobinson2000/po-util/builds/346118492

Try the following fix:

cd ~/.po-util/src
curl -sLO https://cytranet.dl.sourceforge.net/project/dfu-util/dfu-util-0.9.tar.gz
tar -xvf dfu-util-0.9.tar.gz
mv dfu-util-0.9 dfu-util
po install

Should I disable Linux Builds for Travis CI?

Recently Travis CI has been always failing when testing po-util for Linux, and I do not think that this is an accurate reflection of po-util. I am proposing to use Travis CI exclusively for OSX builds, and to continue doing Linux builds on Circle CI.

Make it so that po-util supports adding libraries better

I want to make it so that po-util will support adding a lib/ folder in the root of a project. This means that a full po-util project would look like this:

po-util_project/
  ├ firmware/
  | └ main.cpp
  ├ lib/
  | ├ lib1.cpp
  | └ lib1.h
  ├ bin/
  | ├ firmware.bin
  | └ ...
  ├ devices.txt
  └ README.md

This would be a good feature because it would let users organize their libraries better but putting them in a separate directory.

I have tried creating a function for editing the firmware/user/build.mk file by adding something like INCLUDE_DIRS += $(SOURCE_PATH)/../lib, but the libraries do not seem to get imported properly.

conflicting types for 'STM32_Pin_Info'

I got the following error when trying to build a simple "hello wold" project using po-util to build:
I am using Ubuntu 18.04
device: photon

In file included from /home/ai/Particle/firmware/hal/src/stm32f2xx/dac_hal.c:30: /home/ai/Particle/firmware/hal/src/stm32f2xx/pinmap_impl.h:47:3: error: conflicting types for 'STM32_Pin_Info' 47 | } STM32_Pin_Info; | ^~~~~~~~~~~~~~ In file included from ../hal/inc/pinmap_hal.h:64, from ../hal/inc/dac_hal.h:31, from /home/ai/Particle/firmware/hal/src/stm32f2xx/dac_hal.c:27: ../hal/src/stm32f2xx/pinmap_impl.h:40:22: note: previous declaration of 'STM32_Pin_Info' was here 40 | typedef Hal_Pin_Info STM32_Pin_Info; | ^~~~~~~~~~~~~~ In file included from /home/ai/Particle/firmware/hal/src/stm32f2xx/dac_hal.c:30: /home/ai/Particle/firmware/hal/src/stm32f2xx/pinmap_impl.h:50:17: error: conflicting types for 'HAL_Pin_Map' 50 | STM32_Pin_Info* HAL_Pin_Map(void); | ^~~~~~~~~~~ In file included from ../hal/inc/pinmap_hal.h:64, from ../hal/inc/dac_hal.h:31, from /home/ai/Particle/firmware/hal/src/stm32f2xx/dac_hal.c:27: ../hal/src/stm32f2xx/pinmap_impl.h:42:15: note: previous declaration of 'HAL_Pin_Map' was here 42 | Hal_Pin_Info *HAL_Pin_Map(void); | ^~~~~~~~~~~ ../build/module.mk:269: recipe for target '../build/target/user/platform-6-m/firmware/hal/src/stm32f2xx/dac_hal.o' failed make[2]: *** [../build/target/user/platform-6-m/firmware/hal/src/stm32f2xx/dac_hal.o] Error 1 ../../../build/recurse.mk:11: recipe for target 'user' failed make[1]: *** [user] Error 2 ../build/recurse.mk:11: recipe for target 'modules/photon/user-part' failed make: *** [modules/photon/user-part] Error 2

Sorry if I put this in a wrong place!

Po lib get should see if the exact library is already installed

For example if you already have the neopixel library, po lib get neo would exit because it already grepped "neo" out of "neopixel"