platsch / octopnp

OctoPrint plugin for camera based pick 'n place control

OctoPNP

OctoPNP is an OctoPrint plugin for camera assisted pick 'n place applications in 3D-Printers.

Introduction

OctoPNP is an extension that allows Octoprint to control printers and similar devices with additional hardware for handling of SMD-parts. It currently requires the following hardware extensions:

• A Tray consisting of a grid of boxes to store SMD parts in a defined position
• A head camera to locate the exact part position on the tray
• A (second) bed camera to precisely align the parts during the placing operation
• A vacuum nozzle to grip parts

Installation

Prerequirements

To achieve higher compatibility and modularity, OctoPNP doesn't acces the cameras directly. Every time an image is required, OctoPNP executes a user defined script which must be adapted for every installation according to the deployed camera setup. OctoPNP expects a set of correctly cropped and rotated images after executing the script. Filenames and path for images and script must be set in the settings dialog. The calibration feature supports direct camera access via http requests to generate a live view e.g. via mjpg_streamer.

OpenCV on embedded hardware (Raspberry Pi)

OctoPNP requires numpyand OpenCV, both are listed in the requirements.txt and should be automatically resolved via pip during installation. However, on many embedded systems, including the RasPi, opencv-python is not available. To solve this:

• Remove (or comment) the opencv-python line in requirements.txt
• Manually install OpenCV on your system (official tutorial). This should build OpenCV packages for Python 2 and 3 if both versions are installed on the system.
• If OctoPrint is installed in a virtual environment (as recommended), at least in some cases the OpenCV library must be linked (or copied) into the venv like this:

ln -s /usr/local/lib/python3.6/dist-packages/cv2/python-3.6/cv2.cpython-36m-arm-linux-gnueabihf.so ~/OctoPrint/venv3/lib/python3.6/site-packages/

Installing the package

The plugin itself can be installed as any regular python package: pip install https://github.com/platsch/OctoPNP/archive/master.zip

Make sure you use the same Python environment that you installed OctoPrint under, otherwise the plugin won't be able to satisfy its dependencies. Further information can be found in the Octoprint documentation OctoPNP requires numpy and opencv2. Try installing those packages by hand if automatic dependency resolving failes.

Data Format

The information for the PNP system is integrated into normal gcode files as a commented XML flavor somewhere in the gcode. OctoPNP extracts the required information automatically everytime a gcode file is loaded in Octoprint. Example files are included in the utils folder. The XML structure is still under development and will probably change during the next months. The comment-semicolon causes Octoprint to ignore the SMD-data.

;normal printing commands
G1 X10 F3000
G1 Y10 F3000
G1 Z5 F3000

;pick part nr 2 and place it to the destination given in the XML description
M361 P2

;description of the actual parts
;<object name="simple_lines">
;<part id="1" name="ATTINY">
;  <position box="1"/>
;  <size height="1"/>
;  <shape>
;    <point x="-2.6" y="-3.4"/>
;    <point x="-2.6" y="3.4"/>
;    <point x="2.6" y="3.4"/>
;    <point x="2.6" y="-3.4"/>
;  </shape>
;  <pads>
;    <pad x1="-2.155" y1="-4.254" x2="-1.655" y2="-2.054"/>
;    <pad x1="-0.895" y1="-4.254" x2="-0.395" y2="-2.054"/>
;    <pad x1="0.375" y1="-4.254" x2="0.875" y2="-2.054"/>
;    <pad x1="1.645" y1="-4.254" x2="2.145" y2="-2.054"/>
;    <pad x1="-2.155" y1="2.054" x2="-1.655" y2="4.254"/>
;    <pad x1="-0.885" y1="2.054" x2="-0.385" y2="4.254"/>
;    <pad x1="0.385" y1="2.054" x2="0.885" y2="4.254"/>
;    <pad x1="1.655" y1="2.054" x2="2.155" y2="4.254"/>
;  </pads>
;  <destination x="100" y="90" z="2.75" orientation="90"/>
;</part>
;
;<part id="2" name="RES_1206">
;  <position box="2"/>
;  <size height="1"/>
;  <shape>
;    <point x="-1.6891" y="-0.8763"/>
;    <point x="-1.6891" y="0.8763"/>
;    <point x="1.6891" y="0.8763"/>
;    <point x="1.6891" y="-0.8763"/>
;  </shape>
;  <pads>
;    <pad x1="0.622" y1="-0.9015" x2="2.222" y2="0.9015"/>
;    <pad x1="-2.222" y1="-0.9015" x2="-0.622" y2="0.9015"/>
;  </pads>
;  <destination x="108.304" y="90.674" z="2.75" orientation="0"/>
;</part>
;</object>

Configuration

Good configuration and calibration of the printer is absolutely crucial to successfully use multiple extruders and cameras.

Tray

The tray-position is set in relation to the primary extruder (usually the plastic extruder). To find the position, move the primary extruder to the bottom left corner of the tray and note the position. A negative Z-offset can be used if the tray is lower than the printbed.

Extruders / Nozzles

The minimal setup typically requires 3 nozzles / tools:

• The plastic extruder to print the object (primary extruder)
• An Extruder for liquids to print the conductive wires
• A vacuum nozzle to grip SMD-parts

Offsets between tools can be handled by the slicer, by OctoPNP or by the printer firmware. Having the firmware manage all offsets is generally highly encouraged to avoid frequent G-code generation, since for most setups the offset has to be re-calibrated at least after every refill of the extruder. The offset for the liquid extruder must be handled by the slicer or by the firmware, OctoPNP is not aware of this extruder. The offset for the vacuum nozzle must be handled by OctoPNP or by the firmware. OctoPNP provides a calibration tool to quickly correct the firmware offsets.

For toolchanger setups all offsets should generally be measured in relation to a fixed point at the toolhead. This way, once a tool is mounted, the bed camera and tray positions are automatically in the correct frame. If the head camera is mounted to the toolhead instead of being an individual tool, all images should probably be taken with no tool attached (extruder T-1).

For setups with all tools mounted to a single carriage offsets should generally be measured in relation to the first extruder (T0).

Cameras

Position

The camera position is relative to the primary nozzle for the head camera (this camera is mounted somewhere next to the extruder at the X-axis and follows the printheads movements). The bed camera is mounted to the printers frame, the camera position is absolute and again relative to the primary extruder.

• Focus distance defines the printbead Z-position for the optimal focus point.
• Focus axis sets the name of the axis to move the camera. Default is Z for a camera mounted to the printbed. RepRapFirmware supports ABCUV and W to address additional axis. Leave blank if the camera is entirely static and the printhead cannot be moved with respect to the camera in the vertical dimension.
• The px/mm value is used by the image processing to measure offsets and only required for the bed camera, since the head camera utilizes the known box size for on demand calibration.
• The binarization threshold is used by the image processing to seperate object and background. The value depends on the camera and illumination, currently there is no good calibration method.
• Image path and grab script path denote the locations of script and preprocessed images as described in the Prerequirements section.
• http-path can be used if the camera provides an http interface (e.g. with mjpg_streamer). This is only used to generate a live view during calibration, not for actual pick and place operations. Leave empty to use the script base method for calibration. Note that for technical reasons only still-images can be used, mjpg streams or other video formats are not supported!

Image acquisition

The cameras folder contains simple scripts to fetch single images from

• any camera which can be accessed with an http get command (http)
• Basler industrial cameras supporting the pylon framework (pylon, tested with Pilot and Ace models)

Calibration wizard

The calibration "wizard" is an experimental set of tools to quickly achieve several calibration tasks with high precision.

• Calibration of extruders is done by printing a defined pattern (e.g. a cross) on the printbed. To calibrate an offset, the head camera moves to the printed pattern (position given by user) and the user uses the arrows keys to correct possible offsets by using the camera image superimposed by cross hairs. Offsets between extruders can be directly saved to the eeprom if a repetier firmware is used.
• The tray position can be calibrated similar by moving the camera to the corners and correcting possible offsets with manual key control.
• The bed camera can be calibrated against any calibrated extruder that is able to move over the camera. While some extruders contain sticky, liquid content, it might be a good idea to cover the camera with some glass.
• With a calibrated bed camera any extruder that is able to move over the camera, including the vacuum nozzle, can be calibrated.

Development / Code Analyzer

The included static code analyzer scripts might help to find potential errors and give suggstions for possible better solutions.

Preparing for Windows

Installing shellcheck:

1. See shellcheck

Installing pylint:

1. Follow this instruction to install python and pip
2. You may add the python*/Scripts folder to the Path ENV
3. pip install pylint

Installing clang-tidy:

1. See clang-tidy

Enable git hook

For automatic code analysis before every commit, add this hook to your local git installation by symlinking it:

ln -s ../../.ci/pre-push-hook .git/hooks/pre-commit

octopnp's Issues

Cannot install OctoPNP

Hi all,
using the OctoPrint Version 1.4.0, trying to install the OctoPNP plugin I get the following error:

Installing plugin "OctoPNP" from https://github.com/platsch/OctoPNP/archive/master.zip...
/home/pi/oprint/bin/python2 -m pip --disable-pip-version-check install https://github.com/platsch/OctoPNP/archive/master.zip --no-cache-dir
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting https://github.com/platsch/OctoPNP/archive/master.zip
Downloading https://github.com/platsch/OctoPNP/archive/master.zip
Requirement already satisfied: OctoPrint in /home/pi/oprint/lib/python2.7/site-packages (from OctoPNP==0.2) (1.4.0)
Collecting numpy
Downloading https://files.pythonhosted.org/packages/b7/6f/24647f014eef9b67a24adfcbcd4f4928349b4a0f8393b3d7fe648d4d2de3/numpy-1.16.6.zip (5.1MB)
ERROR: Could not find a version that satisfies the requirement opencv-python (from OctoPNP==0.2) (from versions: none)
ERROR: No matching distribution found for opencv-python (from OctoPNP==0.2)
Error!
Could not parse output from pip, see plugin_pluginmanager_console.log for generated output

Thank you

Javascript error during to reload Octoprint

Hello,

i found a solutiuon to install OctoPNP on an RPI3. But if i reload my Octoprint then i run in this javascript error:

Uncaught TypeError: Cannot read property 'setAttribute' of null
    at new OctoPNPViewModel (packed_plugins.js?0afb2d09:584)
    at _createViewModelInstance (packed_core.js?f7d19c53:13069)
    at HTMLDocument.<anonymous> (packed_core.js?f7d19c53:13147)
    at fire (packed_libs.js?d8971830:3187)
    at Object.fireWith [as resolveWith] (packed_libs.js?d8971830:3317)
    at Function.ready (packed_libs.js?d8971830:3536)
    at HTMLDocument.completed (packed_libs.js?d8971830:3552)
OctoPNPViewModel @ packed_plugins.js?0afb2d09:584
_createViewModelInstance @ packed_core.js?f7d19c53:13069
(anonymous) @ packed_core.js?f7d19c53:13147
fire @ packed_libs.js?d8971830:3187
fireWith @ packed_libs.js?d8971830:3317
ready @ packed_libs.js?d8971830:3536
completed @ packed_libs.js?d8971830:3552

I checked this part and here is the code wher the error is happend:

        //white placeholder images
        document.getElementById('headCameraImage').setAttribute( 'src', 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH3wMRCQAfAmB4CgAAABl0RVh0Q29tbWVudABDcmVhdGVkIHdpdGggR0lNUFeBDhcAAAAMSURBVAjXY/j//z8ABf4C/tzMWecAAAAASUVORK5CYII=');
        document.getElementById('bedCameraImage').setAttribute( 'src', 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAsTAAALEwEAmpwYAAAAB3RJTUUH3wMRCQAfAmB4CgAAABl0RVh0Q29tbWVudABDcmVhdGVkIHdpdGggR0lNUFeBDhcAAAAMSURBVAjXY/j//z8ABf4C/tzMWecAAAAASUVORK5CYII=');

Version: OctoPrint 1.3.6.dev37+gb161a1f (maintenance branch)

Installation error with the OctoPrint

Installation does not work. Gives error message:"Could not parse output from pip".
I'm using OctoPrint version 1.3.2 master branch.

For-hire Slic3r work

Hello !

Really sorry for contacting you through an issue, I did not find your email anywhere, so this is the only way I found of getting your attention. Please feel free to ignore this if this is any kind of a bother, I apologize immensely if it is.

I am looking for somebody to hire to do custom work in Slic3r. We are talking about adding features, some for my company, others for two other companies also interested in Slic3r improvements. The features ( most of them ) would be generally useful to users in the community, and therefore we would essentially be paying somebody ( could be you ) to improve Slic3r.

Are you interested/do you have time to do for-hire work coding new features into Slic3r? If so please contact me. If you don't have the time or don't have the interest, could you recommend somebody you think might have time and/or interest in doing so?
This is paid work, the code would, of course, be Open-Source, and most of the code would be generally useful to the Open-Source 3D printing community, this is a win-win-win scenario ( at least from my perspective ).

My contact is : [email protected]

Thanks again !! If you consider this spam I apologize again !

where is the hardware documented?

OctoPNP is an extension that allows Octoprint to control printers and similar devices with additional hardware

Can you point me to somewhere that talks about the additional hardware?

OctoPNP on windows

How can I install (compile) on Windows 64.. thanks.