Can we please get a Mac version? about pid-analyzer

[BRadFPV]

@piperod - I am making progress with virtualenv. No errors so far... Working on generating output currently. Looking like Plasmatree is working! Thanks for hanging in there with me! Got PNGs output!

[Plasmatree]

Instead of buildig blackbox-tools from source, you can use the prebuild version for mac: https://github.com/cleanflight/blackbox-tools/releases

If you got problems installing python and the required packages, you can just install anaconda, which I'm using too. https://www.anaconda.com/download/

Unfortunately I don't have a mac to compile on and haven't set up a virtual machine yet.
Somebody with a mac could try compile it via https://www.pyinstaller.org/
It's basically a one-liner (at least on windows).

[haplm]

I have a mac and I'd like to help with mac releases, but unfortunately it won't be that easy as just running pyinstaller. If you do so, you will get following error when trying to run the resulting executable:

ImportError: C extension: No module named 'pandas._libs.tslibs.timedeltas' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.

I found a solution for that issue here, but then I'm getting errors around matplotlib, with following explanation:

RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.

It seems like it will require a bit more investigation.

Attached is the modified .spec file PID-Analyzer.spec.zip

[spuder]

I've started work on having travis-ci building the mac app in this branch

https://github.com/spuder/PID-Analyzer/blob/travis/.travis.yml

https://travis-ci.org/spuder/PID-Analyzer

Currently I'm stuck on that same step

lldb ./PID-Analyzer.app/
(lldb) target create "./PID-Analyzer.app/"
Current executable set to './PID-Analyzer.app/' (x86_64).
(lldb) process launch
Process 30309 launched: '/private/tmp/PID-Analyzer/dist/PID-Analyzer.app/Contents/MacOS/PID-Analyzer' (x86_64)
Traceback (most recent call last):
  File "PID-Analyzer.py", line 8, in <module>
  File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
    exec(bytecode, module.__dict__)
  File "pandas/__init__.py", line 35, in <module>
ImportError: C extension: No module named np_datetime not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
[30309] Failed to execute script PID-Analyzer
Process 30309 exited with status = 255 (0x000000ff)

[Plasmatree]

I had problems with pandas and scipy and had to add some hidden imports:
pyinstaller -F --noupx PID-Analyzer_0.52.py --hidden-import scipy._lib.messagestream --hidden-import pandas._libs.tslibs.timedeltas

In case you havn't installed pandas or any other requirement, you need to do that. You need to be able to run it, before you can tell pyinstaller to put everything needed into a file.

Travis-ci looks interesting. It let's you openly build on any platform they provide?

[haplm]

I've tried to build it with your command-line, but unfortunately it still fails to start:

MartinH:dist haplm$ ./PID-Analyzer Fontconfig warning: "/usr/local/etc/fonts/fonts.conf", line 146: blank doesn't take any effect anymore. please remove it from your fonts.conf Fontconfig warning: ignoring UTF-8: not a valid region tag Traceback (most recent call last): File "PID-Analyzer.py", line 10, in <module> File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "/usr/local/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 631, in exec_module exec(bytecode, module.__dict__) File "matplotlib/pyplot.py", line 115, in <module> File "matplotlib/backends/__init__.py", line 32, in pylab_setup File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 665, in _load_unlocked File "/usr/local/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 631, in exec_module exec(bytecode, module.__dict__) File "matplotlib/backends/backend_macosx.py", line 19, in <module> File "<frozen importlib._bootstrap>", line 971, in _find_and_load File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 656, in _load_unlocked File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible File "/usr/local/lib/python3.6/site-packages/PyInstaller/loader/pyimod03_importers.py", line 714, in load_module module = loader.load_module(fullname)

I don't know much about Python, unfortunately, so I'm kind of stuck. Unless somebody else solves this, I'll try some of my colleagues to see if they can help (in couple weeks timeframe).

[spuder]

I took another stab at getting this working.

Local Mac

I'm able to build PID-Analyzer using the commands provided

pyinstaller -F --noupx PID-Analyzer_0.52.py --hidden-import scipy._lib.messagestream --hidden-import pandas._libs.tslibs.timedeltas

I still get this error when I try and run it

lldb ./dist/PID-Analyzer.app
(lldb) target create "./dist/PID-Analyzer.app"
Current executable set to './dist/PID-Analyzer.app' (x86_64).
(lldb) process launch
Process 61542 launched: '/Users/sowen/Code/PID-Analyzer/dist/PID-Analyzer.app/Contents/MacOS/PID-Analyzer' (x86_64)
Traceback (most recent call last):
  File "/Users/sowen/Code/PID-Analyzer/dist/PID-Analyzer.app/Contents/Resources/__boot__.py", line 168, in <module>
    _run()
  File "/Users/sowen/Code/PID-Analyzer/dist/PID-Analyzer.app/Contents/Resources/__boot__.py", line 162, in _run
    exec(compile(source, script, 'exec'), globals(), globals())
  File "/Users/sowen/Code/PID-Analyzer/PID-Analyzer.py", line 8, in <module>
    from pandas import read_csv
ImportError: No module named pandas
2018-06-25 22:02:35.469106-0600 PID-Analyzer[61542:6545745] PID-Analyzer Error
Process 61542 exited with status = 255 (0x000000ff)

Even though I've tried the solutions suggested here to install the pandas hook

https://stackoverflow.com/questions/47318119/no-module-named-pandas-libs-tslibs-timedeltas-in-pyinstaller

    if [ ! -e '/Library/Python/2.7/site-packages/PyInstaller/hooks/hook-pandas.py' ]; then
      echo "Writing out hooks-pandas.py"
      echo "hiddenimports = ['pandas._libs.tslibs.np_datetime','pandas._libs.tslibs.nattype','pandas._libs.skiplist']" | sudo tee /Library/Python/2.7/site-packages/PyInstaller/hooks/hook-pandas.py
    else
      echo "hook-pandas.py already exists"
    fi

Travis-CI

On the fresh OSX environment provided by travis-ci, I'm still not able to get pyinstaller installed properly. I'm trying to avoid sudo and instead do pip install PyInstaller --user, but pyinstaller does not appear to be installed.

https://travis-ci.org/spuder/PID-Analyzer/builds/396695008

Advice from anyone who knows pip would be greatly appreciated.

[BRadFPV]

I too am having issues with pandas:

bash-3.2$ python PID-Analyzer.py --blackbox_decode ../blackbox-tools/obj/blackbox_decode -l ~/Downloads/BTFL_BLACKBOX_LOG_Amped-Air_20180702_sp4.BBL Traceback (most recent call last): File "PID-Analyzer.py", line 8, in from pandas import read_csv ImportError: No module named pandas

Pandas appears to have loaded properly:

Installing collected packages: numpy, scipy, six, python-dateutil, pytz, pandas, subprocess32, pyparsing, functools32, cycler, matplotlib
Successfully installed cycler-0.10.0 functools32-3.2.3.post2 matplotlib-2.0.0 numpy-1.11.3 pandas-0.22.0 pyparsing-2.2.0 python-dateutil-2.7.3 pytz-2018.5 scipy-1.0.0 six-1.11.0 subprocess32-3.5.2

I verified that pandas exists:

ultibots:PID-Analyzer Brad$ pip install pandas
Requirement already satisfied: pandas in /usr/local/lib/python2.7/site-packages (0.22.0)
Requirement already satisfied: python-dateutil in /usr/local/lib/python2.7/site-packages (from pandas) (2.7.3)
Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python2.7/site-packages (from pandas) (1.11.3)
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python2.7/site-packages (from pandas) (2018.5)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python2.7/site-packages (from python-dateutil->pandas) (1.11.0)

Any direction appreciated - TIA - Brad

[BRadFPV]

OK... I was able to successfully build

pyinstaller -F --noupx PID-Analyzer_0.52.py --hidden-import scipy._lib.messagestream --hidden-import pandas._libs.tslibs.timedeltas

See attached output file. pyinstaller.txt

But I too have errors with Python not installed as a framework:

ultibots:~ Brad$ /Users/Brad/PID-Analyzer/dist/PID-Analyzer ; exit;
Traceback (most recent call last):
File "PID-Analyzer.py", line 10, in
File "/private/var/folders/fr/fb37fbsd39b660n00dh6syvw0000gn/T/pip-install-EvWD8T/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
File "matplotlib/pyplot.py", line 115, in
File "matplotlib/backends/init.py", line 32, in pylab_setup
File "/private/var/folders/fr/fb37fbsd39b660n00dh6syvw0000gn/T/pip-install-EvWD8T/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 396, in load_module
File "matplotlib/backends/backend_macosx.py", line 19, in
File "/private/var/folders/fr/fb37fbsd39b660n00dh6syvw0000gn/T/pip-install-EvWD8T/pyinstaller/PyInstaller/loader/pyimod03_importers.py", line 687, in load_module
RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework. See the Python documentation for more information on installing Python as a framework on Mac OS X. Please either reinstall Python as a framework, or try one of the other backends. If you are using (Ana)Conda please install python.app and replace the use of 'python' with 'pythonw'. See 'Working with Matplotlib on OSX' in the Matplotlib FAQ for more information.
[2789] Failed to execute script PID-Analyzer

Would love to get this working. Thanks - Brad

[piperod]

Hey @BRadFPV did you get it to work ?

Something that worked for me was adding this line to PID-Analyzer.py at the beginning:

import matplotlib
matplotlib.use('agg')

Maybe this works for you too.

[BRadFPV]

@piperod - tried again after upgrading to Mojave and reinstalling XCode, CLT, and updating Brew, Git, etc.. Added your suggestion and I am still stuck with pandas errors.

[piperod]

hey @BRadFPV could you put your full log again. I will try to replicate.

[BRadFPV]

@piperod - my pal RichFPV suggested I add the following to my bash_profile to simplify running Plasmatree.

function plasmatree() { python ~/plasmatree/PID-Analyzer/PID-Analyzer.py --blackbox_decode ~/plasmatree/blackbox-tools/obj/blackbox_decode -l $1 }

The following are the errors I am now getting:

`ultibots:PID-Analyzer Brad$ plasmatree good_tune.BBL
Traceback (most recent call last):
File "/Users/Brad/plasmatree/PID-Analyzer/PID-Analyzer.py", line 10, in
from pandas import read_csv
ImportError: No module named pandas
/Users/Brad/plasmatree/blackbox-tools/obj/blackbox_decode: option '-l' is invalid

ultibots:PID-Analyzer Brad$ pip install wheel
Requirement already satisfied: wheel in /usr/local/lib/python2.7/site-packages (0.31.1)

ultibots:PID-Analyzer Brad$ pip install pandas
Requirement already satisfied: pandas in /usr/local/lib/python2.7/site-packages (0.22.0)
Requirement already satisfied: python-dateutil in /usr/local/lib/python2.7/site-packages (from pandas) (2.7.3)
Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python2.7/site-packages (from pandas) (1.11.3)
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python2.7/site-packages (from pandas) (2018.5)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python2.7/site-packages (from python-dateutil->pandas) (1.11.0)

ultibots:PID-Analyzer Brad$ plasmatree good_tune.BBL
Traceback (most recent call last):
File "/Users/Brad/plasmatree/PID-Analyzer/PID-Analyzer.py", line 10, in
from pandas import read_csv
ImportError: No module named pandas
/Users/Brad/plasmatree/blackbox-tools/obj/blackbox_decode: option '-l' is invalid`

Thanks for looking at this.

[piperod]

@BRadFPV
Can you run this line :

pip install pandas

and try again?

[BRadFPV]

I ran 'pip install pandas' again and got the same result:

ultibots:PID-Analyzer Brad$ pip install pandas
Requirement already satisfied: pandas in /usr/local/lib/python2.7/site-packages (0.22.0)
Requirement already satisfied: python-dateutil in /usr/local/lib/python2.7/site-packages (from pandas) (2.7.3)
Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python2.7/site-packages (from pandas) (1.11.3)
Requirement already satisfied: pytz>=2011k in /usr/local/lib/python2.7/site-packages (from pandas) (2018.5)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python2.7/site-packages (from python-dateutil->pandas) (1.11.0)

[piperod]

It seems that your system is seeing a different python. Can you do this:

which python

just to check you are installing to the same python you are running the plasmatree.

[BRadFPV]

ultibots:~ Brad$ which python
/opt/local/bin/python

ultibots:~ Brad$ python
Python 2.7.14 (default, Sep 22 2017, 00:06:07)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

ultibots:~ Brad$ /opt/local/bin/python
Python 2.7.14 (default, Sep 22 2017, 00:06:07)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

[piperod]

So yes, it is using a different python. Your system is running : /opt/local/bin/python, but your pip ins running on /usr/local/lib/python2.7/. Let's try this :
python -m ensurepip
That makes sure pip is installed for the python your system is using. Then run again pip install pandas.

The better solution would be just to create a virtualenv:
first cd into your plasmatree-pid folder..
then :
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
python ~/plasmatree/PID-Analyzer/PID-Analyzer.py --blackbox_decode ~/plasmatree/blackbox-tools/obj/blackbox_decode