leodj / p2pro-viewer

A viewer and API for the InfiRay P2 Pro thermal camera module

InfiRay P2 Pro Viewer

⚠️ [WIP]
See below for a roadmap.

This project aims to be an open-source image viewer and API for the InfiRay P2 Pro thermal camera module.

The communication protocol was reverse-engineered, to avoid needing to include the proprietary precompiled InfiRay libraries.

Disclaimer: This is my first big Python multi-module project. So there are bound to be horrific architectural decisions and many other beginner's mistakes. Please excuse that.
The premise is to get to the first milestone with as little premature optimization as possible, otherwise I would never arrive there.

Notices

Windows

For sending vendor control transfers to the UVC camera in addition to opening the camera as a normal UVC camera, libusb needs to be installed as an upper filter driver.
This can be installed relatively easily using Zadig.

• Options > List all devices
• Select "USB Camera (Interface 0)"
• Scroll to "libusb-win32" and select "Install filter driver" from the dropdown besides "Replace driver"

Also, the camera video stream needs to be opened first before using the script to send commands to it, otherwise the call to libusb will just hang for whatever reason.

Linux

Still needs to be tested, but should work™

Where to buy

The cheapest vendor in Germany appears to be Peargear.
Pergear also has an international shop for other countries, but I'm not sure if they're the cheapest there.

Additional Resources

• Review and teardown video by mikeselectricstuff
• Extensive review thread by Fraser
• General discussion thread with some interesting resources

Roadmap

• USB Vendor Commands
  • Read/Write commands
    • "standard" cmd
    • "long" cmd
    • wait for camera ready
  • Pseudo color
  • NUC shutter control (auto/manual/trigger)
  • High/low temperature range
  • Other parameters (emissivity, distance, etc)
  • Switch to actual raw sensor readings?
  • Recalibrate lens?
  • Remaining (less relevant) commands
• Recording
  • Still image
    • JPEG and radiometry data in one file
      • Standardized format? R-JPEG?
    • Metadata (rotation, camera settings, location?, etc)
  • Video
    • MKV file with radiometry data as second lossless video track
    • Audio
    • Metadata
    • Rotation (on-the-fly possible? :D)
    • Render overlays into video (scale, min/max/center temps, etc)
    • Timelapse?
    • Min/Max/Center temperature in subtitles? :D
    • Standardized video format? (don't know any)
  • Find / build tool to analyze recordings later (or export to other formats)
  • Virtual webcam output with temperature scale overlays?
• GUI
  • Display video stream
  • Overlays
    • Temperature scale
    • Min/max/center temperature
    • Cursor hover temperature
  • Palette selection
  • Shutter control
  • Gain selection (camera temperature range)
  • Parameters (emissivity, distance, etc)
  • Recording controls (take picture, start/stop video, recording indicator and time)
  • Image rotation / flip
  • Manually set min/max range (probably need to apply own pseudo color from raw temperature data)
  • Own analysis controls (points, lines, rectangles, etc)
  • Log measurements to CSV
  • Plot measurements
  • ...
• Documentation
  • How to use
  • USB vendor command protocol
  • My video format (if I don't find a more standardized one)
  • P2Pro Android app JPEG file format, that has radiometry data embedded
• Very long-term plans:
  • Small device with a socket for the P2 Pro to convert it into a hand-held device
  • Android App

p2pro-viewer's Issues

Could not open video capture device with index 1

Running python main.py failed:

[ WARN:[email protected]] global cap_v4l.cpp:997 open VIDEOIO(V4L2:/dev/video1): can't open camera by index
[ERROR:[email protected]] global obsensor_uvc_stream_channel.cpp:159 getStreamChannelGroup Camera index out of range
Exception in thread Thread-1 (open):
Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/P2Pro-Viewer/P2Pro/video.py", line 84, in open
    raise ConnectionError(f"Could not open video capture device with index {camera_id}, is the module connected?")
ConnectionError: Could not open video capture device with index 1, is the module connected?

IR Camera Infiray P2Pro, recognized as /dev/video2 /dev/video3
System: Arch Linux

# uname -a
Linux ROG 6.8.7-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 17 Apr 2024 15:20:28 +0000 x86_64 GNU/Linux

# lsusb
Bus 003 Device 002: ID 0bda:5830 Realtek Semiconductor Corp. USB Camera

v4l2-ctl -d /dev/video2 --all
Driver Info:
	Driver name      : uvcvideo
	Card type        : USB Camera: USB Camera
	Bus info         : usb-0000:3d:00.0-1
	Driver version   : 6.8.7
	Capabilities     : 0x84a00001
		Video Capture
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Media Driver Info:
	Driver name      : uvcvideo
	Model            : USB Camera: USB Camera
	Serial           : 200901010001
	Bus info         : usb-0000:3d:00.0-1
	Media version    : 6.8.7
	Hardware revision: 0x00001101 (4353)
	Driver version   : 6.8.7
Interface Info:
	ID               : 0x03000002
	Type             : V4L Video
Entity Info:
	ID               : 0x00000001 (1)
	Name             : USB Camera: USB Camera
	Function         : V4L2 I/O
	Flags            : default
	Pad 0x01000007   : 0: Sink
	  Link 0x02000010: from remote pad 0x100000a of entity 'Extension 4' (Video Pixel Formatter): Data, Enabled, Immutable
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
	Width/Height      : 256/384
	Pixel Format      : 'YUYV' (YUYV 4:2:2)
	Field             : None
	Bytes per Line    : 512
	Size Image        : 196608
	Colorspace        : sRGB
	Transfer Function : Rec. 709
	YCbCr/HSV Encoding: ITU-R 601
	Quantization      : Default (maps to Limited Range)
	Flags             : 
Crop Capability Video Capture:
	Bounds      : Left 0, Top 0, Width 256, Height 384
	Default     : Left 0, Top 0, Width 256, Height 384
	Pixel Aspect: 1/1
Selection Video Capture: crop_default, Left 0, Top 0, Width 256, Height 384, Flags: 
Selection Video Capture: crop_bounds, Left 0, Top 0, Width 256, Height 384, Flags: 
Streaming Parameters Video Capture:
	Capabilities     : timeperframe
	Frames per second: 25.000 (25/1)
	Read buffers     : 0

User Controls

                     brightness 0x00980900 (int)    : min=-64 max=64 step=1 default=0 value=0
                       contrast 0x00980901 (int)    : min=0 max=100 step=1 default=50 value=50
                     saturation 0x00980902 (int)    : min=0 max=100 step=1 default=64 value=64
                            hue 0x00980903 (int)    : min=-180 max=180 step=1 default=0 value=0
        white_balance_automatic 0x0098090c (bool)   : default=1 value=1
                          gamma 0x00980910 (int)    : min=100 max=500 step=1 default=300 value=300
           power_line_frequency 0x00980918 (menu)   : min=0 max=2 default=1 value=1 (50 Hz)
				0: Disabled
				1: 50 Hz
				2: 60 Hz
      white_balance_temperature 0x0098091a (int)    : min=2800 max=6500 step=10 default=4600 value=4600 flags=inactive
                      sharpness 0x0098091b (int)    : min=0 max=100 step=1 default=50 value=50
         backlight_compensation 0x0098091c (int)    : min=0 max=1 step=1 default=0 value=0

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=3 default=3 value=3 (Aperture Priority Mode)
				1: Manual Mode
				3: Aperture Priority Mode
         exposure_time_absolute 0x009a0902 (int)    : min=50 max=10000 step=1 default=166 value=166 flags=inactive
     exposure_dynamic_framerate 0x009a0903 (bool)   : default=0 value=1

The camera works when using the command:

ffmpeg -input_format yuyv422 -video_size 256x384 -i /dev/video2 -vf 'crop=h=(ih/2):y=(ih/2)' -pix_fmt yuyv422 -f rawvideo - | ffplay -pixel_format gray16le -video_size 256x192 -f rawvideo -i - -vf 'normalize=smoothing=10, format=pix_fmts=rgb48, pseudocolor=p=inferno'

usb.core can't find usb device by idVendor=0x0BDA and idProduct=0x5830

The camera is connected and I can open it using the default camera app in windows 10.
If I use:

import win32com.client
wmi = win32com.client.GetObject ("winmgmts:")
for usb in wmi.InstancesOf ("Win32_USBHub"):
print(usb.DeviceID)

It returns:
USB\VID_0BDA&PID_5830\200901010001

when I run:
python main.py

It returns:
INFO:P2Pro.recorder:Starting video recording to file test.mkv ...
Traceback (most recent call last):
File "C:\path\P2Pro-Viewer-main\main.py", line 25, in
cam_cmd = P2Pro_CMD.P2Pro()
File "C:\path\P2Pro-Viewer-main\P2Pro\P2Pro_cmd.py", line 84, in init
raise FileNotFoundError("Infiray P2 Pro thermal module not found, please connect and try again!")
FileNotFoundError: Infiray P2 Pro thermal module not found, please connect and try again!

The problem is in line 84: self._dev = usb.core.find(idVendor=0x0BDA, idProduct=0x5830)

I tried this line only
import usb.core
dev = usb.core.find(idVendor=0x0BDA, idProduct=0x5830)
print(dev)

It returns:
None

Please help.

Missing requirements.txt

If I want to run this code I have to have all of the packages installed on my system. Otherwise I would have to weed through the code to find all the dependent libraries. One solution would be to add a requirements.txt file or use a dependency manager like Poetry.

Resolution/FPS of camera id 1 doesn't match. It's probably not a P2 Pro.

It seems to use my laptop camera.

This is the image I captured using obs:

Would it work with iphone version

Initially only the Android USB-C was available but now we also have iphone lightning, example here: https://www.aliexpress.com/item/1005004816021872.html

Also the iOS app: https://apps.apple.com/us/app/p2-pro/id1619041411

The reason I'm asking is that I'm trying to decide which version to buy. Since I am an iphone user, I would of course prefer the iphone version, but I would also like the ability to plug it into a computer such as raspberry pi.

Provided the physical usb adapter would not be a problem, is there any particular concern from a usb driver point of view?

Infiray types

hello,
well done on this project @LeoDJ.
I'm trying to work with 2 modules xcore micro 3 lite, but I'm trying to customize a Python code for specific requirements.
does this code work with my modules? does it read two cameras at the same time? also if this code can open an irv video recorded files.
thank you very much.
hope i hear from you soon.
Ajami

Current state

Heya!

I wanna potentially work on fitting this and jerry rigging an app for vr headsets with the purpose of hunting, what is the current state of this before I purchase one? Not hoping for like picture captures and such, just hoping for basically red is hot functionality.

Rasperry pi

Does this code work for Raspberry Pi, is there a sample video of trying this code, thank you.

Share picture of driver setup?

Hi,

I am trying to use your code for a T2S+ cam - but i have some issues to install the filter driver / use it, will it come up in device manager?

How does it look when it is correct?

Thanks

What to change for Topdon TC001 camera?

Hey LeoDJ, I'm truly amazed by your code, it's truly brilliant!

I'm currently trying to get the Topon TC001 up and running on Windows, and since it has the same data stream, it should work. However, I'm facing two issues:

1. What do I need to specifically modify for the resolution and FPS?
2. If I install the libusbK driver via Zadig, I can no longer run the default Topdon program as it stops functioning.

I noticed that someone was able to accomplish it using a Raspberry Pi (https://github.com/leswright1977/PyThermalCamera), but unfortunately, it doesn't work at all for Windows. Even with the added knowledge of separating the data stream that you provided, I haven't been successful in getting it to run.
I know this doesn't have much to do with your code and your camera, but I would greatly appreciate your expert opinion.

Best regards!

P2 Support ?

Hi,

I wonder the code can be used with the P2. I happen to have one.

Thank you.

Question: Are you able to see Serial Number, Chip ID or something unique?

I see you have a get device info call that looks like it returns the following below:
Can you tell me if you have been able to get any values for the SN "Serial Number?". Got a couple of these on the way and wondering if I can uniquely identify them.
class DeviceInfoType(enum.IntEnum):
DEV_INFO_CHIP_ID = 0
DEV_INFO_FW_COMPILE_DATE = 1
DEV_INFO_DEV_QUALIFICATION = 2
DEV_INFO_IR_INFO = 3
DEV_INFO_PROJECT_INFO = 4
DEV_INFO_FW_BUILD_VERSION_INFO = 5
DEV_INFO_GET_PN = 6
DEV_INFO_GET_SN = 7
DEV_INFO_GET_SENSOR_ID = 8

No module kivy!

i download zip to my laptop. convert with py exe to executable windows app and i got error wher run application.
Traceback (most recent call last): File "gui.py", line 3, in <module> from kivy.app import App ModuleNotFoundError: No module named 'kivy'

or is normal human windows version? For now is tottaly shit for me...sory.

linux

hi @LeoDJ,

great project! could you add some tutorial how to run it with a raspberry pi 2 w?

is there another open source api or driver to graph images or live video from the InfiRay P2 Pro?

is your project able to live grab a photo or video from the InfiRay P2 Pro?

whats the diff between the InfiRay P2 Pro a InfiRay Tiny1-B and a InfiRay Tiny1-C camera?
how much is the InfiRay Tiny1-B/Tiny1-C and where can you order it?
Can the InfiRay Tiny1-B/Tiny1-C can display the same images (RGB, etc.) as the InfiRay P2 Pro?

@TilCreator i see you use linux too - could you give some feedback?
@julled would also like to get your feedback on the SDK - could you give some feedback? is there an official SDK for the InfiRay P2 Pro?
Edit: InfriRay Tiny1-C SDK seems to be here: https://github.com/qw88888/rv1126

@LeoDJ did you look at https://github.com/yomboprime/yombir ? seems it works for both cameras?

Thanks

Xinfrared T2 Pro

RAW Data

Hi @LeoDJ ,
i really appreciate your work on getting this sensor interfaced!
Do you know if its possible to get the raw data from the sensor ? So instead of receiving RGB or any other HOTCOLD encoded images?
Thank you!

Possibly remove dependence on Zadig installer on Windows

Hi, I just found your project, and it looks pretty cool so I thought I'd give a couple of suggestions:

• It looks like there is a python package that contains compiled libusb binaries for major operating systems, and I have tested it to work out of the box on windows 10 with no separate installation needed. One caveat is that it seemed to require pyusb to be installed before it would enumerate all my usb devices, but it looks like you are using that already so likely not a problem.
• I would highly recommend using some sort of virtual environment system for python, as this really helps with dependency management and version control, for making sure each user that installs your software has the same intended behavior. My preferred choice is poetry, but there are several options, none of which I could confidently say is the best.
• Here is a custom enum definition that encapsulates the length field within DeviceInfoType:

class DeviceInfoType(enum.IntEnum):
    def __new__(cls, value: int, length: int):
        obj = int.__new__(cls, value)
        obj._value_ = value
        return obj
    
    def __init__(self, value: int, length: int):
        self.length = length
    
    DEV_INFO_CHIP_ID = (0, 8)
    DEV_INFO_FW_COMPILE_DATE = (1, 8)
    DEV_INFO_DEV_QUALIFICATION = (2, 8)
    DEV_INFO_IR_INFO = (3, 26)
    DEV_INFO_PROJECT_INFO = (4, 4)
    DEV_INFO_FW_BUILD_VERSION_INFO = (5, 50)
    DEV_INFO_GET_PN = (6, 48)
    DEV_INFO_GET_SN = (7, 16)
    DEV_INFO_GET_SENSOR_ID = (8, 4)

You can still use the enum member as before to retrieve the index as an int, but now if you call DeviceInfoType.DEV_INFO_GET_PN.length, it will retrieve the length instead.

Hope this helps!