Coder Social home page Coder Social logo

senseshift / senseshift-firmware Goto Github PK

View Code? Open in Web Editor NEW
94.0 94.0 10.0 1.11 MB

Open-source firmware for VR accessories. Build your own DIY haptic vest, gloves, and more!

Home Page: https://senseshift.github.io/

License: GNU General Public License v3.0

C++ 98.43% CMake 0.11% C 0.48% Python 0.28% Shell 0.11% Dockerfile 0.59%
arduino bhaptics diy esp32 firmware hacktoberfest haptic-feedback haptics virtual-reality vrglove wearable-devices

senseshift-firmware's People

Contributors

dependabot[bot] avatar leon0399 avatar sanjay900 avatar sweep-ai[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

senseshift-firmware's Issues

[Bug]: esp32 DevkitC v4 Looses Connection

For some reason, the Windows Client is unable to connect to the ESP, as soon as it is found and an attempt to connect is made it loses the connection to the board over BT.
this is the error created by the player:

2022-11-28 15:47:40,361 [1] ERROR BhapticsPlayer.Internal.BhapticsLiteDB: BhapticsDB DB created
2022-11-28 15:47:40,438 [1] ERROR BhapticsPlayer.Utils.Utilities: Util Get BhapticsSettingValue 1
2022-11-28 15:47:40,439 [1] ERROR BhapticsPlayer.Utils.Utilities: CurrentCulture: de-AT
2022-11-28 15:47:40,439 [1] ERROR BhapticsPlayer.Utils.Utilities: Properties.Settings.Default.language Before: en-US
2022-11-28 15:47:40,439 [1] ERROR BhapticsPlayer.Utils.Utilities: Properties.Settings.Default.language: en-US
2022-11-28 15:47:40,731 [1] WARN  BhapticsPlayer.Internal.SettingsStore: CheckDfuFailure
2022-11-28 15:47:40,840 [1] ERROR BhapticsPlayer.Utils.AudioToHapticConfig: Load default setting
2022-11-28 15:47:41,014 [1] ERROR BhapticsPlayer.Audio.Utils.AudioDeviceWatcher: AudioDeviceWatcher() default selected. Die Sequenz enthält kein übereinstimmendes Element.
2022-11-28 15:47:41,131 [1] ERROR BhapticsPlayer.WebSocket.StudioReceiver: Listening on port 15881, and providing WebSocket services:
2022-11-28 15:47:41,145 [1] ERROR BhapticsPlayer.WebSocket.StudioReceiver: Listening on port 15882, and providing WebSocket Secure services: True
2022-11-28 15:47:41,145 [1] WARN  BhapticsPlayer.Utils.Utilities: Release Id : 2009
2022-11-28 15:47:41,145 [1] WARN  BhapticsPlayer.Utils.Utilities: releaseNum : 2009
2022-11-28 15:47:41,655 [1] WARN  BhapticsPlayer.Internal.SettingsStore: CheckDfuFailure
2022-11-28 15:47:41,745 [5] ERROR BhapticsModLauncher.Config.BhapticsContentsRepository: HttpWebResponse status: OK
2022-11-28 15:47:42,448 [5] ERROR BhapticsModLauncher.Config.BhapticsContentsRepository: BhapticsContentsRepository Exists DB
2022-11-28 15:47:42,486 [5] ERROR BhapticsModLauncher.Config.BhapticsContentsRepository: BhapticsContentsRepository SaveGameContents()
2022-11-28 15:47:52,765 [6] WARN  BhapticsPlayer.Bluetooth.DeviceWatcherService: Remove Device TactosyDevice {Name=TactSuitX40, Id=70518412007866, Paired=False, Connected=Disconnected}
2022-11-28 15:48:00,282 [30] WARN  BhapticsPlayer.Bluetooth.DeviceWatcherService: Remove Device TactosyDevice {Name=TactSuitX40, Id=70518412007866, Paired=False, Connected=Disconnected}

Serial output does not produce anything bt this:

>>      onConnect()
>>      onConnect(*param)
>>      onDisconnect()

[Request]: OpenGloves compatablity

Implementation PR

  • No FFB, No Splay (#60)
  • FreeRTOS task (#64)
  • Bluetooth Serial Communication (#69)
  • Add Splay
  • Add FFB
  • Put in task
  • Unified firmware for TactGlove + OpenGloves compatibility

Reference Issues

Will require merging #39, thus closing #29, #31 and #36

Summary

  • Add support for OpenGloves protocol (USB Serial and Bluetooth Serial)
  • Add support for servo outputs (already done? the same output writers)
  • Add inputs for finger tracking (using ISensor? use Battery as a reference for throttled update rate)
  • Add buttons (with interrupts? or looped?)

Basic Example

There are references:

Drawbacks

Creating a proper config file for such a configuration will be hard. Possible options are:

  • TactGlove + OpenGlove (Serial + flexion + splay + FFB)
  • TactGlove + OpenGlove (Serial + flexion + FFB)
  • TactGlove + OpenGlove (Serial + flexion + splay)
  • TactGlove + OpenGlove (Serial + flexion)

Multiply that with BT Serial instead of Serial, and if someone wants to use it without Haptic Feedback, and configuration becomes quite hard

Unresolved questions

No response

[Bug]: Connection errors for Quest 3

What happened?

I did install the last version 0.7.0-rc1 and work perfectly in IOS and Android mobiles phone. But in quest, are detected correctly in bluetooth but into the Haptic player don't... and don't work in games.

I have a screen shot that I report the Version of the Bhaptic player and the version of Quest are 66.
Is detected by the BHaptic, can see into the list, but can't connect and use it.

Greetings.

Target Device

x16 haptic

Hardware Configuration

ESP32-WROOM-32

Firmware Version

0.7.0 RC1

What OS or environment are you seeing the problem on?

Meta Quest

bHaptics Player Version

1.2.3

Relevant log output

Quest 3 version 66.

Serial output

NO serial output

Contact Details

[email protected]

Code of Conduct

  • I agree to follow this project's Code of Conduct

Install guide

We really need one, I only found how I need to build it out from analyzing the files and trial and error (I guess because I use other software versions). But I get it to build the code.

What you need (for others):

  1. Vistual Studio Code
  2. Inside VSC the Extensions PlatformIO IDE, Espressif IDF and maybe Arduino(?), at the last not sure, I simply installed it.
  3. Clone the Git Code from here to your drive and open it with PlatformIO, on their website is a tutorial how.

I get an error "src_filter is missing inside [Common] in platformio.ini" (something like that). I copied build_src_filter inserted it before and remove "build_", it worked.

But when I want to upload it, it load all files from all bHaptics product one after another up, so you need to make a backup of ini/bhaptics.ini and delete all except the one you want to build and upload.

And after this... holly cow, it works perfect in the bHaptics Feedback Test! Couldn't find an issue here. Now I can start to build my vest in the next weeks. I tested it with the Tactosi for feets, because it only has 3 rumble motors, easier to test. All 3 ports worked as they should inclusive the intensity.

[WIP] [RFC]: Firmware Architecture

C4Component
  title OpenHaptics Firmware Architecture Component Diagram

  Boundary(games, "Games", "boundary") {
    System_Ext(game_1, "Game 1")
    Rel(game_1, hub, "Uses", "WS")
  
    System_Ext(game_2, "Game 2")
    Rel(game_2, hub, "Uses", "WS [bHaptics]")
  
    System_Ext(game_3, "Game 3")
    Rel(game_3, hub, "Uses", "OSC")
  }

  Container(hub, "OpenHaptics Hub", "PC, Quest")

  Container_Boundary(firmware, "Firmware") {
    Component(connection, "Connection", "BLE Connection", "Allows wearables to be connected to VR devices.")
    Rel(connection, output, "Updates output")
    Rel(connection, indicator, "Displays connection status")

    Component(output, "Output", "Closest Output", "Writes output.")
    Rel(output, output_component_pca, "Write output", "CHEST_FRONT")
    Rel(output, output_component_ledc, "Write output", "PALM_RIGHT_FRONT")

    Component(output_component_pca, "OutputComponent", "PCA9685", "Writes output.")
    Component(output_component_ledc, "OutputComponent", "ledc", "Writes output.")

    Component(battery, "Battery", "ADC", "Handles battery charge")

    Component(indicator, "Indicator", "WS2812", "Visually display device status to user")
    Rel_Back(indicator, battery, "Queries and displays charge status")

    ComponentQueue(other_components, "Other Components")
  }

  Rel(hub, connection, "Connects", "BLE")
  Rel_Back(hub, connection, "Reports configuration", "BLE")
Loading

[Request]: refactor to use internal `lib` directory

Implementation PR

No response

Reference Issues

Will require #29 first

Summary

Reorganize project to use lib directory as described herethe

Basic Example

- lib
  - Core
    - Component
    - Output
  - OutputLedc
    - OutputLedc
  - OutputPCA9685
    - OutputPCA9685
  - ...

Drawbacks

Will require to create it's own internal library for each major firmware part

Unresolved questions

  1. What is our desired structure?
  2. What components to extract?

[WIP] [RFC]: OpenHaptics protocol

Here I am just writing ideas from my mind to future implement in our own protocol, when current protocol from bHaptics when it ceases to meet the needs

OpenHaptics Protocol

Coordinates encoding

Each axis (X and Y) is uint8_t between 0 and 255

Coordinates preview
(0, 0)                                 (128, 0)                                 (255, 0)









(0, 128)                               (128, 128)                               (255, 128)









(0, 255)                               (128, 255)                               (255, 255)

Intensity

Intensity is uint8_t:

0: 0%, 255: 100%

Effects layer

Effect layer is 16 possible values (4 bits - uint8_t << 4)

Proposed Effects layer IDs (WIP)
  • Vibro: 0x0
  • Touch (LRA): 0x1
  • Temperature (Peltier): 0x2
  • Electricity (TENS): 0x3

Body Parts IDs mapping

Every diverging part of the body is separated into Front and Back faces.
Coordinate path has 255 possible values (8 bits - uint8_t)

Proposed Body Parts IDs (WIP)
  • human://chest:
    • human://chest/front: 0x00
    • human://chest/back: 0x01
  • human://arm:
    • human://arm/left: 0x02
    • human://arm/right: 0x03
  • human://forearm:
    • human://forearm/left: 0x04
    • human://forearm/right: 0x05
  • human://hand:
    • human://hand/left:
      • human://hand/left/palm:
        • human://hand/left/palm/front: 0x06
        • human://hand/left/palm/back: 0x07
      • human://hand/left/finger:
        • human://hand/left/finger/thumb: 0x08
        • human://hand/left/finger/index: 0x09
        • human://hand/left/finger/middle: 0x0a
        • human://hand/left/finger/ring: 0x0b
        • human://hand/left/finger/pinky: 0x0c
    • human://hand/right:
      • human://hand/right/palm:
        • human://hand/right/palm/front: 0x0d
        • human://hand/right/palm/back: 0x0e
      • human://hand/right/finger:
        • human://hand/right/finger/thumb: 0x0f
        • human://hand/right/finger/index: 0x10
        • human://hand/right/finger/middle: 0x11
        • human://hand/right/finger/ring: 0x12
        • human://hand/right/finger/pinky: 0x13
  • human://upper_leg:
    • human://upper_leg/left: 0x14
    • human://upper_leg/right: 0x15
  • human://lower_leg:
    • human://lower_leg/left: 0x16
    • human://lower_leg/right: 0x17
  • human://foot:
    • human://foot/left:
      • human://foot/left/top: 0x18
      • human://foot/left/bottom: 0x19
    • human://foot/right:
      • human://foot/right/top: 0x1a
      • human://foot/right/bottom: 0x1b
  • human://neck: 0x1c
  • human://head:
    • human://head/face: 0x1d
    • human://head/back: 0x1e
    • human://head/top: 0x1f

[Request]: ForceTube Protocol

Implementation PR

No response

Reference Issues

Required for #50

Summary

Create a communication, that allows ForceTube SDK to connect
It is 100% possible, since ForceTube itself is originally built on ESP32

Basic Example

BluetoothSerial(“ForceTube xxxxxxxxx”)

Drawbacks

  • It is one more integration in 3rd-party SDK, future usage will not allow interconnection
  • Incompatible with #44, since both use BluetoothSerial
  • Probably incompatible with BLE stack

Unresolved questions

No response

[RFC] `uint8_t` coordinates for outputs

Using current uint16_t for coordinates is too excessive.

We can save some memory and future transmission bytes by reducing every axis (X and Y) and output intensity to uint8_t, giving us $2^{8}=256$ values for each variable

This RFC is a part of transition to #9

[Request]: Refactor `auto` and `auto_margins` output

Original comment issue in #24

I decoupled output writes themselves (e.g. activate closest motors to requested output, or interpolate between surrounding motors) and registering motors input, like auto and auto with margins (btw, auto with margins makes more sense IMO)
These output changes will simplify things for part of #6 issue - in-cloud on-demand firmware builder for WebSerial flash

Difference between `auto` and `auto with margins`

* - motor
= - empty space

Auto:

*=========*
*=========*
*=========*

Auto with margins:

===*===*===
===*===*===
===*===*===

But I'm not entirely sold on my solution now, cause:

  1. We need to have 4 different options (Coordinates based on this: #9):
    • both X and Y axes use auto
    • both X and Y axes use auto with margins
    • X uses auto and Y uses auto with margins
    • X uses auto with margins and Y uses auto
    • Something more custom:
      image
  2. For a more DRY approach, the same functions should be used for the make_point functions in all current bHaptics configs

As for now, 2nd requirement is partly achieved, but due to 1st is not met, it is not customizable at all

That is all because my vision for the future is entirely different from the current bHaptics BLE protocol allows:

  • it is limited to 40 output points max
  • because of DIY nature of project, other configurations must be achievable also (e.g. 32 vest, with only PCAs used, without ledc, x8 vest for very lightweight setups, etc)

Originally posted by @leon0399 in #24 (comment)

[Request]: Setup Unit tests

Implementation PR

No response

Reference Issues

Will require #36 first

Summary

Create Unit tests to insure, that all components of the firmware are working as expected

Basic Example

Test closest output, auto output mapper, etc.

Drawbacks

Is TDD a drawback? 😃

Unresolved questions

No response

Flash firmware without PlatformIO

Using PlatformIO and PlatformIO IDE is not as straightforward as using Arduino IDE, so an alternative is required

As proposed, WebFlash API may be used to achieve a more effortless firmware flashing experience. Using newly-created openhaptics.github.io it is possible to create a WebFlash-based page. To achieve this following steps are required:

Original issue

As project is developed using PlatformIO IDE, it is not that user-friendly, especially for entry-level developers, so it will be easier for most of people to configure and flash it using Arduino IDE

Need to investigate how Marlin IDE is made this happen. They also use PlatformIO and flash it with Arduino IDE

Sweep: Replace C-style casts with C++ casts

Implementation PR

No response

Reference Issues

No response

Summary

Replace existing C-style casts with C++ casts across all codebase.

Basic Example

Replace (T)foo with static_cast, reinterpret_cast, const_cast and dynamic_cast

Drawbacks

Ugly code, need to understand how it works first

Unresolved questions

No response

Checklist
  • lib/bhaptics/bh_utils.cpp ⚠️ No Changes Made
  • CMakeLists.txt ✅ Sandbox ran successfully
  • lib/bhaptics/bh_utils.cpp ❌ Sandbox failed so I made additional changes
  • test/test_bhaptics/main.cpp
  • lib/bhaptics_ble/connection_bhble.cpp
  • lib/haptics/haptic_body.cpp
  • lib/haptics/haptic_plane.cpp
  • lib/opengloves/og_alpha_encoding.cpp
  • lib/opengloves/og_ffb.hpp
  • lib/opengloves/og_protocol.hpp
  • lib/opengloves/og_serial_communication.hpp
  • lib/opengloves/opengloves_task.hpp
  • lib/opengloves/sensor/og_finger.hpp
  • lib/opengloves/sensor/og_gesture.hpp
  • lib/opengloves/sensor/og_sensor.hpp
  • lib/output/abstract_actuator.hpp
  • lib/sensor/sensor.hpp
  • CMakeLists.txt ✅ Commit e747c84

Flowchart

[Request]: Config SerialPlotter port via #define

Implementation PR

No response

Reference Issues

No response

Summary

Allow changing SerialPlotter port with #defines, to later use with Configurable firmware builder

Basic Example

Add config for serial plotter component: enable, port, baudrate

Accept HardwareSerial port as input for SerialPlotter, open port in setup() function

Drawbacks

With this implementation submitted port must be reserved for using only with SerialPlotter

Unresolved questions

No response

[Request]: Switch connection between devices

Implementation PR

No response

Reference Issues

No response

Summary

Add an option to change an active connected devices with a push of a button (quest/pcvr)

Basic Example

Logitech MX Master 3

Drawbacks

  1. Will require a button
  2. Is it really necessary?

Unresolved questions

Technical details: how does it work? Does it change MAC? Or smth else?

[Request]: ADC battery component

Implementation PR

#22

Reference Issues

Part of #17

Summary

Most basic Battery level monitor, using raw battery input to ADC pin

Basic Example

https://randomnerdtutorials.com/power-esp32-esp8266-solar-panels-battery-level-monitoring
https://theiotprojects.com/esp8266-monitor-its-own-battery-level-using-iot
https://github.com/SlimeVR/SlimeVR-Tracker-ESP/blob/main/src/batterymonitor.h
https://github.com/SlimeVR/SlimeVR-Tracker-ESP/blob/main/src/batterymonitor.cpp

Drawbacks

Will require soldering some resistors to create voltage divider, since ESP32 runs on 3.3V, but max battery output is around ~4V

Unresolved questions

No response

[RFC]: Battery component support

To increase the UX of firmware with the bHaptics player on devices, powered by an internal battery (not an external power bank), the battery level needs to be reported.

bHaptics uses 6e400008-b5a3-f393-e0a9-e50e24dcca9e Battery Level characteristic to report battery level (0 - 100) in Motor Service (6e400001-b5a3-f393-e0a9-e50e24dcca9e)

Additionally, for better UX on host devices without the need to check bHaptics software and native battery level notifications, this battery level may be reported to default 0x2A19 Battery Level characteristic in 0x180F Battery Service

Supported battery level sensors/gauges:

[Request]: CD74HC4067 Multiplexer Support

Implementation PR

No response

Reference Issues

Required for #65 and partially #44

Summary

Add a CD74HC4067 multiplexer sensor to firmware and allow using it as AnalogSensor
Required for OpenGloves Splay and MultiCurl

Basic Example

Drawbacks

Harder to maintain easy INI configuration

Unresolved questions

No response

[Request]: Auxiliary Haptic Accessories

Implementation PR

#49

Reference Issues

No response

Summary

Provide an option to integrate not only body-replated haptic accessories, such as gunstock for recoil effect

Basic Example

See #49

Drawbacks

Not currently supported by bHaptics' BLE, will require either option for future:

  • use our own API
  • emulate ProTubeVR protocol

Unresolved questions

No response

[Request]: `atan2` sensor

Implementation PR

No response

Reference Issues

No response

Summary

Create a new sensor that utilizes the atan2 function to calculate the angle from two analog inputs: cos and sin

Basic Example

auto* sin_sensor = new AnalogSensor(PIN_SIN);
auto* cos_sensor = new AnalogSensor(PIN_COS);

// (0.0F, 1.0F) => (-1.0F, 1.0F)
auto* sincos_filter = new LambdaFilter<float>([](float value) {
    return value * 2.0F - 1.0F;
});

sin_sensor->addFilter(sincos_filter);
cos_sensor->addFilter(sincos_filter);

atan2_sensor = new Atan2Sensor(sin_sensor, cos_sensor);
/// Sensor that calculates the atan2 of two sensors
template<typename Tp = float>
class Atan2Sensor : public Sensor<Tp> {
    static_assert(std::is_floating_point_v<Tp>, "Tp must be a floating point type");

  public:
    using Source = Sensor<Tp>;

    Atan2Sensor(Source* sin, Source* cos) : sin_(sin), cos_(cos){};

    void init() override
    {
        SS_SUBSENSOR_INIT(this->sin_, false, [this](float /*value*/) {
            this->recalculateState();
        });

        SS_SUBSENSOR_INIT(this->cos_, false, [this](float /*value*/) {
            this->recalculateState();
        });
    }

    inline void tick() override { this->recalculateState(); }

    void recalculateState()
    {
        const Tp sin = this->sin_->getValue();
        const Tp cos = this->cos_->getValue();

        if (sin == 0.0F && cos == 0.0F) {
            this->publishState(0.0F);
            return;
        }

        const Tp radians = std::atan2(sin, cos);

        this->publishState(radians);
    }

  private:
    Source* sin_;
    Source* cos_;
};

More examples:

  1. https://github.com/LucidVR/lucidgloves/blob/proto5/firmware/lucidgloves-firmware/input.ino#L315

Drawbacks

  1. The approach above requires applying a custom filter to the sensor.

Unresolved questions

No response

[Request]: IMU support with SlimeVR

Implementation PR

No response

Reference Issues

No response

Summary

Develop a SlimeVR-compatible layer

Basic Example

Take a look at: https://github.com/SlimeVR/SlimeVR-Tracker-ESP

struct IMU_State {
  Vector3 gyroscope, accelerometer, magnetometer;
};

class BNO055 : public ISensor<IMU_State> {
  ...
};

Drawbacks

  • A separate task for updates
  • Will only work with WiFi
  • Will require either recompile for each WiFi AP, or developing a provisioning module

Unresolved questions

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.