Coder Social home page Coder Social logo

takeyamayuki / nonmouse Goto Github PK

View Code? Open in Web Editor NEW
86.0 2.0 10.0 104.56 MB

a webcam-based virtual gesture mouse that is easy to use with hands on the desk.

Home Page: https://zenn.dev/ninzin/articles/94b05fdb9edf53

License: Apache License 2.0

Python 100.00%
python mediapipe mouse pynput opencv hands hci gesture gesture-control gestures

nonmouse's Issues

ZeroDivisionErrorでクラッシュする

cfps = int(cap.get(cv2.CAP_PROP_FPS))
でcfpsが1になって
if cfps < 30: cap.set(cv2.CAP_PROP_FRAME_WIDTH, cap_width) cap.set(cv2.CAP_PROP_FRAME_HEIGHT, cap_height) cfps = int(cap.get(cv2.CAP_PROP_FPS))
を通してもcfpsが1になり、
ran = int(cfps/10)
でranが0になってしまい、ZeroDivisionErrorでクラッシュします。

環境は以下の通りです
PC: MacBook Pro 2020 Intel
OS: macOS Monterey
カメラ: Mac内蔵Webカメラ
Pythonバージョン: 3.9.7

need refactor

  • Low code quality
  • No environment-dependent resolution -> PyPI support
  • Need appropriate build environments (pyinstaller usage)
  • Need CI with github actions

How to set screen size?

Hello the mouse only works on left two thirds of the screen. I gues it has something to do with screen size, my lapto has 1920x1080. Can I reconfigure it somewhere?

pip install cannot find mediapipe==0.810.1

When I run pip install nonmouse as mentioned in the readme, I get the following error: Perhaps the mediapipe version can be updated?

ERROR: Could not find a version that satisfies the requirement mediapipe==0.8.10.1 (from nonmouse) (from versions: 0.9.0.1, 0.9.1.0, 0.9.2.1, 0.9.3.0, 0.10.0, 0.10.1, 0.10.2, 0.10.3, 0.10.5, 0.10.7, 0.10.8)
ERROR: No matching distribution found for mediapipe==0.8.10.1

Thank you for this package.

tasks

  • スクロールの機能変更(手を回転)
  • 手首基準座標軸追加
  • app-official.pyにargs追加
  • GUI化
  • スクロールの機能変更(親指以外全部くっつける)

tasks2

  • タッチディスプレイモード追加
  • アプリケーション化

USB cameras not working?

Sorry for not posting a lot of debug info here, but I am trying to test this and only can get it working for my mounted webcam. Anytime I try to use a usb web cam the device does not appear to work properly for the program.

I will try to pull down the project soon and see if I can get some more info as to what or why they are not working.

error: PyObjC requires macOS to build

Hello. When installing requirements on Linux I get this error:

Collecting pyobjc-core==7.3
  Downloading pyobjc-core-7.3.tar.gz (684 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 684.2/684.2 KB 5.4 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      running egg_info
      error: PyObjC requires macOS to build
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

MacOS MontereyでカメラFPSが5Hz程度になる

NonMouse/app.py

Lines 117 to 121 in 1e7f7bc

cfps = int(cap.get(cv2.CAP_PROP_FPS))
if cfps < 30:
cap.set(cv2.CAP_PROP_FRAME_WIDTH, cap_width)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, cap_height)
cfps = int(cap.get(cv2.CAP_PROP_FPS))

cfps = int(cap.get(cv2.CAP_PROP_FPS))で取得するカメラのFPSが5Hzになってしまい、カーソルが小刻みに揺れてしまう。
これは恐らく、cv2の問題だと思われる。

[bug report] Sometimes the orientation of the hand does not match the orientation of the image

NonMouse/app.py

Lines 115 to 155 in 4603acc

# 手の向きでモードを変える
nowLandX = hand_landmarks.landmark[4].x - \
hand_landmarks.landmark[20].x
nowLandY = hand_landmarks.landmark[12].y - \
hand_landmarks.landmark[0].y
if m == 0:
preLandX = nowLandX
preLandY = nowLandY
m = 1
if nowLandX > 0:
mode = 2 # 左右反転
if nowLandY > 0:
mode = 1 # 上下反転
if nowLandY < 0 and nowLandX < 0:
mode = 0
if mode == 0:
if nowLandX > 0 and np.sign(nowLandX) != np.sign(preLandX):
mode = 2 # 左右反転
if nowLandY > 0 and np.sign(nowLandY) != np.sign(preLandY):
mode = 1 # 上下反転
if nowLandY < 0 and nowLandX < 0 and \
(np.sign(nowLandY) != np.sign(preLandY) or np.sign(nowLandX) != np.sign(preLandX)):
mode = 0
elif mode == 1:
if nowLandX > 0 and np.sign(nowLandX) != np.sign(preLandX):
mode = 2 # 左右反転
if nowLandY < 0 and np.sign(nowLandY) != np.sign(preLandY):
mode = 1 # 上下反転
if nowLandY > 0 and nowLandX < 0 and \
(np.sign(nowLandY) != np.sign(preLandY) or np.sign(nowLandX) != np.sign(preLandX)):
mode = 0
elif mode == 2:
if nowLandX < 0 and np.sign(nowLandX) != np.sign(preLandX):
mode = 2 # 左右反転
if nowLandY > 0 and np.sign(nowLandY) != np.sign(preLandY):
mode = 1 # 上下反転
if nowLandY < 0 and nowLandX > 0 and \
(np.sign(nowLandY) != np.sign(preLandY) or np.sign(nowLandX) != np.sign(preLandX)):
mode = 0
preLandX = nowLandX
preLandY = nowLandY

カメラや手が傾くと、自分の指先移動量に対するマウス移動量が異なってしまう

たとえば、カメラを斜めに取り付けると、カメラに映る指は縦方向に圧縮されたものになり、当然移動量もその分小さくなる。
現在の自分の環境だと手に向けてまっすぐ取り付けられているので問題はないが、初めて使う人にとってはこのアプリケーションを使う上での大きな壁となる。

「解決策」
まず、基準となる指の関節座標を用意しておく。
その基準となる座標群と現在の座標群を比較して手の傾き具合を求める。
それによって、dx,dy変える

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.