This is a project created by QINGHAO LIU. The basic idea is that:
The program of the system has two parts: Hardware(Embedded Board) and Software(PC).
This README Only talks about Software part.
The software part is based on Python. If you are not familiar with it, please learn about it by yourself(REALLY EASY).
This repository contains many files, which are almost independent on each other.
- RealTimeShow.py (Real-Time show the data of 10 magnetometers and 1 motion sensor as their physical postion)
- RealTimeShow_Relative.py (Choose a pivot. Show the relative value of each magnetometer to the pivot)
- Magnets_Permutation.py (Collect data when magnet is put on different area on your wrist)
- plot_data.py (Use the TXT file output by Magnets_Permutation.py and visualize the sensitive area for each finger)
- MultiFinger_Robust_Rotation.py (Collect Calibration&Single-Finger&Multi-Finger data)
- PCA_Display.py (Visualize the data on 2-D space after PCA)
- PCA_Display_Lib.py (The library of PCA_Display.py)
No matter which OS(MacOS,Win,Unix,Linux) you are using, you are strongly recommended to run the program in Terminal.
-
Install Python3(There may be some problems with Python2, so please kindly use Python3). You can download it here. Confirm you have installed python3 successfully.
- MacOS: Open Terminal. Type
python3
. If you can enter the shell, congratulations! - Win/Linux: Open Terminal. Type
python
. If you can enter the shell, congratulations!
- MacOS: Open Terminal. Type
-
Install Pip. Maybe the installation program above has installed pip for you. Confirm it:
- MacOS: Open Terminal. Type
pip3 -V
. If the version number shows up, congratulations! - Win/Linux: Open Terminal. Type
pip -V
. If the version number shows up, congratulations! Otherwise, Google how to install pip for your PC.
- MacOS: Open Terminal. Type
-
Install packages needed by pip.
(For MacOS, replace pip
with pip3
)
pip install numpy
pip install matplotlib
pip install pyserial
pip install serial
pip install sklearn
Open Terminal. Type the code as below(For MasOS, replace python
with python3
):
python -m serial.tools.list_ports
If no problem, you can see a list of serial port names of your PC.
Then, connect the embedded board to your PC by a microUSB-USB or microUSB-USB(Type-C) wire.
Execute the commmand above again. Please find out the new added port name.
- For MacOS: The port name format looks like this /dev/cu.usbmodem0F00577F
- For Win: The port name format looks like this COM1
/* HMC5983 Part */
X_Axis_Data Y_Axis_Data Z_Axis_Data 0
X_Axis_Data Y_Axis_Data Z_Axis_Data 1
X_Axis_Data Y_Axis_Data Z_Axis_Data 2
X_Axis_Data Y_Axis_Data Z_Axis_Data 3
...
X_Axis_Data Y_Axis_Data Z_Axis_Data 9
/* MPU9250 Part */
Acc_X Acc_Y Acc_Z Gyr_X Gyr_Y Gyr_Z 0
That means you have to read 11 lines to get all sensors' data.
For the first 10 lines, which are data of magnetometers, the number in the end of the line is the number of the magnetometer shown here.
For the last line, which is data of motion sensor, the number in the end of the line is just the number of I2C master. No need to cara about it.
This is a Real-time program that can plot the data of 10 magnetometers and 1 motion sensor vividly!
To run the program, please follow the steps below:
- Modify port name. Open RealTimeShow.py with the editor you prefer. Find the code below:
ser = serial.Serial('/dev/cu.usbmodem0F00577F',1500000)
Replace the /dev/cu.usbmodem0F00577F
with the port name of your device. For example:
ser = serial.Serial('COM3',1500000)
The second parameter 1500000
is clock frequency of UART. Please keep it the same with the hardware part. If you have no extra requirement, you do not need to change it.
- Open Teriminal and locate to the repository. Execute the command below:
(For MacOS, replace python
with python3
)
python RealTimeShow.py
- If you can see a dynamic figure like this, congratulations!(Please try moving or rotating the magnetometer array to observe the curve)
Explanation:
- There are 11 windows, 10 around for magnetometers, 1 in middle for motion sensors.
- In each magetometer window, there are 3 lines, Blue for X-axis data, Orange for Y-axis data, Green for Z-axis data.
- In motion-sensor window, there would be 6 lines, 3 for accelermeter, 3 for gyroscope(Not shown on the picture).
- The range of vertical axis will change automatically with the max and min value of the data so that you can watch it elegantly.
- The range of horizontal axis will change periodically to subsitute new data for old data.
Possible problem:
- All three lines of a magnetometer may keep constant(0/+-1024/+-2048)(See Right-Up side of the picture). Solution: Check the connection between embedded board and magnetometer array.
- Some lines or data may be unstable and change dramatically. Solution: The same as above.
- All lines become frozen suddenly. Solution: Close the figure, re-connect embedded board, and try again.
- Only one of the line of a magnetometer keeps constant. That's the worst case. Probably your sensor chip has some problem. Unfortunately, there is no good solution until now but changing chip.
If you run this program, you can get a dynamic figure that will show you the relative value of 9 magnetometers to the other one(reference magnetometer), which means the substraction of each axis of each magenetomer to the corresponding axis of the reference magnetometer. We call the reference magnetometer pivot.
To run the program, please follow the steps below:
-
Modify port name. Refer to the title before
-
Modify pivot number as you want. In the editor, search
pivot =
and you will get a line of code likepivot = 3
. The number means which magnetometer you choose as a reference one. You can refer to the image for the map of sensors. -
Open Teriminal and locate to the repository. Execute the command below:
(For MacOS, replace python
with python3
)
python RealTimeShow_Relative.py
This part we use two programs:
- Magnets_Permutation.py
- plot_data.py
Please follow the instructions below:
- Decide how to grid your wrist, in another word, how many rows, how many columns. e.g. row:4 col:9.
- Decide how many vectors you want to sample for each finger(Must be a number that can be divided by 5). e.g. 10.
- Open Magnets_Permutation.py and skip the function definition part and you will find three assignment.
batch = 5
row_num = 4
col_num = 9
The batch
is how many vectors you want to sample for each finger. Modify it as you want but It must be a number that can be divided by 5.
Modify the row_num
and the col_num
too.
-
Modify port name. Refer to the title before
-
Choose one of your hand. e.g. left. Direct the inner side of your wrist to your face. Draw the grid on you wrist as below.
8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 | 0 | . |
---|---|---|---|---|---|---|---|---|---|
-- | -- | -- | -- | -- | -- | -- | -- | -- | 0 |
-- | -- | -- | -- | -- | -- | -- | -- | -- | 1 |
-- | -- | -- | -- | -- | -- | -- | -- | -- | 2 |
-- | -- | -- | -- | -- | -- | -- | -- | -- | 3 |
Yes! The column number is from right to left(Very strange but that`s it).
- Put a magnet on postion 00(row 0 col 0). And run the program as:
(For MacOS, replace python
with python3
)
python Magnets_Permutation.py lqh
Yes! There is a main argument. You can modify it as your name or whatever you want but you have to know that the argument you input will be a part of the name of the output file.
- Follow the instructions in the program. The sequence of magnet position is 01, 02, 03,..., 08, 10, 11,..., 18,..., 38.
- If you follow the instructions in the program correctly, then you will get a txt file named like finger_vectors_file_xxx.txt in the folder.
Please follow the instructions below:
- Modify the input file name(The file you got just now). Open plot_data.py and search
f = open
. You will find a line of code:
f = open('finger_vectors_file_yifeng.txt','r')
Change the first parameter to the file name you get finger_vectors_file_xxx.txt
.
- Modify the configuration of output figure.Search
fig = draw_power_for_each_finger
and you will find a line of code:
fig = draw_power_for_each_finger(single,interpol='bilinear',norm='off')
You can change the interpolation to None as this:
fig = draw_power_for_each_finger(single,interpol=None,norm='off')
Or you can use other interpolation as this:
fig = draw_power_for_each_finger(single,interpol='nearest',norm='off')
-
If you want to use Normalization(which means that the 5 subplot for 5 fingers will use the same color-bar range), modify the
norm
parameter to'on'
. Then, searchnorm_min =
and you will find two assignment. Modify the Max and Min for the normalization. -
Run the program.
(For MacOS, replace python
with python3
)
python plot_data.py
Huh, maybe you have determined magnet-deployment. Then, fix Mag-Array and magnets(on wristband) on your wrist and ready to collect data. You can use either MultiFinger_Robust_Rotation.py or MutiFinger_Backup.py to log data.
- The program will ignore the data of motion sensor. If you have any idea to take advantage of it, please modify the program or write a new program by yourself.
- We define a vector as 30 axes of all magnetometers. Look like this
[sensor0_X, sensor0_Y, sensor0_Z, sensor1_X, sensor1_Y, sensor1_Z, ... , sensor9_X, sensor9_Y, sensor9_Z]
- Of course, a vector has a length of 30.
- We define a null vector as the vector you collect when you bend no finger.
- We define a finger vector as the vector you collect when you bend one or more finger(s).
- We define a batch as how many vectors you collect for each gesture or finger.
- We define K as how many gestures you want to recognize. Default: 32(All combinations of your fingers).
- The data collection contains three main parts.Calibration, Single-Finger, Multi-Finger.
- Calibration: 2-D Array with size N x 30. N depends on the time you spend on collecting.
- Single-Finger: Collect batch/5 null vectors, batch index finger vectors, batch/5 null vectors,batch middle finger vectors, batch/5 null vectors, batch ring finger vectors, batch/5 null vectors, batch little finger vectors, batch/5 null vectors, batch thumb vectors.
Output format: 6 x batch x 30
- Multi-Finger:
Output format: K x batch/5 and K x batch x 30.
Please follow the instructions below:
-
Open MultiFinger_Robust_Rotation.py with editor. Skip all definitions of functions and you will find two assignments. One of them is batch. Change it as you want.
-
Modify port name. Refer to the title before
-
Open Terminal and locate to your folder. Run the command like below:
(For MacOS, replace python
with python3
)
python MultiFinger_Robust_Rotation.py lqh 1
- Parameter 1: Volunteer name. You can use your name's abbreviation like
lqh
. - Parameter 2: Position name. This is used to log your pose. You can use
pos1 pos2 pos3
or1 2 3
and so on.
- Find 4 output files. To use this files, you can refer to data format.
- lqh_1_calibration.npy
- lqh_1_single.npy
- lqh_1_null_multiple.npy
- lqh_1_multiple.npy
(Not Ready)