Coder Social home page Coder Social logo

schollz / howmanypeoplearearound Goto Github PK

View Code? Open in Web Editor NEW
6.9K 168.0 385.0 1.73 MB

Count the number of people around you :family_man_man_boy: by monitoring wifi signals :satellite:

License: MIT License

Python 99.99% Dockerfile 0.01%
tshark wifi location sensor

howmanypeoplearearound's Introduction

howmanypeoplearearound

Count the number of people around you ๐Ÿ‘จโ€๐Ÿ‘จโ€๐Ÿ‘ฆ by monitoring wifi signals ๐Ÿ“ก.

howmanypeoplearearound calculates the number of people in the vicinity using the approximate number of smartphones as a proxy (since ~70% of people have smartphones nowadays). A cellphone is determined to be in proximity to the computer based on sniffing WiFi probe requests. Possible uses of howmanypeoplearearound include: monitoring foot traffic in your house with Raspberry Pis, seeing if your roommates are home, etc.

Tested on Linux (Raspbian and Ubuntu) and Mac OS X.

It may be illegal to monitor networks for MAC addresses, especially on networks that you do not own. Please check your country's laws (for US Section 18 U.S. Code ยง 2511) - discussion.

Getting started

For a video walkthrough on how to install, checkout PNPtutorials.

Dependencies

Python 2.7 or preferably Python 3 must be installed on your machine with the pip command also available.

  python -V
  pip -V

WiFi adapter that supports monitor mode

There are a number of possible USB WiFi adapters that support monitor mode. Here's a list that are popular:

Namely you want to find a USB adapter with one of the following chipsets: Atheros AR9271, Ralink RT3070, Ralink RT3572, or Ralink RT5572.

Mac OS X

  brew install wireshark
  brew cask install wireshark-chmodbpf

You need to dissociate from any AP before initiating the scanning:

sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -z

Linux tshark

sudo apt-get install tshark

Then update it so it can be run as non-root:

sudo dpkg-reconfigure wireshark-common     (select YES)
sudo usermod -a -G wireshark ${USER:-root}
newgrp wireshark

Install

pip install howmanypeoplearearound

Run

Quickstart

To run, simply type in

$ howmanypeoplearearound
Using wlan1 adapter and scanning for 60 seconds...
[==================================================] 100%        0s left
There are about 3 people around.

You will be prompted for the WiFi adapter to use for scanning. Make sure to use an adapter that supports "monitor" mode.

Docker alternative

If Docker is installed locally and you want to take howmanypeoplearearound out for a quick spin, you can try the following:

  1. Copy Dockerfile from this repo in your current working directory
  2. docker build -t howmanypeoplearearound . # that . at the end is important
  3. docker run -it --net=host --name howmanypeoplearearound howmanypeoplearearound

NOTE: This Docker alternative is known to work on Ubuntu but not on Mac OS X. Feedback on other platforms would be appreciated.

Options

You can modify the scan time, designate the adapter, or modify the output using some command-line options.

$ howmanypeoplearearound --help

Options:
  -a, --adapter TEXT   adapter to use
  -z, --analyze TEXT   analyze file
  -s, --scantime TEXT  time in seconds to scan
  -o, --out TEXT       output cellphone data to file
  -v, --verbose        verbose mode
  --number             just print the number
  -j, --jsonprint      print JSON of cellphone data
  -n, --nearby         only quantify signals that are nearby (rssi > -70)
  --nocorrection       do not apply correction
  --loop               loop forever
  --sort               sort cellphone data by distance (rssi)

Print JSON

You can generate an JSON-formatted output to see what kind of phones are around:

$ howmanypeoplearearound -o test.json -a wlan1
[==================================================] 100%         0s left
There are about 4 people around.
$ cat test.json | python3 -m json.tool
[
  {
    "rssi": -86.0,
    "mac": "90:e7:c4:xx:xx:xx",
    "company": "HTC Corporation"
  },
  {
    "rssi": -84.0,
    "mac": "80:e6:50:xx:xx:xx",
    "company": "Apple, Inc."
  },
  {
    "rssi": -49.0,
    "mac": "ac:37:43:xx:xx:xx",
    "company": "HTC Corporation"
  }
]

A higher rssi means closer (one of these phones is mine, and the other two are my roommates' who were upstairs).

Run forever

You can add --loop to make this run forever and append new lines an output file, test.json:

$ howmanypeoplearearound -o test.json -a wlan1 --loop

Visualize

You can visualize the output from a looped command via a browser using:

$ howmanypeoplearearound --analyze test.json 
Wrote index.html
Open browser to http://localhost:8001
Type Ctl+C to exit

Then just open up index.html in a browser and you should see plots. The first plot shows the number of people over time. Here you can see that people start arriving at work place around 8-9am (when work starts!).

newplot

The second plot shows the RSSI values for the mac addresses seen. You can double-click on one of them in particular to highlight that trajectory, as I have done here for my phone (you can see when I leave from and when I arrive to work!):

newplot 1

How does it work?

howmanypeoplearearound counts up the number of probe requests coming from cellphones in a given amount of time. The probe requests can be "sniffed" from a monitor-mode enabled WiFi adapter using tshark. An accurate count does depend on everyone having cellphone and also scanning long enough (1 - 10 minutes) to capture the packet when a phone pings the WiFi network (which happens every 1 to 10 minutes unless the phone is off or WiFi is disabled).

This is a simplification of another program I wrote, find-lf which uses a similar idea with a cluster of Raspberry Pis to geolocate positions of cellphones within the vicinity.

License

MIT

howmanypeoplearearound's People

Contributors

0xflotus avatar ansell avatar asilvino avatar asutoshpalai avatar juergenhoetzel avatar lodour avatar rawmind avatar rlucia avatar schollz avatar sente avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

howmanypeoplearearound's Issues

UnicodeDecodeError

error

that is the error i'm getting. I installed it using pip and that's it. Is there anything else required?

OUI

Hi!
Thanks very much for your work, it's very useful.
I collected some info about oui that may help

Mac that maybe cellphones
Microsoft Corporation", "mac": "48:50:73
Google, Inc.", "mac": "54:60:09

Mac that are cellphones based on scan
TCT mobile ltd", "mac": "cc:fd:17: ( former alcatel)

Mac not present in OUI that is a cellphone based on having the device
"48:db:50": "HUAWEI TECHNOLOGIES CO.,LTD"

Seeing many Not in OUi
Not in OUI", "mac": "86:42:99
Not in OUI", "mac": "da:a1:19
Not in OUI", "mac": "7a:f4:f7
Not in OUI", "mac": "7a:94:e2

TargetMacs

Is there a way to have the JSON output print names of devices that I manually input into a seperate file?
For example, if I know that my android has a mac address of 1A:DC:5C:8E:15:7B, could I manually configure a file for the script to refer to so that while running it would associate 1A:DC:5C:8E:15:7B with "myandroid" and print out the name "myandroid" for me to quickly recognize rather than the mac address?

Enhancement: OpenCV antenna counting

WiFi signals are also bent in specific ways by physical objects. Could we access the antenna directly and use something like OpenCV to count people nearby?

ModuleNotFoundError: No module named '_curses'

I am using python 3 on windows, run command in command line like this:
pip install howmanypeoplearearound
and finally it shows:
Successfully installed howmanypeoplearearound-0.3.1 netifaces-0.10.5 pick-0.6.3
However, when run command:
howmanypeoplearearound
failed:
Traceback (most recent call last): File "C:\Users\win10\AppData\Local\Programs\Python\Python36-32\Scripts\howmanypeoplearearound-script.py", line 11, in <module> load_entry_point('howmanypeoplearearound==0.3.1', 'console_scripts', 'howmanypeoplearearound')() File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 560, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 2648, in load_entry_point return ep.load() File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 2302, in load return self.resolve() File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 2308, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\howmanypeoplearearound\__main__.py", line 11, in <module> from pick import pick File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pick\__init__.py", line 3, in <module> import curses File "c:\users\win10\appdata\local\programs\python\python36-32\lib\curses\__init__.py", line 13, in <module> from _curses import * ModuleNotFoundError: No module named '_curses' PS C:\Users\win10> howmanypeoplearearound Traceback (most recent call last): File "C:\Users\win10\AppData\Local\Programs\Python\Python36-32\Scripts\howmanypeoplearearound-script.py", line 11, in <module> load_entry_point('howmanypeoplearearound==0.3.1', 'console_scripts', 'howmanypeoplearearound')() File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 560, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 2648, in load_entry_point return ep.load() File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 2302, in load return self.resolve() File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pkg_resources\__init__.py", line 2308, in resolve module = __import__(self.module_name, fromlist=['__name__'], level=0) File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\howmanypeoplearearound\__main__.py", line 11, in <module> from pick import pick File "c:\users\win10\appdata\local\programs\python\python36-32\lib\site-packages\pick\__init__.py", line 3, in <module> import curses File "c:\users\win10\appdata\local\programs\python\python36-32\lib\curses\__init__.py", line 13, in <module> from _curses import * ModuleNotFoundError: No module named '_curses'

What's the problem?

Will it detect only certain cellphones

Help needed- Does its detect only phones listed in the code? It detects wonderfully my HTC phone, but not OnePlus, MI or Xiomi phone.

Issue - The count of cellphones takes into account rssi~0 also, which seems to be incorrect output.

Unable to capture packets [Malformed packet]

Hi @schollz ,

First of all, great work here. I found this project really interesting.

Getting to the point, I was trying to run the command and it kept saying that no signals were found. I ran the tshark command manually to see what's wrong. It appears that the packets are malformed and I suspect it has something to do with the addition of radiotap headers. See the output below.

% tshark -i wlp3s0 -I -a duration:60
Capturing on 'wlp3s0'
1 0.000000000 -> WLAN 87 Radiotap Capture v255, Length 65535[Malformed Packet]
2 0.052283376 -> WLAN 136 Radiotap Capture v255, Length 65535[Malformed Packet]
3 0.241639018 -> WLAN 90 Radiotap Capture v255, Length 65535[Malformed Packet]
4 0.302993069 -> WLAN 136 Radiotap Capture v255, Length 65535[Malformed Packet]
5 0.553821626 -> WLAN 136 Radiotap Capture v255, Length 65535[Malformed Packet]

I was hoping you could help me with this. Thanks a ton !

Sanketh

Multiple MAC addresses for a single rssi in dumped JSON

For one timepoint my dumped JSON file contains the following (redacted), with two MAC addresses separated by a comma, where there is usually a single MAC address:

{"rssi": -77.0, "company": "Apple, Inc.", "mac": "48:43:7c:XX:XX:XX,23:eb:f7:XX:XX:XX"}

This causes a syntax issue in the Javascript generated for analysis that looks like the following, causing the analysis to not display:

var mac48437cXXXXXX,23ebf7XXXXXX = {
  x: timex,
  y: [rssi-numbers-omitted],
 name: '48:43:7c:XX:XX:XX,23:eb:f7:XX:XX:XX', mode: 'lines', type:'scatter' };

Not sure how to replicate it, as I don't have a raw Wireshark dump file matching it. There seems to be the implicit assumption that wireshark will never emit two mac addresses in a single line like that, which should probably be improved on by either dropping the line or splitting the mac address field and duplicating the result. However, given I am not sure why it occurred, I am not sure what the best course of action should be.

Make OUI check optional

It would be great if the OUI check was optional, particularly to understand the effects of randomized MAC addresses where they fall outside of the recognised ranges.

Flag to only collect count of people and be GDPR compliant?

Related: #31 and #4

Would it be possible to add a flag so we do NOT store MAC addresses, and only see an aggregate count of devices at a given timestamp? The program is great, but as it stands, can not be used on public networks in Europe over GDPR compliancy :/

potential bug in __main__.py

There maybe a potential bug in main.py due to the usage of path.strip('"').

The code in the which() function is similar to the code in version 6 of this Stack Overflow answer and in version 7 of the answer (5 years later), the call to path.strip('"') was removed for the reason that "'neither WIndows nor POSIX appear to encourage quoted PATH items"

Can someone kindly verify if this may cause an issue for this project?

Additional Context

We came across this issue while investigating open-source projects to see if they contain code that is similar to (or reused from) Stack Overflow and whether the code on Stack Overflow underwent any bug or security fixes. We found that the code in the which() method is somewhat similar to the one in this answer version. The issue was subsequently fixed 5 years later in version 7, however, the fix is not reflected in this function.

Most Basic Hardware Implementation

What is the simplest standalone hardware setup you would recommend for this?

What service would you run to be able to access the graphed results on another device's web browser?

No such command (analyze, loop, etc)

When I try to use analyze or loop as per the instructions, it says no such command. Some of the other ones work, and when I pull up the help info, it only lists these:


  -a, --adapter TEXT   adapter to use
  -s, --scantime TEXT  time in seconds to scan
  -o, --out TEXT       output cellphone data to file
  -v, --verbose        verbose mode
  --number             just print the number
  -j, --jsonprint      print JSON of cellphone data
  -n, --nearby         only quantify signals that are nearby (rssi > -70)
  --nocorrection       do not apply correction
  --help               Show this message and exit.

Any ideas?

Flag to plot only number of people over time

I've been running this for about 6 days now, and am literally unable to render the graphs (mainly the bottom one) because of the 2000 something mac addresses its trying to plot. Would it be possible to omit the bottom graph, possibly with a flag or something?

Am I missing any step?

Hi @schollz , first things first, thanks for making such a cool project.

I was trying to use this on my mac osx. And here's what I did.

  1. Connected to wifi
  2. Ran
$ brew install wireshark
$ brew cask install wireshark-chmodbpf
$ sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -z
  1. Installed the pip package, and ran howmanypeoplearearound

I see a very big list of options

image

I'm not sure which one is my wifi adapter that supports monitor mode. Is there any way to figure that out?

Also, I tried to bruteforce and give it a shot with all the option but everytime the response after scanning is Found no signals, are you sure en0 supports monitor mode?

Can you please help if I'm missing any step?

Raspbian Jessie + Stretch Install Failures / Permission Issues

On which version of Raspbian did you successfully install howmanypeoplearearound?

On Raspbian Jessie - I was unable to get the proper permissions for tshark working AFTER a successful install.

On Raspbian Stretch - I am unable to successfully install with Docker OR pip.
The Docker command fails with: 'returned a non-zero code: 1'
The pip command is successful, but 'howmanypeoplearearound' is not found by bash when executing: i.e. 'howmanypeoplearearound: command not found'

Video Tutorials

It's basically self explanatory but still there are very less video explainations,kindly add some video Tutorials as there are many noobs and this project is very cool

Cannot find any devices

I've tried this with two machines now and I cannot find any devices with either. Is there some way to check if the wifi devices support monitor mode?

I already did this:
From iw phy:
[...]
Supported interface modes:

  • IBSS
  • managed
  • AP
  • AP/VLAN
  • monitor
    [...]

So, something more than just the output of iw would be good.

Frequency hopping

Hey! Nice work!
I have actually used your find-lf code to develop my own verison of howmanypeoplearearound.

It is ofcourse hard to track un-associated MAC addresses because of MAC address randomizaition.
A solution to this could be, we can tack the known SSID list a particular device sends and then we can map the randomized MACs to be the same?

Another main issue I have faced is frquency hopping. I don't think tshark hops frequncies to find the devices in all the channels.
I think airodump-ng does this. I'm trying to implement that.

What is legality of monitoring traffic for mac addresses

Hey, nice job on this :) I just wanted to mention that in case you are not aware, it is against the law to sniff packets. The only exception is to sniff on your own network, and only to protect it.

It is unfortunate, otherwise it would be really nice to come up with ways to use it!

At least you should put a "big fat" warning that the use of sniffing is most likely be illegal.

MAC address randomization

Hey! Cool library. Out of curiosity, how does this handle MAC address randomization? Will it double count?

Docker image?

A Docker image would make it even easier to use, no installing / fiddling around with dependencies :-)!

Does not detect associated smart phones.

Thanks for the software, much appreciated!

AFAIK smart phones will only issue probe request when they are not currently connected to an access point. This might lead to a severe underestimation of the number of smart phones in areas where most phones are connected to an AP (e.g. bars with open WiFi, universities, ...). Not sure how this applies to Bluetooth.

Maybe this is worth a note in the README?

No Module Named _Curses - Linux

Hi I am running linux and the program called howmanypeoplearearound seems to respond with:

Traceback (most recent call last):
File "/usr/local/bin/howmanypeoplearearound", line 11, in
load_entry_point('howmanypeoplearearound==0.4.0', 'console_scripts', 'howmanypeoplearearound')()
File "/usr/local/lib/python3.6/site-packages/pkg_resources/init.py", line 565, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/local/lib/python3.6/site-packages/pkg_resources/init.py", line 2631, in load_entry_point
return ep.load()
File "/usr/local/lib/python3.6/site-packages/pkg_resources/init.py", line 2291, in load
return self.resolve()
File "/usr/local/lib/python3.6/site-packages/pkg_resources/init.py", line 2297, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/usr/local/lib/python3.6/site-packages/howmanypeoplearearound/main.py", line 17, in
from pick import pick
File "/usr/local/lib/python3.6/site-packages/pick/init.py", line 3, in
import curses
File "/usr/local/lib/python3.6/curses/init.py", line 13, in
from _curses import *
ModuleNotFoundError: No module named '_curses'

Can anyone help me with this issue as I have tried to work around it with the response to the older version but that does not work either....

howmanypeoplearearound using Windows with a Python 3 Anaconda/Spyder shell

First, you may have no reason to support this environment. If not, feel free to close and resolve. I am testing it because I am curious only. Thanks, -Pete

But I did want see the package and was able to run the install in the Spyder IDE using

pip.main(['install', 'howmanypeoplearearound'])

Following was then run and output is not informative:

import howmanypeoplearearound
howmanypeoplearearound
Out[13]: <module 'howmanypeoplearearound' from 'C:\Users\Peter\Anaconda3\lib\site-packages\howmanypeoplearearound\init.py'>

So not getting the output I expected... per the doc for this module:
Using wlan1 adapter and scanning for 60 seconds...
[==================================================] 100% 0s left
There are about 3 people around.

Edit: I should add. The TShark module was not imported. This may help explain the results.

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.