defaultnamehere / zzzzz Goto Github PK
View Code? Open in Web Editor NEWRoleplay as the NSA by making creepy graphs of your friends
License: MIT License
Roleplay as the NSA by making creepy graphs of your friends
License: MIT License
Have other experienced unexpected crashes of fetcher.py?
when I load facebook.com/pull
It says Page not found, neither I get client_id
in browser GET request, am I missing something?
Here is the full error:
/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Is it just me or everyone is having this?
Hi there guys,
I'm trying this out, but don't have any programming experience.. I've installed phyton 3.9, but never used this before. Can someone explain me step-by-step how to get this script working? As for the secrets.txt, how should the cookie information in this txt file like like..maybe an example?
I've ran the fetcher.py script but with next output error:
Traceback (most recent call last):
File "\fetcher.py", line 6, in
import requests
ModuleNotFoundError: No module named 'requests'
any idea?
thank for your patience ;-)
MK
As title says, would be nice to have an extra script that just grabs your friend list and dumps it to a file for easy Name->UID mapping.
Many projects include a Dockerfile which allows other folks to quickly run the project in a controlled environment.
Provide one for this project.
Traceback (most recent call last):
File "./fetcher.py", line 182, in <module>
f.start_request()
File "./fetcher.py", line 97, in start_request
resp = self.make_request()
File "./fetcher.py", line 54, in make_request
response_obj = requests.get(url, params=self.params, headers=self.REQUEST_HEADERS)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 67, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 53, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 447, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: SysCallError(104, 'ECONNRESET')",)
This should not result in a crash but in a retry
So when I run python fetcher.py
I just repeatedly get this line in the console:
Response:{u'reason': 110, u't': u'refresh', u'seq': 0}
Can anyone shed any light on what it is and how to fix it? Nothing is going into the log directory at all.
If I fiddle around with the SECRETS.txt
file and make something invalid it fails for different reasons, so I can only assume the cookie and IDs are working ok.
This is on Windows 7 if that makes a difference...
Hello,
you should note python-dev and libssl-dev as requirement. I had some SSL issues which I got rid by installing both and going for
pip install 'requests[security]'
afterwards. This got me going.
However now I get the following response. Not sure where the request is failing.
#python fetcher.py
>
Got error from request, restarting...
>
Got error from request, restarting...
>
Got error from request, restarting...
>
Got error from request, restarting...
^CTraceback (most recent call last):
File "fetcher.py", line 177, in <module>
time.sleep(SLEEP_TIME)
KeyboardInterrupt
I would debug the HTTP requests but it is all TLS1.2 encrypted. Any idea how to proceed?
unable to obtain client id, gives authentication error.
Probably same result can be achieved via official api in similar way Pidgin uses.
I keep getting the "Got error from request, restarting..."
and facebook seems like changed the response
"buddyList":{"id":{"lat":1462613641},"id":{"lat":1462687695},"id":{"lat":1462694839},"id":{"lat":0},"id":{"lat":1462798728,"p":2,"vc":10}
this is the error message:
Traceback (most recent call last):
File "./fetcher.py", line 180, in <module>
f.start_request()
File "./fetcher.py", line 122, in start_request
self._log_lat(uid, str(item["overlay"][uid]["la"]))
KeyError: 'la'
I "fixed" this by checking whether the key exists first. But I'm not sure whether this might break something down the road e.g. in graph.py
Hello! Can you please include a screen shot in your github so we all know what kind of graphs you can create? Thanks!
Hi,
uhm..
Do you have an example for this cookie?
I have several entries in my "facebook cookie" ;)
Thanks,
What I get from the pull request is a list of friends with the following information:
"<user_id>":{ "lat":1481207023, "p":2, "vc":10 }
someone knows what the fields "p" and "vc" stand for?
In README.md, clarify that cookie in SECRETS.txt needs single-quotes
Hey ! I see that implemented a blacklist for contacts you don't want to see. However, to be really usable for people with many facebook friends, I think it could be better to have a configurable 'whitelist', to include only selected friends
Thank you for your time !
I made a PHP version of this, perhaps I could get a link here as the PHP fork of this project.
When I run python app.py
and then visit http://0.0.0.0:5000/, I can see the input form. However, giving a user ID there, it then goes to a 405 Method Not Allowed
page. Not sure what I am doing wrong.
After producing output for a few minutes, fetcher.py crashes with this error:
Traceback (most recent call last):
File "fetcher.py", line 175, in <module>
f.start_request()
File "fetcher.py", line 124, in start_request
user_data.append(json.dumps(item["overlay"][uid]["p"]))
KeyError: 'p'
When I run pip install -r requirements.txt
gives this error
my requirements.txt:
requests==2.9.1
Flask==0.10.1
Downloading/unpacking requests==2.9.1 (from -r requirements.txt (line 1))
Downloading requests-2.9.1-py2.py3-none-any.whl (501kB): 501kB downloaded
Downloading/unpacking Flask==0.10.1 (from -r requirements.txt (line 2))
Downloading Flask-0.10.1.tar.gz (544kB): 544kB downloaded
Running setup.py (path:/tmp/pip_build_deploy/Flask/setup.py) egg_info for package Flask
warning: no files found matching '*' under directory 'tests'
warning: no previously-included files matching '*.pyc' found under directory 'docs'
warning: no previously-included files matching '*.pyo' found under directory 'docs'
warning: no previously-included files matching '*.pyc' found under directory 'tests'
warning: no previously-included files matching '*.pyo' found under directory 'tests'
warning: no previously-included files matching '*.pyc' found under directory 'examples'
warning: no previously-included files matching '*.pyo' found under directory 'examples'
no previously-included directories found matching 'docs/_build'
no previously-included directories found matching 'docs/_themes/.git'
Downloading/unpacking Werkzeug>=0.7 (from Flask==0.10.1->-r requirements.txt (line 2))
Downloading Werkzeug-0.11.4-py2.py3-none-any.whl (305kB): 305kB downloaded
Downloading/unpacking Jinja2>=2.4 (from Flask==0.10.1->-r requirements.txt (line 2))
Downloading Jinja2-2.8-py2.py3-none-any.whl (263kB): 263kB downloaded
Downloading/unpacking itsdangerous>=0.21 (from Flask==0.10.1->-r requirements.txt (line 2))
Downloading itsdangerous-0.24.tar.gz (46kB): 46kB downloaded
Running setup.py (path:/tmp/pip_build_deploy/itsdangerous/setup.py) egg_info for package itsdangerous
warning: no previously-included files matching '*' found under directory 'docs/_build'
Downloading/unpacking MarkupSafe (from Jinja2>=2.4->Flask==0.10.1->-r requirements.txt (line 2))
Downloading MarkupSafe-0.23.tar.gz
Running setup.py (path:/tmp/pip_build_deploy/MarkupSafe/setup.py) egg_info for package MarkupSafe
Installing collected packages: requests, Flask, Werkzeug, Jinja2, itsdangerous, MarkupSafe
Found existing installation: requests 2.2.1
Not uninstalling requests at /usr/lib/python2.7/dist-packages, owned by OS
Can't roll back requests; was not uninstalled
Cleaning up...
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1436, in install
requirement.install(install_options, global_options, _args, *_kwargs)
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 672, in install
self.move_wheel_files(self.source_dir, root=root)
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 902, in move_wheel_files
pycompile=self.pycompile,
File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 206, in move_wheel_files
clobber(source, lib_dir, True)
File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 193, in clobber
os.makedirs(destsubdir)
File "/usr/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/requests-2.9.1.dist-info'
Storing debug log for failure in /home/deploy/.pip/pip.log
I might be missing something here, but shouldn't there be a requirements.txt
file?
Running pip install -r requirements.txt
gives me:
Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
I have the fetcher continuously updating in the background, and running graph.py gets me great csv's, but when I run app.py the web app does not use the updated csv's. I tried deleting the generated_graphs folder and then re-generating it with graph.py, but that didn't help. How do I make the web app use the latest data?
I have not registered any activity for these stati. Maybe their key has changed?
I tried:
python /home/openhabian/zzzzz/fetcher.py &
But it didn't work. Even
python /home/openhabian/zzzzz/fetcher.py
says
SECRETS.txt: no such file or directory
I have to manually navigate to zzzzz folder using cd command and then run
python fetcher.py
in order to make it run
Same goes for other scripts. How to run them in background?
I can't find it -_-
I had no problem finding the 2 IDs, but I can't find the cookie.
I checked logs, and my router was unreachable for 5 minutes but because the script crashed - I lost about a few hours of potential data. Here is the trace:
Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
File "fetcher.py", line 61, in make_request
data = json.loads(raw_response)
File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "fetcher.py", line 176, in <module>
f.start_request()
File "fetcher.py", line 93, in start_request
resp = self.make_request()
File "fetcher.py", line 64, in make_request
print(content)
NameError: name 'content' is not defined
I started the fetcher using
python2.7 fetcher.py
After two failed attempts I quit the script usinc CTRL+C.
It showed this Traceback:
>
Got error from request, restarting...
>
Got error from request, restarting...
>
^CTraceback (most recent call last):
File "fetcher.py", line 176, in <module>
f.start_request()
File "fetcher.py", line 93, in start_request
resp = self.make_request()
File "fetcher.py", line 49, in make_request
response_obj = requests.get(url, params=self.params, headers=self.REQUEST_HEADERS)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 67, in get
return request('get', url, params=params, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 53, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 376, in send
timeout=timeout
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
body=body, headers=headers)
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 784, in _validate_conn
conn.connect()
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/connection.py", line 252, in connect
ssl_version=resolved_ssl_version)
File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 300, in ssl_wrap_socket
cnx.do_handshake()
File "/usr/local/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1441, in do_handshake
result = _lib.SSL_do_handshake(self._ssl)
KeyboardInterrupt
Probably an issue with SECRETS.txt.
the file looks like this now: (with the numbers letters altered and the cookie redacted)
uid=100000232323239
cookie=datr=JuOSKadJDmdiU8_gn3djr4B; locale=en_US; c_user=[Redacted cookie]wd=882x749
client_id=88c9g39b
Terminal outputs:
Felipes-MacBook-Air:zzzzz-master felipe$ python fetcher.py
Traceback (most recent call last):
File "fetcher.py", line 8, in
import graph
File "/Users/felipe/Downloads/zzzzz-master/graph.py", line 6, in
import flap
File "/Users/felipe/Downloads/zzzzz-master/fbapi.py", line 4, in
import fetcher
File "/Users/felipe/Downloads/zzzzz-master/fetcher.py", line 25, in
class Fetcher():
File "/Users/felipe/Downloads/zzzzz-master/fetcher.py", line 31, in Fetcher
'cookie': secrets.cookie,
AttributeError: 'Namespace' object has no attribute 'cookie'
Hello Guys,
The code is running well, but most of the time when I check the status of the 'fetcher.py' I find that the connection is interrupted with the following response. Do anyone have the same issue? if yes, did you try to overcome it?
Response:{u'ms': [{u'type': u'buddylist_overlay', u'realtime_viewer_fbid': 100002051235984L, u'overlay': {u'701918867': {u'a': 2, u'ol': 0, u'vc': 8}}}], u'u': 100002051235984L, u't': u'msg', u'seq': 1330}
Traceback (most recent call last):
File "fetcher.py", line 179, in
f.start_request()
File "fetcher.py", line 121, in start_request
self._log_lat(uid, str(item["overlay"][uid]["la"]))
KeyError: 'la'
The code below showed up
Traceback (most recent call last): File "fetcher.py", line 179, in <module> f.start_request() File "fetcher.py", line 95, in start_request resp = self.make_request() File "fetcher.py", line 69, in make_request print("Response:" + str(data)) File "C:\Users\TKGMoMoSheep\AppData\Local\Programs\Python\Python35-32\lib\encodings\cp437.py", line 19, in encode return codecs.charmap_encode(input,self.errors,encoding_map)[0] UnicodeEncodeError: 'charmap' codec can't encode character '\u6bdb' in position 1413: character maps to <undefined>
Running fetcher.py yields this.
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:31 5: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause t he server to present an incorrect TLS certificate, which can cause validation fa ilures. For more information, see https://urllib3.readthedocs.org/en/latest/secu rity.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:12 0: InsecurePlatformWarning: A true SSLContext object is not available. This prev ents urllib3 from configuring SSL appropriately and may cause certain SSL connec tions to fail. For more information, see https://urllib3.readthedocs.org/en/late st/security.html#insecureplatformwarning.
InsecurePlatformWarning
Got error from request, restarting...
How do I get it to access the facebook api?
_
Sorry, I've never written code or anything, I literally have no programming experience and this is the first creation that I decided to attempt to use.
_
I ran python graph.py
then python app.py
then used chrome and mozilla firefox to access both 0.0.0.0:5000 and localhost:8000; got the "This webpage is unavailable" error for both.
What did I do wrong?
I think I got through all the steps (unless SECRET.txt has to be formatted in some different way?); all the information is right and cmd is no longer giving me an error.
Trying to run
python graph.py
yields the following error
Traceback (most recent call last): File "graph.py", line 64, in <module> g.generate_all_csvs(start_time=now - 3 * ONE_DAY_SECONDS, end_time=now) File "graph.py", line 56, in generate_all_csvs self.to_csv(uid, start_time, end_time) File "graph.py", line 27, in to_csv status_history = history.StatusHistory(uid) File "/Users/vinted/Documents/zzzzz-master/history.py", line 18, in __init__ with open("log/{uid}.txt".format(uid=uid)) as f: IOError: [Errno 2] No such file or directory: 'log/.txt'
A workaround is to replace {uid} with a concrete uid
fetcher.py is throwing this error.
python fetcher.py
Traceback (most recent call last):
File "fetcher.py", line 8, in <module>
import graph
File "/home/user/apps/Stalky/graph.py", line 6, in <module>
import fbapi
File "/home/user/apps/Stalky/fbapi.py", line 4, in <module>
import fetcher
File "/home/user/apps/Stalky/fetcher.py", line 17, in <module>
setattr(secrets, vals[0].lower(), vals[1])
IndexError: list index out of range
I have Python2.7 and have this error in both virtualenv and outside virtualenv.
I renamed the app folder "Stalky", added "SECRETS.txt with appropriate info as far as I can tell.
With python2.7:
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Got error from request, restarting...
python3:
Traceback (most recent call last):
File "fetcher.py", line 6, in <module>
import requests
ImportError: No module named requests
Yes, I read the page recommended for python2.7. No, I am not not able to fix it with the recommended solutions.
Yes, I ran all that "pip install requests" etc, and the rest you can find here.
using my raspberry 2 with debian.
edit:
got a little further with: sudo apt-get install python3-requests
Traceback (most recent call last):
File "fetcher.py", line 194, in <module>
f.start_request()
File "fetcher.py", line 110, in start_request
resp = self.make_request()
File "fetcher.py", line 52, in make_request
raw_response = response_obj.text
File "/usr/lib/python3/dist-packages/requests/models.py", line 831, in text
content = str(self.content, encoding, errors='replace')
TypeError: str() argument 2 must be str, not None
Hey, cool project. I used it a monthish ago to gather some data and it work great. However, now when I try to use it (after replacing my cookie with a more recent one), I get a json blob with what I assume is the expected data and then immediately the error:
Traceback (most recent call last):
File "fetcher.py", line 175, in <module>
f.start_request()
File "fetcher.py", line 124, in start_request
user_data.append(json.dumps(item["overlay"][uid]["p"]))
KeyError: 'p'
Thoughts? Any tips on how to fix this or where to look? I'm using it for a data vis project and while I'm too busy to fix it right now, I could submit a PR in the near future.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.