martinbrose / cloudflarepycli

Python CLI and python class for retrieving user's realtime performance statistics

🤖 AI & ML Enthusiast | 📊 Data Scientist | 💼 Quantitative Developer | 🌐 Model Risk Consultant

🚀 Welcome to my GitHub profile! I'm an AI & ML enthusiast, quantitative developer, data scientist, and model risk consultant with an interest in blending innovation and data for a smarter world.

🛠️ Technical Stack

• Python 🐍
• SQL 📊
• TensorFlow 🧠
• Scikit-Learn 🧪
• React ⚛️
• Docker 🐳
• AWS ☁️
• Azure ☁️
• GCP ☁️

🤝 Open Source Contributions

• huggingface/huggingface_hub: The official Python client for the Huggingface Hub.
• tevslin/cloudflarepycli: Python CLI and Python class for retrieving user's realtime performance statistics
• geerlingguy/internet-pi: A Raspberry Pi Configuration for Internet connectivity
• lballabio/QuantLib: The free/open-source library for quantitative finance
• pi-hole: A black hole for Internet advertisements - Network-wide ad blocking via your own Linux hardware

💬 Let's Connect

🙌 Excited to collaborate or chat about AI and data? Don't hesitate to reach out! 📩

If you like what I've created, please consider buying me a coffee:

cloudflarepycli's Issues

Feature Request: Add 100MB Download Tests to Match Cloudflare Web Test

Hi there,

Would it be possible to add a 100 MB test to match that of the Cloudflare speed test?
Currently the website shows 100 kB, 1 MB, 10 MB, 25 MB and 100 MB.
However, the latest version of cloudflarepycli only has the first 4.

Feature request: remove Numpy dependency

I'd like to propose to remove the dependency on the Numpy package.

Reason

I'm using cloudflarepycli as part of docker containers that I use for a selection of OS flavors, e.g. linux/arm/v7 and linux/arm/v6. Numpy doesn't seem to be available for these platforms and as a result, a wheel is being built during the docker build process. This is very time-consuming and actually increases the resulting docker image size significantly, from 90MB to 160MB.

Upload speed is incorrect

Hi @tevslin,

unfortunately, the upload speed is incorrect when using cloudflarepycli.

This behaviour started on 21st of May:

I have run the Python package myself and could replicate the strange upload measurements:

But when I run the speedtest on the Cloudflare-Website, everything seems fine:

There might have been a change to the servertimes we use in the calculation in lines 212-213 of cloudflareclass.py.

Breaking change for Linux with introduction of the wres package

Thanks for writing this little helper class for the cloudflare speedtest.

I received the following error when I tried to use it on a Mac / Linux machine:

Traceback (most recent call last):
  File "/home/vagrant/anaconda3/envs/speedtest/bin/cfspeedtest", line 8, in <module>
    sys.exit(main())
  File "/home/vagrant/anaconda3/envs/speedtest/lib/python3.8/site-packages/cloudflarepycli/__main__.py", line 35, in main
    thedict=thetester.runalltests()
  File "/home/vagrant/anaconda3/envs/speedtest/lib/python3.8/site-packages/cloudflarepycli/cloudflareclass.py", line 154, in runalltests
    fulltimes,servertimes,requesttimes=self.download(1,self.latencyreps) #measure latency and jitter
  File "/home/vagrant/anaconda3/envs/speedtest/lib/python3.8/site-packages/cloudflarepycli/cloudflareclass.py", line 101, in download
    import wres  
  File "/home/vagrant/anaconda3/envs/speedtest/lib/python3.8/site-packages/wres/__init__.py", line 16, in <module>
    ntdll = ctypes.WinDLL('NTDLL.DLL')
AttributeError: module 'ctypes' has no attribute 'WinDLL'

I think you introduced a Windows DLL dependency in commit 841f452 when you added import wres. Was this intentional?

Your PyPI version 1.0.3 works fine, but version 1.6.0 won't run on Linux machines.

bs4 dependency missing

On all systems where I tried this I had to manually add the bs4 package, can you please add it to the dependencies?

Big thanks!

Option --json doesn't return proper json with version 1.8

Hi @tevslin,

when using command cfspeedtest --json with version 1.8, the tool doesn't return proper json.
When running the output through json.loads(), I receive the following error message: json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 1151).

I believe the reason are the additional prints you introduced in src/cloudflarepycli/__main__.py lines 38-42.

If you prefer to keep them, would it be possible to put them into an else as part of the if args.json:?

Thank you!

Allow specifying use of IPv4 or IPv6, and other housekeeping

Hey there, I'm using this package via a Docker container another user has built ( https://github.com/martinbrose/cloudflare-speedtest-exporter ) and I had a handful of thoughts.

Per your README;

There is also a request to ipdatabase.com and the return is screen-scraped for the actual ISP name.

This is not (no longer?) necessary as the Cloudflare stats expose this information in their data.

$ cfspeedtest --json
{"version": {"time": 1658013890.4583306, "value": 1.5}, "your_ip": {"time": 1658013890.5290189, "value": "24.9.13.37"}, …

Additionally, ipdatabase doesn't have AAAA records, so it will never report an IPv6 address even if you have one. For me, speed.cloudflare.com as well as cfspeedtest on my Mac does. I'm unsure why it only uses IPv4 in my Docker container, but there is a lot about Docker networking I don't know yet.

➤ cfspeedtest
version: 1.6
your ip: 2601:285:c133:7:dead::beef

I'm eager to be able to stagger v4 and v6 speed tests and collect those metrics uniquely.

Version 1.8.1 on PyPI

Hi @tevslin,

when will version 1.8.1 be available on PyPI?

Please let me know if I can help with anything.

Best regards,
Martin

Code breaks when ipdatabase is down or unreachable

Trace -
Code breaks when ipdatabase is down (As of the opening of this issue)

version: 1.7.0
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/cfspeedtest", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cloudflarepycli/__main__.py", line 35, in main
    thedict=thetester.runalltests()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cloudflarepycli/cloudflareclass.py", line 153, in runalltests
    colo,ip=self.getcolo()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/cloudflarepycli/cloudflareclass.py", line 74, in getcolo
    r=self.mequests.get('http://speed.cloudflare.com/cdn-cgi/trace')
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/requests/adapters.py", line 547, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))```

ipdatabase.com has shutdown

Apparently, ipdatabase.com has shutdown and the domain is up for sale.
This leads to the library failing due to the response sent by the webserver. https://github.com/tevslin/cloudflarepycli/blob/29ea4530ac8bf227c4090237ea2f91b8b4730d45/src/cloudflarepycli/cloudflareclass.py#L81-L88

$ curl -L https://ipdatabase.com/ip/ip_address
<html>
  <head>
    <meta http-equiv='refresh' content='0; url=https://ipdatabase.com/' />
  </head>
</html>

I wonder if it would be possible to remove the dependencies from external services, since Cloudflare should provide all the relevant information.

--bits option

Hello.
Can you add "show output in bits" option to collect data in monitoring systems (like zabbix)?
Look at #7