Coder Social home page Coder Social logo

pythonstuff's Introduction

PythonStuff:

Some Free Python App Hosting Options

from a longer list by Nik Tomazic at: https://testdriven.io/blog/heroku-alternatives/

Some Python code that I tend to copy & morph

New environment? Install Python:

Debian-based Linux, for example, Ubuntu or one of its variants:

sudo apt-get update
sudo apt-get install python3 python3-venv python3-pip

RedHat/Fedora-based Linux:

sudo dnf install python3

Or check out a more comprehensive description of setting up a new environment at https://github.com/mccright/PythonStuff/blob/main/New-environment-notes.md

Get the Python Launcher for Unix: https://github.com/brettcannon/python-launcher

minpyver

Sometimes it is important to enforce a minimum Python version. See Nicholas Hairs' "Summary of Major Changes Between Python Versions" for a history of some reasons why that is...

  • 'minpyver.py' - In some situations it is important to use a very specific Python version.
    Yes, it might be better to just add:
if sys.version_info < (3, 10):
    raise Exception("Use only with Python 3.10 or higher")

Python Logging

I have another repo with with some Python logging content https://github.com/mccright/PythonLoggingExamples/

Python Regex Cheatsheet

From Debuggex: https://www.debuggex.com/cheatsheet/regex/python

Python re(gex) -- a magical tool for text processing

By Sundeep Agarwal https://learnbyexample.github.io/py_regular_expressions/ or the entire book in a single markdown file at https://github.com/learnbyexample/py_regular_expressions/blob/master/py_regex.md with supporting code at https://github.com/learnbyexample/py_regular_expressions

stringSearch

  • This is a harness for evaluating the contents of files (in a directory and all child directories) using a collection of your own regex's. If you need specialized secrets-hunting utilities, see TruffleHog or Burp Suite Extension SecretFinder. I used ideas & code from both of them in this string search utility.

simpleAPIClient [REST]

(env) C:\temp\prob>python simpleAPIClient.py --help
usage: sampleAPIClient [-h] [-d] [-t] [-s] [-p] [-x PROXY_URL] -u API_URL

sampleAPIClient: an API Client POST skeleton for problem-solving

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Use to send debug logging to console
  -t, --timing          Use to send timing to console
  -s, --disable_cert_security
                        Use to disable SSL security checking. This can be a security risk concern. Use with caution.
  -p, --proxy_needed    Use to enable a proxy
  -x PROXY_URL, --proxy_url PROXY_URL
                        When enabling a proxy, this is the full URL
  -u API_URL, --api_url API_URL
                        Target api-endpoint - full URL

(env) C:\temp\prob>
  • This is not a point-and-shoot utility.
  • At a minimum, you need to modify the code: Replace the "HEADER_PARAMS" and " POST_DATA" dicts with content relevant to your problem/target.
  • Don't assume that I know what I am doing. This served its purpose on some weekend work. It has not seen many different use cases yet and may have serious limitations.
  • This is not an attempt to deal with APIs that are protected by one or another OAuth implementation or other session-related interface. Its target are those simple POST-and-done APIs that ought to be simple to use, but sometimes are not.

Sometimes you need to know what types of files are in a github repo along with their layout in order to prepare for a risk-reasonable static analysis.

  • getGHtree.py is a model for extracting a list of files in tree format from user repositories.
  • getGHorgtree.py is a model for extracting a list of files in tree format from non-public organization repositories.

getSomeIPInfo

  • getSomeIPInfo is just a reminder for me about navigating simple json. First using hard-coded references, and then iterating through every key/value pair. Both approaches have their place. There is another example used in getGHtree.py and getGHorgtree.py.

useRandomUserAgent

checkResponseCodes

  • checkResponseCodes is a list of all the http Codes from the IANA Hypertext Transfer Protocol (HTTP) Status Code Registry in the form of a long 'case statement.' I wanted it around so that I could copy ot the subset that I needed at any given time. It is not meant to be used as is.

http-response-codes

  • http-response-codes is a CLI script that emits a list of all the http Codes from your current http module plus their short description & long description.
    I usually pipe its output through grep for the code I am trying to understand. I found that when troubleshooting people's cloud-hosted lambdas & functions I run into more obscure response codes and need to check their meaning.

createRandomStrings.py

  • createRandomStrings is some unfinished experimenting with different ways to create 'unique' strings, a common requirement...

encryptstr.py

encryptstr is a sketal set of AES-CBC string encryption/decryption functions.

get-pdf-text.py

get-pdf-text is an informal approach to using 'pypdf' to extract text from PDF files that often works well-enough for me. It is constructed from examples in the pypdf docs. pypdf can extract a range of PDF components.

otherNotes.md

  • otherNotes.md is just a collection of short code fragments that act as reminders for me.

AWS Lambdas

Minify HTML

Scrape some text on the Web

Run your Jupyter notebook on the command line

https://github.com/jsvine/nbexec

Python and Visual Studio Code (VSCode)

See: "Python Development in Visual Studio Code." by Jon Fincher
https://realpython.com/python-development-visual-studio-code/
and "Advanced Visual Studio Code for Python Developers." by Anthony Shaw
https://realpython.com/advanced-visual-studio-code-python/

Python Enhancement Proposal 20 -- PEP-20

(available via: import this)
The Zen of Python by Tim Peters (1999). This is often described as the core philosophy of Python.
https://peps.python.org/pep-0020/#the-zen-of-python
or
https://github.com/python/peps/blob/main/pep-0020.txt

Python conventions

  • Put a space before a comment: # This is a comment
  • Don't make lines longer than ~80 characters
  • Constants in all-caps: MY_CONSTANT
  • Use underscores to separate words in variable names: my_variable
  • Avoid meaningless variable names. Avoid numbers in variable names. Wrong: thing1, thing2. Right: cat_list, fluffy_cat_list.
  • When ambiguous, put variable type in name: my_list or my_set. This is particularly important for collections. Is it a dict or a list?
  • Document code with triple quotes (multiline comments): """My documentation"""
  • Write functions when you find yourself repeating code
  • When importing modules, don't import specific functions. Import the whole module, and use the module name and function together. Right: import time; time.sleep(1). Wrong: from time import sleep; sleep(1). (Are there exceptions to this rule?)
  • When you find yourself checking if items are in a list, use a set
  • Write a snippet of documentation at the top of your file to help you remember what the file does.
  • Write inputs and outputs to functions in a comment in the function body.
  • Debugging has its place, but don't hesitate to use print() statements.
    (The original conventions in this list are from a readme by https://github.com/georgeberry. Thank you George Berry.)
  • Use better-exceptions during local development, and use care to keep it out of your production deployments.
  • Another approach would be to use pymg, a CLI tool that can interpret Python files by the Python interpreter and display the error message in a more readable way if an exception occurs https://github.com/mimseyedi/pymg
  • What is the difference between using "_" and "__" in variable or function names? The responses get at some Python conventions: https://old.reddit.com/r/learnpython/comments/s5z0l8/can_someone_explain_and_in_python_clearly_for_me/

External References

pythonstuff's People

Contributors

dependabot[bot] avatar mccright avatar mend-bolt-for-github[bot] avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar

pythonstuff's Issues

CVE-2023-45803 (Medium) detected in urllib3-1.26.17-py2.py3-none-any.whl - autoclosed

CVE-2023-45803 - Medium Severity Vulnerability

Vulnerable Library - urllib3-1.26.17-py2.py3-none-any.whl

HTTP library with thread-safe connection pooling, file post, and more.

Library home page: https://files.pythonhosted.org/packages/48/fe/a5c6cc46e9fe9171d7ecf0f33ee7aae14642f8d74baa7af4d7840f9358be/urllib3-1.26.17-py2.py3-none-any.whl

Path to dependency file: /encryptTest/requirements.txt

Path to vulnerable library: /encryptTest/requirements.txt

Dependency Hierarchy:

  • requests-2.31.0-py3-none-any.whl (Root Library)
    • urllib3-1.26.17-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 9d29714401fd5f0cb4d8d4b41051851bcf065aa7

Found in base branch: main

Vulnerability Details

urllib3 is a user-friendly HTTP client library for Python. urllib3 previously wouldn't remove the HTTP request body when an HTTP redirect response using status 301, 302, or 303 after the request had its method changed from one that could accept a request body (like POST) to GET as is required by HTTP RFCs. Although this behavior is not specified in the section for redirects, it can be inferred by piecing together information from different sections and we have observed the behavior in other major HTTP client implementations like curl and web browsers. Because the vulnerability requires a previously trusted service to become compromised in order to have an impact on confidentiality we believe the exploitability of this vulnerability is low. Additionally, many users aren't putting sensitive data in HTTP request bodies, if this is the case then this vulnerability isn't exploitable. Both of the following conditions must be true to be affected by this vulnerability: 1. Using urllib3 and submitting sensitive information in the HTTP request body (such as form data or JSON) and 2. The origin service is compromised and starts redirecting using 301, 302, or 303 to a malicious peer or the redirected-to service becomes compromised. This issue has been addressed in versions 1.26.18 and 2.0.7 and users are advised to update to resolve this issue. Users unable to update should disable redirects for services that aren't expecting to respond with redirects with redirects=False and disable automatic redirects with redirects=False and handle 301, 302, and 303 redirects manually by stripping the HTTP request body.

Publish Date: 2023-10-17

URL: CVE-2023-45803

CVSS 3 Score Details (4.2)

Base Score Metrics:

  • Exploitability Metrics:
    • Attack Vector: Adjacent
    • Attack Complexity: High
    • Privileges Required: High
    • User Interaction: None
    • Scope: Unchanged
  • Impact Metrics:
    • Confidentiality Impact: High
    • Integrity Impact: None
    • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-g4mx-q9vg-27p4

Release Date: 2023-10-17

Fix Resolution: urllib3 - 1.26.18,2.0.7


Step up your Open Source Security Game with Mend here

CVE-2022-23491 (Medium) detected in certifi-2021.10.8-py2.py3-none-any.whl - autoclosed

CVE-2022-23491 - Medium Severity Vulnerability

Vulnerable Library - certifi-2021.10.8-py2.py3-none-any.whl

Python package for providing Mozilla's CA Bundle.

Library home page: https://files.pythonhosted.org/packages/37/45/946c02767aabb873146011e665728b680884cd8fe70dde973c640e45b775/certifi-2021.10.8-py2.py3-none-any.whl

Path to dependency file: /encryptTest/requirements.txt

Path to vulnerable library: /encryptTest/requirements.txt

Dependency Hierarchy:

  • certifi-2021.10.8-py2.py3-none-any.whl (Vulnerable Library)

Found in base branch: main

Vulnerability Details

Certifi is a curated collection of Root Certificates for validating the trustworthiness of SSL certificates while verifying the identity of TLS hosts. Certifi 2022.12.07 removes root certificates from "TrustCor" from the root store. These are in the process of being removed from Mozilla's trust store. TrustCor's root certificates are being removed pursuant to an investigation prompted by media reporting that TrustCor's ownership also operated a business that produced spyware. Conclusions of Mozilla's investigation can be found in the linked google group discussion.

Publish Date: 2022-12-07

URL: CVE-2022-23491

CVSS 3 Score Details (6.8)

Base Score Metrics:

  • Exploitability Metrics:
    • Attack Vector: Network
    • Attack Complexity: Low
    • Privileges Required: High
    • User Interaction: None
    • Scope: Changed
  • Impact Metrics:
    • Confidentiality Impact: None
    • Integrity Impact: High
    • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://www.cve.org/CVERecord?id=CVE-2022-23491

Release Date: 2022-12-07

Fix Resolution: certifi - 2022.12.07


Step up your Open Source Security Game with Mend here

CVE-2024-35195 (Medium) detected in requests-2.31.0-py3-none-any.whl

CVE-2024-35195 - Medium Severity Vulnerability

Vulnerable Library - requests-2.31.0-py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl

Path to dependency file: /datetime-from-timeapi/requirements.txt

Path to vulnerable library: /datetime-from-timeapi/requirements.txt,/encryptTest/requirements.txt

Dependency Hierarchy:

  • requests-2.31.0-py3-none-any.whl (Vulnerable Library)

Found in base branch: main

Vulnerability Details

Requests is a HTTP library. Prior to 2.32.0, when making requests through a Requests Session, if the first request is made with verify=False to disable cert verification, all subsequent requests to the same host will continue to ignore cert verification regardless of changes to the value of verify. This behavior will continue for the lifecycle of the connection in the connection pool. This vulnerability is fixed in 2.32.0.

Publish Date: 2024-05-20

URL: CVE-2024-35195

CVSS 3 Score Details (5.6)

Base Score Metrics:

  • Exploitability Metrics:
    • Attack Vector: Local
    • Attack Complexity: High
    • Privileges Required: High
    • User Interaction: Required
    • Scope: Unchanged
  • Impact Metrics:
    • Confidentiality Impact: High
    • Integrity Impact: High
    • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-9wx4-h78v-vm56

Release Date: 2024-05-20

Fix Resolution: requests - 2.32.0


Step up your Open Source Security Game with Mend here

CVE-2023-43804 (High) detected in urllib3-1.26.15-py2.py3-none-any.whl - autoclosed

CVE-2023-43804 - High Severity Vulnerability

Vulnerable Library - urllib3-1.26.15-py2.py3-none-any.whl

HTTP library with thread-safe connection pooling, file post, and more.

Library home page: https://files.pythonhosted.org/packages/7b/f5/890a0baca17a61c1f92f72b81d3c31523c99bec609e60c292ea55b387ae8/urllib3-1.26.15-py2.py3-none-any.whl

Path to dependency file: /datetime-from-timeapi/requirements.txt

Path to vulnerable library: /datetime-from-timeapi/requirements.txt,/datetime-from-timeapi/requirements.txt

Dependency Hierarchy:

  • urllib3-1.26.15-py2.py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 8f2d8dafe96118c117f69ad4a1d41e9e97d669f2

Found in base branch: main

Vulnerability Details

urllib3 is a user-friendly HTTP client library for Python. urllib3 doesn't treat the Cookie HTTP header special or provide any helpers for managing cookies over HTTP, that is the responsibility of the user. However, it is possible for a user to specify a Cookie header and unknowingly leak information via HTTP redirects to a different origin if that user doesn't disable redirects explicitly. This issue has been patched in urllib3 version 1.26.17 or 2.0.5.

Publish Date: 2023-10-04

URL: CVE-2023-43804

CVSS 3 Score Details (8.1)

Base Score Metrics:

  • Exploitability Metrics:
    • Attack Vector: Network
    • Attack Complexity: Low
    • Privileges Required: Low
    • User Interaction: None
    • Scope: Unchanged
  • Impact Metrics:
    • Confidentiality Impact: High
    • Integrity Impact: High
    • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: https://www.cve.org/CVERecord?id=CVE-2023-43804

Release Date: 2023-10-04

Fix Resolution: 1.26.17


Step up your Open Source Security Game with Mend here

CVE-2023-32681 (Medium) detected in requests-2.28.2-py3-none-any.whl

CVE-2023-32681 - Medium Severity Vulnerability

Vulnerable Library - requests-2.28.2-py3-none-any.whl

Python HTTP for Humans.

Library home page: https://files.pythonhosted.org/packages/d2/f4/274d1dbe96b41cf4e0efb70cbced278ffd61b5c7bb70338b62af94ccb25b/requests-2.28.2-py3-none-any.whl

Path to dependency file: /datetime-from-timeapi/requirements.txt

Path to vulnerable library: /datetime-from-timeapi/requirements.txt

Dependency Hierarchy:

  • requests-2.28.2-py3-none-any.whl (Vulnerable Library)

Found in HEAD commit: 66b888b6600690f770dc68d55772b85ae6e856e7

Found in base branch: main

Vulnerability Details

Requests is a HTTP library. Since Requests 2.3.0, Requests has been leaking Proxy-Authorization headers to destination servers when redirected to an HTTPS endpoint. This is a product of how we use rebuild_proxies to reattach the Proxy-Authorization header to requests. For HTTP connections sent through the tunnel, the proxy will identify the header in the request itself and remove it prior to forwarding to the destination server. However when sent over HTTPS, the Proxy-Authorization header must be sent in the CONNECT request as the proxy has no visibility into the tunneled request. This results in Requests forwarding proxy credentials to the destination server unintentionally, allowing a malicious actor to potentially exfiltrate sensitive information. This issue has been patched in version 2.31.0.

Publish Date: 2023-05-26

URL: CVE-2023-32681

CVSS 3 Score Details (6.1)

Base Score Metrics:

  • Exploitability Metrics:
    • Attack Vector: Network
    • Attack Complexity: High
    • Privileges Required: None
    • User Interaction: Required
    • Scope: Changed
  • Impact Metrics:
    • Confidentiality Impact: High
    • Integrity Impact: None
    • Availability Impact: None

For more information on CVSS3 Scores, click here.

Suggested Fix

Type: Upgrade version

Origin: GHSA-j8r2-6x86-q33q

Release Date: 2023-05-26

Fix Resolution: requests -2.31.0


Step up your Open Source Security Game with Mend here

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.