Coder Social home page Coder Social logo

python-tls-client's Introduction

Python-TLS-Client

Python-TLS-Client is an advanced HTTP library based on requests and tls-client.

Installation

pip install tls-client

Examples

The syntax is inspired by requests, so its very similar and there are only very few things that are different.

Example 1 - Preset:

import tls_client

# You can also use the following as `client_identifier`:
# Chrome --> chrome_103, chrome_104, chrome_105, chrome_106, chrome_107, chrome_108, chrome109, Chrome110,
#            chrome111, chrome112, chrome_116_PSK, chrome_116_PSK_PQ, chrome_117, chrome_120
# Firefox --> firefox_102, firefox_104, firefox108, Firefox110, firefox_117, firefox_120
# Opera --> opera_89, opera_90
# Safari --> safari_15_3, safari_15_6_1, safari_16_0
# iOS --> safari_ios_15_5, safari_ios_15_6, safari_ios_16_0
# iPadOS --> safari_ios_15_6
# Android --> okhttp4_android_7, okhttp4_android_8, okhttp4_android_9, okhttp4_android_10, okhttp4_android_11,
#             okhttp4_android_12, okhttp4_android_13
#
# more client identifiers can be found in settings.py

session = tls_client.Session(
    client_identifier="chrome112",
    random_tls_extension_order=True
)

res = session.get(
    "https://www.example.com/",
    headers={
        "key1": "value1",
    },
    proxy="http://user:password@host:port"
)

Example 2 - Custom:

import tls_client

session = tls_client.Session(
    ja3_string="771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0",
    h2_settings={
        "HEADER_TABLE_SIZE": 65536,
        "MAX_CONCURRENT_STREAMS": 1000,
        "INITIAL_WINDOW_SIZE": 6291456,
        "MAX_HEADER_LIST_SIZE": 262144
    },
    h2_settings_order=[
        "HEADER_TABLE_SIZE",
        "MAX_CONCURRENT_STREAMS",
        "INITIAL_WINDOW_SIZE",
        "MAX_HEADER_LIST_SIZE"
    ],
    supported_signature_algorithms=[
        "ECDSAWithP256AndSHA256",
        "PSSWithSHA256",
        "PKCS1WithSHA256",
        "ECDSAWithP384AndSHA384",
        "PSSWithSHA384",
        "PKCS1WithSHA384",
        "PSSWithSHA512",
        "PKCS1WithSHA512",
    ],
    supported_versions=["GREASE", "1.3", "1.2"],
    key_share_curves=["GREASE", "X25519"],
    cert_compression_algo="brotli",
    pseudo_header_order=[
        ":method",
        ":authority",
        ":scheme",
        ":path"
    ],
    connection_flow=15663105,
    header_order=[
        "accept",
        "user-agent",
        "accept-encoding",
        "accept-language"
    ]
)

res = session.post(
    "https://www.example.com/",
    headers={
        "key1": "value1",
    },
    json={
        "key1": "key2"
    }
)

Pyinstaller / Pyarmor

If you want to pack the library with Pyinstaller or Pyarmor, make sure to add this to your command:

Linux - Ubuntu / x86:

--add-binary '{path_to_library}/tls_client/dependencies/tls-client-x86.so:tls_client/dependencies'

Linux Alpine / AMD64:

--add-binary '{path_to_library}/tls_client/dependencies/tls-client-amd64.so:tls_client/dependencies'

MacOS M1 and older:

--add-binary '{path_to_library}/tls_client/dependencies/tls-client-x86.dylib:tls_client/dependencies'

MacOS M2:

--add-binary '{path_to_library}/tls_client/dependencies/tls-client-arm64.dylib:tls_client/dependencies'

Windows:

--add-binary '{path_to_library}/tls_client/dependencies/tls-client-64.dll;tls_client/dependencies'

Acknowledgements

Big shout out to Bogdanfinn for open sourcing his tls-client in Golang. Also I wanted to keep the syntax as similar as possible to requests, as most people use it and are familiar with it!

python-tls-client's People

Contributors

biggie0344 avatar erict10 avatar florianregaz avatar nftbolt avatar qoft avatar qtlunya avatar seadhy avatar wizz1337 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-tls-client's Issues

"session.cookies.clear()" does not work properly

tls_client 0.1.6

Python 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tls_client
>>> session = tls_client.Session('chrome_108')
>>> session.get('https://httpbin.org/cookies/set/testcookie/12345')
<Response [302]>
>>> session.get('https://httpbin.org/cookies').json()
{'cookies': {'testcookie': '12345'}}
>>> session.cookies.get_dict()
{'testcookie': '12345'}
>>> session.cookies.clear()
>>> session.cookies.get_dict()
{}
>>> session.get('https://httpbin.org/cookies').json()
{'cookies': {'testcookie': '12345'}}
>>>

Encountered an error

internal buffer error : Memory allocation failed : growing buffer
fatal error: runtime: C malloc failed
goroutine 107184 [IO wait]:
internal/poll.runtime_pollWait(0x2a0f7b4b7b8, 0x72)
/Users/bogdan/.gvm/gos/go1.18.7/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc007196370?, 0x2c?, 0x0)
/Users/bogdan/.gvm/gos/go1.18.7/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.execIO(0xc0055fcf18, 0x7ffeabec51a0)
/Users/bogdan/.gvm/gos/go1.18.7/src/internal/poll/fd_windows.go:175 +0xe5
internal/poll.(*FD).Read(0xc0055fcf00, {0xc006314000, 0xc2e4, 0xc2e4})
/Users/bogdan/.gvm/gos/go1.18.7/src/internal/poll/fd_windows.go:441 +0x25f
net.(*netFD).Read(0xc0055fcf00, {0xc006314000?, 0xc005d451c0?, 0xc00631ffaa?})
/Users/bogdan/.gvm/gos/go1.18.7/src/net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc000006298, {0xc006314000?, 0x151b?, 0xc005d451c0?})
/Users/bogdan/.gvm/gos/go1.18.7/src/net/net.go:183 +0x45
github.com/bogdanfinn/utls.(*atLeastReader).Read(0xc0079ae690, {0xc006314000?, 0x0?, 0x2a0a3eaffff?})
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:791 +0x3d
bytes.(*Buffer).ReadFrom(0xc005bb02b0, {0x7ffeabf24560, 0xc0079ae690})
/Users/bogdan/.gvm/gos/go1.18.7/src/bytes/buffer.go:204 +0x98
github.com/bogdanfinn/utls.(*Conn).readFromUntil(0xc005bb0000, {0x2a0f6764ff8?, 0xc000006298}, 0x33f?)
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:813 +0xe5
github.com/bogdanfinn/utls.(*Conn).readRecordOrCCS(0xc005bb0000, 0x0)
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:620 +0x116
github.com/bogdanfinn/utls.(*Conn).readRecord(...)
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:586
github.com/bogdanfinn/utls.(*Conn).Read(0xc005bb0000, {0xc0042aa000, 0x1000, 0xc001704c60?})
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/conn.go:1300 +0x16f
bufio.(*Reader).Read(0xc001704ba0, {0xc005839c38, 0x9, 0xc006049d68?})
/Users/bogdan/.gvm/gos/go1.18.7/src/bufio/bufio.go:236 +0x1b4
io.ReadAtLeast({0x7ffeabf24020, 0xc001704ba0}, {0xc005839c38, 0x9, 0x9}, 0x9)
/Users/bogdan/.gvm/gos/go1.18.7/src/io/io.go:331 +0x9a
io.ReadFull(...)
/Users/bogdan/.gvm/gos/go1.18.7/src/io/io.go:350
github.com/bogdanfinn/fhttp/http2.readFrameHeader({0xc005839c38?, 0x9?, 0xc005c6d4d0?}, {0x7ffeabf24020?, 0xc001704ba0?})
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/http2/frame.go:238 +0x6e
github.com/bogdanfinn/fhttp/http2.(*Framer).ReadFrame(0xc005839c00)
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/http2/frame.go:494 +0x95
github.com/bogdanfinn/fhttp/http2.(*clientConnReadLoop).run(0xc000abbfa0)
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/http2/transport.go:2020 +0x165
github.com/bogdanfinn/fhttp/http2.(*ClientConn).readLoop(0xc005b56340)
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/http2/transport.go:1942 +0x79
created by github.com/bogdanfinn/fhttp/http2.(*Transport).newClientConn
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/http2/transport.go:821 +0xbea

goroutine 106957 [select]:
github.com/bogdanfinn/fhttp.setRequestCancel.func4()
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/client.go:398 +0x8b
created by github.com/bogdanfinn/fhttp.setRequestCancel
/Users/bogdan/.gvm/pkgsets/go1.18.7/global/pkg/mod/github.com/bogdanfinn/[email protected]/client.go:397 +0x439

Encoding error.

There is this 1 specific site where I am trying to parse an csrf token. But I cant cause tls client is currently not decoding the requests right.
How can I fix that? (in the normal requests libary it works fine)

image

form data is still sent wrong.

Since version 0.1.6 > form data isn't sent right.
Has anyone else experienced this kinda issue?

with the version 0.1.6 I can pass the login on a website, where form data is being sent, while when I am on a higher version than 0.1.6 I do get 403 from the page thrown.

"Killed" error

Hello!
I've been using the client for a while now and it works great!
From time to time, if a script has been using the tls client for a while, the process will stop itself, giving the Killed error.
I guess it has to do probably with the number of files opened, f.e. something in the memory has not been freed correctly.
Do you guys have an idea on how to fix it? Or at least how to manage it by my side?

Thank you!

memory leak

Hello, I saw that the memory leak was fixed by a recent pull request, but I still get an error from it.
Any idea on how to avoid it? thx

not support response.content

I especially want to support response.content. Like the requests library he supports content in order to store images

tls: error decoding message

Hi,
im getting this error while using ur package.
This is my session:

s = tls_client.Session(
            ja3_string="771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21,29-23-24,0",
            pseudo_header_order=[
                ":method",
                ":authority",
                ":scheme",
                ":path"],
            supported_versions=["GREASE", "1.3", "1.2"],
            cert_compression_algo="brotli",
            key_share_curves=["GREASE", "X25519"],
            
 )

and this is error text:
failed to do request: Get "https://discord.com/api/v9/experiments": remote error: tls: error decoding message

Support streaming response

Is accepting a streaming response already implemented ?
It would be nice being able to use something like:

response = requests.get('https://example.com/stream', stream=True)
for chunk in response.iter_content(chunk_size=1024)
    print(chunk)

TLS: Illegal parameter

Getting this error with custom TLS as a session
failed to do request: Get "MY_URL_HERE": remote error: tls: illegal parameter
didn't change nothing from the example

How can I to get custom settings?

I want to create sessions using a 'custom' settings as you gave in the examples. I also want to do this using multiple custom settings, but I have no idea where to find these values. Can you give an idea about how we can find/create these values?

ARM support

  File "/home/kot/.local/share/virtualenvs/chatGPT-discord-bot-BNM29DVl/lib/python3.11/site-packages/revChatGPT/revChatGPT.py", line 4, in <module>
    import tls_client
  File "/home/kot/.local/share/virtualenvs/chatGPT-discord-bot-BNM29DVl/lib/python3.11/site-packages/tls_client/__init__.py", line 15, in <module>
    from .sessions import Session
  File "/home/kot/.local/share/virtualenvs/chatGPT-discord-bot-BNM29DVl/lib/python3.11/site-packages/tls_client/sessions.py", line 1, in <module>
    from .cffi import request
  File "/home/kot/.local/share/virtualenvs/chatGPT-discord-bot-BNM29DVl/lib/python3.11/site-packages/tls_client/cffi.py", line 15, in <module>
    library = ctypes.cdll.LoadLibrary(f'{root_dir}/dependencies/tls-client{file_ext}')
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kot/.asdf/installs/python/3.11.0/lib/python3.11/ctypes/__init__.py", line 454, in LoadLibrary
    return self._dlltype(name)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/kot/.asdf/installs/python/3.11.0/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /home/kot/.local/share/virtualenvs/chatGPT-discord-bot-BNM29DVl/lib/python3.11/site-packages/tls_client/dependencies/tls-client-amd64.so: cannot open shared object file: No such file or directory

arch is aarch64, amd64 will not work here.

Segmentation fault on Mac (Catalina 10.15.7)

... the error occurs directly at where the library is loaded:
library = ctypes.cdll.LoadLibrary(f'{root_dir}/dependencies/tls-client{file_ext}')

File exists and filepath is ok.
Obtain: Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

Selected file is: tls-client-x86.dylib

Running with:
Python 3.10.8

Unknown problem Stream error

Hi! Help me please, sometimes getting this error

ERROR:main:Exception while handling an update:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/telegram/ext/_application.py", line 945, in process_update
await coroutine
File "/usr/local/lib/python3.10/site-packages/telegram/ext/_handler.py", line 135, in handle_update
return await self.callback(update, context)
File "/Users/pleshkov/Documents/pets/chatgpt_telegram_bot/bot.py", line 38, in prompt_handle
r = chatbot.ask(prompt, conversation_id=None, parent_id=None) # You can specify custom conversation and parent ids. Otherwise it uses the saved conversation (yes. conversations are automatically saved)
File "/usr/local/lib/python3.10/site-packages/revChatGPT/ChatGPT.py", line 128, in ask
response = self.session.post(
File "/usr/local/lib/python3.10/site-packages/tls_client/sessions.py", line 389, in post
return self.execute_request(method="POST", url=url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.10/site-packages/tls_client/sessions.py", line 346, in execute_request
raise TLSClientExeption(response_object["body"])
tls_client.exceptions.TLSClientExeption: stream error: stream ID 1; INTERNAL_ERROR

importing tls_client fails with "dependency missing error", even though it is present

Hey, I am facing next error, when trying to import tls_client:

>>> import tls_client
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/tls_client/__init__.py", line 15, in <module>
    from .sessions import Session
  File "/usr/local/lib/python3.9/site-packages/tls_client/sessions.py", line 1, in <module>
    from .cffi import request
  File "/usr/local/lib/python3.9/site-packages/tls_client/cffi.py", line 15, in <module>
    library = ctypes.cdll.LoadLibrary(f'{root_dir}/dependencies/tls-client{file_ext}')
  File "/usr/local/lib/python3.9/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/usr/local/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.9/site-packages/tls_client/dependencies/tls-client-amd64.so: cannot open shared object file: No such file or directory

I checked, and I definitely have this file downloaded(I can even open it with normal with open(...) statement.

Also, ran ctypes library separately:

>>> import ctypes
>>> library = ctypes.cdll.LoadLibrary('/usr/local/lib/python3.9/site-packages/tls_client/dependencies/tls-client-amd64.so')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/usr/local/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/local/lib/python3.9/site-packages/tls_client/dependencies/tls-client-amd64.so: cannot open shared object file: No such file or directory

same issue. Any ideas on how to fix this? I am running this code on docker python:3.9 image on m1 macbook.

when I run it not in docker, everything works perfectly.

can't compile with Nuitka

It gives this error:
FileNotFoundError: Could not find module 'C:\Users\TrixTM\Desktop\GENV102\GENSER1.DIS\tls_client\dependencies\tls-client.dll' (or one of its dependencies). Try using the full path with constructor syntax.

How can i include the dll?

Not working Rapberry Pi 2B

I cant get tls-client running on my Raspberry Pi 2B, which is running ARMv7 32 bit. When running the client, I get OSError: /home/*****/.local/lib/python3.9/site-packages/tls_client/dependencies/tls-client-amd64.so: cannot open shared object file: No such file or directory. Any solution for this?

tls_client_x86.so 64 bit binary?

file tls-client-x86.so
tls-client-x86.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=207118fca7a105e464dddc1f7e95c9dbb2044eac, not stripped

so useless to distribute this. but i need a working 32bit binary.

i know 32bit is dead but some old jardware is still usefull

Async

Hi, is it possible to use it asynchronously?

signature_algorithms issue

i want to add
ECDSA with SHA-256
RSASSA-PSS with SHA-256
RSASSA-PKCS1-v1_5 with SHA-256
ECDSA with SHA-384
RSASSA-PSS with SHA-384
RSASSA-PKCS1-v1_5 with SHA-384
RSASSA-PSS with SHA-512
RSASSA-PKCS1-v1_5 with SHA-512
,

i tried

 supported_signature_algorithms=[
  "ECDSAWithSHA256",
  "SHA256",
  "SHA384",
  "SHA512",
  "SHA256WithRSAEncryption",
  "SHA384WithRSAEncryption",
  "SHA512WithRSAEncryption",
  "ECDSAWithSHA384"
    ],

https://www.cryptosys.net/pki/manpki/pki_SupportedAlgorithms.html

but i face this error

ECDSAWithSHA256 is not a valid supportedSignatureAlgorithm

Same fingerprint

I have tested multiple version of chrome, but i see same fingerprint.

using the following website: https://tls.browserleaks.com/json

test chrome_108: "ja3_hash":"53ff64ddf993ca882b70e1c82af5da49"
test chrome_105: "ja3_hash":"53ff64ddf993ca882b70e1c82af5da49"
test chrome_103: "ja3_hash":"53ff64ddf993ca882b70e1c82af5da49"

the purpose of this library was to change fingerprint and replicate real browser. is there anything wrong i am doing ?

self.client = tls_client.Session(
            client_identifier=self.chrome_version
        )

resp = self.client.get('https://tls.browserleaks.com/json',proxy=self.settings['proxies'],headers=self.headers)

print(resp.text)

32-bit support

Error in 32bit Python

Traceback (most recent call last):
  File "E:\python\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "E:\python\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "E:\python\lib\site-packages\revChatGPT\__main__.py", line 1, in <module>
    from revChatGPT.revChatGPT import Chatbot
  File "E:\python\lib\site-packages\revChatGPT\revChatGPT.py", line 4, in <module>
    import tls_client
  File "E:\python\lib\site-packages\tls_client\__init__.py", line 15, in <module>
    from .sessions import Session
  File "E:\python\lib\site-packages\tls_client\sessions.py", line 1, in <module>
    from .cffi import request
  File "E:\python\lib\site-packages\tls_client\cffi.py", line 15, in <module>
    library = ctypes.cdll.LoadLibrary(f'{root_dir}/dependencies/tls-client{file_ext}')
  File "E:\python\lib\ctypes\__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "E:\python\lib\ctypes\__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 Is not a valid Win32 application.

acheong08/ChatGPT#74

Sources of binaries?, Architecture-sensitive installation?

This bundles binary libraries.

But what is their source?

Please bundle their source (and build recipe), or link to their source.

Also, installing this python module installs binary libraries for all architectures, not just for the one where this module is installed. This is useless bloating. Please make the installation procedure architecture-sensitive by only installing libraries for the current architecture.

I have made β†—β€Šan Arch Linux package, where I have tackled the second part by a workaround by deleting the libraries not suiting.

Regards!

Problem Setting Custom Cookies

I am trying to set up custom cookies with the client but I am running into some hiccups.

Below is the code I am running

import tls_client


session = tls_client.Session(
            ja3_string="771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513,29-23-24,0",
            h2_settings={
                "HEADER_TABLE_SIZE": 65536,
                "MAX_CONCURRENT_STREAMS": 1000,
                "INITIAL_WINDOW_SIZE": 6291456,
                "MAX_HEADER_LIST_SIZE": 262144
            },
            h2_settings_order=[
                "HEADER_TABLE_SIZE",
                "MAX_CONCURRENT_STREAMS",
                "INITIAL_WINDOW_SIZE",
                "MAX_HEADER_LIST_SIZE"
            ],
            supported_signature_algorithms=[
                "ECDSAWithP256AndSHA256",
                "PSSWithSHA256",
                "PKCS1WithSHA256",
                "ECDSAWithP384AndSHA384",
                "PSSWithSHA384",
                "PKCS1WithSHA384",
                "PSSWithSHA512",
                "PKCS1WithSHA512",
            ],
            supported_versions=["GREASE", "1.3", "1.2"],
            key_share_curves=["GREASE", "X25519"],
            cert_compression_algo="brotli",
            pseudo_header_order=[
                ":method",
                ":authority",
                ":scheme",
                ":path"
            ],
            connection_flow=15663105,
            header_order=[
                "content-length",
                "cache-control",
                "sec-ch-ua",
                "sec-ch-ua-mobile",
                "sec-ch-ua-platform",
                "origin",
                "dnt",
                "upgrade-insecure-requests",
                "content-type",
                "user-agent",
                "accept",
                "sec-fetch-site",
                "sec-fetch-mode",
                "sec-fetch-user",
                "sec-fetch-dest",
                "referer",
                "accept-encoding",
                "accept-language",
            ]
        )

headers = {'sec-ch-ua': '"Google Chrome";v="107", "Chromium";v="107", "Not=A?Brand";v="24"',
                    'sec-ch-ua-mobile': '?0',
                    'sec-ch-ua-platform': '"Windows"',
                    'dnt': '1',
                    'upgrade-insecure-requests': '1',
                    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36',
                    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
                    'sec-fetch-site': 'none',
                    'sec-fetch-mode': 'navigate',
                    'sec-fetch-user': '?1',
                    'sec-fetch-dest': 'document',
                    'accept-encoding': 'gzip, deflate, br',
                    'accept-language': 'en-US,en;q=0.9'}

session.cookies.set('one', 'two')

a = session.get('https://httpbin.org/cookies', headers=headers)
print(a.text)

This code outputs
{ "cookies": {} }

However, when I do not send any defined headers like this

a = session.get('https://httpbin.org/cookies')
print(a.text)

What is outputted is the following:
{ "cookies": { "one": "two" } }

I would also like to mention that in the above example, if you were to capture the request with Charles, the headers being sent are the following:

user-agent: tls-client/0.1.1
accept: */*
accept-encoding: gzip, deflate, br

But if you were to include headers that say, did not have a set user agent like the following:

headers = {'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9'}

session.cookies.set('one', 'two')

a = session.get('https://httpbin.org/cookies', headers=headers)
print(a.text)

the following is outputted:

accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
user-agent: Go-http-client/2.0

There is a user-agent inconsistency with no headers being set and with headers being sent that do not have a user agent set. Also note that the header order is not conserved either. I have user-agent placed before accept, but it is outputted here at the bottom. I get it does that because I didn't define the user-agent header myself, but if it is being sent anyways shouldn't it still follow the header order defined?

Question about Custom Example

I am wondering if it is required to change the other settings on the custom example shown on the readme when the ja3 string is changed or if they can be kept the same. I'm not quite sure keeping them the same make the requests look sus or not.

If i need to change the values depending on the ja3 string, how would i be able to do this?

No Respons or Dead Lock while sending request in sub-process

Env

MacOS M1 chip
Python3.9, using the tls-client.dylib

Phenomenon

  1. When I call the dylib.request method in single main process, everything works well.
  2. But when I call the dylib.request in the sub-process created by Celery, the request call never return value. it just blocked forever.

Question

here is the original question link

I want to know how I could inspect deeper for this problem, since I just use the PyCharm Debugger, but which get stucked in the function entry point. I got no idea what the next step I should do.

Could you give me some advices ? Thanks a lot

Increase request timeout

Hello, I'm currently facing this issue a lot: tls_client.exceptions.TLSClientExeption: failed to do request: Get "https://discord.com/api/v9/invites/T6cNBdYw9W": read tcp 192.168.178.24:37874->38.20.233.249:26023: i/o timeout and so i want to increase the timeout to lets say 20 sec but i don't know, how i would be able to do this with this libary.

Bad request

Keeps sending bad requests, when replacing my tls_session with requests, it works perfectly fine. (custom client), ja3 correctly set, headers and json correctly set,...

tls_client.exceptions.TLSClientExeption: failed to do request||| unknown extension with id 22 provided

tls_client.exceptions.TLSClientExeption: failed to do request||| unknown extension with id 22 provided
i 'got this error with tls client
for session.post
session = tls_client.Session(
#client_identifier="chrome110",
ja3_string=self.getJA3(headers),
h2_settings={"HEADER_TABLE_SIZE": 65536,"MAX_CONCURRENT_STREAMS": 1000,"INITIAL_WINDOW_SIZE": 6291456,"MAX_HEADER_LIST_SIZE": 262144},
h2_settings_order=["HEADER_TABLE_SIZE","MAX_CONCURRENT_STREAMS","INITIAL_WINDOW_SIZE","MAX_HEADER_LIST_SIZE"],
supported_signature_algorithms=["ECDSAWithP256AndSHA256","PSSWithSHA256","PKCS1WithSHA256","ECDSAWithP384AndSHA384","PSSWithSHA384","PKCS1WithSHA384","PSSWithSHA512","PKCS1WithSHA512",],
supported_versions=["GREASE", "1.3", "1.2"],
key_share_curves=["GREASE", "X25519"],
cert_compression_algo="brotli",
connection_flow=15663105
)
response = session.post(
url="",
headers=headers,
proxy="")

not working timeout argument

not possible to add timeout argument for links giving timeouts?
like the headers argument ?

good job! im testing different github's TLS clients and works fine keep working!

cookie issues

I am trying to reproduce a get request stream and I noticed that duplicate cookies are being sent, I don't know if I did something wrong, but every time I find the usual cookies with key, value and domain two times

failed to do request: Get "https://tls.peet.ws/api/all": remote error: tls: unexpected message

tls_client.exceptions.TLSClientExeption: failed to do request: Get "https://www.baidu.com": unknown extension with id 24 provided

config = {'ja3_string': '771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21-41,29-23-24,0', 'h2_settings': {'HEADER_TABLE_SIZE': 65536, 'MAX_CONCURRENT_STREAMS': 1000, 'INITIAL_WINDOW_SIZE': 6291456, 'MAX_HEADER_LIST_SIZE': 262144}, 'h2_settings_order': ['HEADER_TABLE_SIZE', 'MAX_CONCURRENT_STREAMS', 'INITIAL_WINDOW_SIZE', 'MAX_HEADER_LIST_SIZE'], 'supported_signature_algorithms': ['ECDSAWithP256AndSHA256', 'PSSWithSHA256', 'PKCS1WithSHA256', 'ECDSAWithP384AndSHA384', 'PSSWithSHA384', 'PKCS1WithSHA384', 'PSSWithSHA512', 'PKCS1WithSHA512', 'PKCS1WithSHA1'], 'supported_delegated_credentials_algorithms': None, 'supported_versions': ['GREASE', '1.3', '1.2'], 'key_share_curves': ['GREASE', 'X25519'], 'cert_compression_algo': 'brotli', 'pseudo_header_order': [':method', ':authority', ':scheme', ':path'], 'connection_flow': 15663105, 'priority_frames': None, 'header_order': ['cache-control', 'sec-ch-ua', 'sec-ch-ua-mobile', 'sec-ch-ua-platform', 'upgrade-insecure-requests', 'user-agent', 'accept', 'sec-fetch-site', 'sec-fetch-mode', 'sec-fetch-user', 'sec-fetch-dest', 'accept-encoding', 'accept-language'], 'header_priority': {'weight': 255, 'streamDep': 0, 'exclusive': True}}
session = tls_client.Session(**config)
headers = {
"cache-control": "max-age=0",
"sec-ch-ua": '".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"',
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": '"Windows"',
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3;q=0.9",
"sec-fetch-site": "none",
"sec-fetch-mode": "navigate",
"sec-fetch-user": "?1",
"sec-fetch-dest": "document",
"accept-encoding": "gzip, deflate, br",
"accept-language": "zh-CN,zh;q=0.9"
}
session.update(headers)
response = session.get(url="https://www.baidu.com")
print(response.text)

Custom TLS Fingerprints aren't working

Hi i just wanted to submit, that custom fingerprints aren't working.
With presets everything is working fine.

I am on MacOS with M1 and on the recent version of your package.

I thought I did something wrong but even with your custom example I get the following error:

File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tls_client/sessions.py", line 334, in get
return self.execute_request(method="GET", url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/tls_client/sessions.py", line 317, in execute_request
raise TLSClientExeption(response_object["body"])
tls_client.exceptions.TLSClientExeption: failed to do request: Get "https://tls.peet.ws/api/all": unknown extension with id 27 provided

cannot unmarshal array into Go struct field RequestInput.headers of type string

Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/home/infinimonster/Documents/Privat/Python/discordphoneverify/new version/main.py", line 145, in verify
checktoken()
File "/home/infinimonster/Documents/Privat/Python/discordphoneverify/new version/main.py", line 135, in checktoken
response = session.get(url="https://discord.com/api/v9/users/@me", headers=HEADERS, proxy=proxyauth if proxytype != "" else None)
File "/home/infinimonster/.local/lib/python3.10/site-packages/tls_client/sessions.py", line 364, in get
return self.execute_request(method="GET", url=url, **kwargs)
File "/home/infinimonster/.local/lib/python3.10/site-packages/tls_client/sessions.py", line 347, in execute_request
raise TLSClientExeption(response_object["body"])
tls_client.exceptions.TLSClientExeption: json: cannot unmarshal array into Go struct field RequestInput.headers of type string

memory leak after executing request

My program runs multiple threads of a function performing a get request, and although the number of threads remains constant throughout the program, the memory usage and object count gradually increases over time until no free memory is left on the system. After doing some digging I found that base requests library has some memory leaking issues with the response object that can be fixed by using .close() method on the session and response like so:

   r = session.get(url=url, headers=headers)
   r.close()
   session.close()

I can't seem to find a solution to solving this issue within this library.

Sample code:

def func1():
#populate list


while True:
        threads = []
        for item in list:
            t = threading.Thread(target=func2, args=(item, v, q))
            threads.append(t)
            t.start()
        for thread in threads:
            thread.join()
            print(f"Retrying... {retry}")
 
        retry += 1


def func2(i, v, q):
  headers = {'':''}
  
  session = Session(client_identifier='chrome_105')
  r = session.get('url', headers=headers)
  response = json.loads(r.text)
 #do stuff with response
 return

Sending requests from under worker celery

Hello.
Faced a problem with the tls-client when launched through celery.
Without Celery, all requests go well.
But if you run the celery task, then the worker hangs on sending the request, regardless of the method (get, post, etc.)

code example:

session = tls_client.Session(client_identifier="chrome_105")
response = session.get(url='https://example.com')

worker hangs on the second line.
during debugging, I found that the hang occurs on the line:

response = request(dumps(request_payload).encode('utf-8'))

it gets to her just fine. but it hangs on it.
without error output.

have to force stop the worker

Please let me know if there is a way to fix this issue. We are very dependent on celery in the current project.

Library versions:

celery==5.2.7
tls-client==0.1.5

can't send data

I can't send data like this: r= session.post(f"https://example.com/", headers={....}, data=data}
Only json=json works, but i need it to work with data=data too.

failed to do request: Get "https://tls.peet.ws/api/all": remote error: tls: unexpected message

success ja3

771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21,29-23-24,0

error ja3

771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21-41,29-23-24,0

test code

config = {'ja3_string': '771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21,29-23-24,0', 'h2_settings': {'HEADER_TABLE_SIZE': 65536, 'MAX_CONCURRENT_STREAMS': 1000, 'INITIAL_WINDOW_SIZE': 6291456, 'MAX_HEADER_LIST_SIZE': 262144}, 'h2_settings_order': ['HEADER_TABLE_SIZE', 'MAX_CONCURRENT_STREAMS', 'INITIAL_WINDOW_SIZE', 'MAX_HEADER_LIST_SIZE'], 'supported_signature_algorithms': ['ECDSAWithP256AndSHA256', 'PSSWithSHA256', 'PKCS1WithSHA256', 'ECDSAWithP384AndSHA384', 'PSSWithSHA384', 'PKCS1WithSHA384', 'PSSWithSHA512', 'PKCS1WithSHA512', 'PKCS1WithSHA1'], 'supported_delegated_credentials_algorithms': None, 'supported_versions': ['GREASE', '1.3', '1.2'], 'key_share_curves': ['GREASE', 'X25519'], 'cert_compression_algo': 'brotli', 'pseudo_header_order': [':method', ':authority', ':scheme', ':path'], 'connection_flow': 15663105, 'priority_frames': None, 'header_order': ['cache-control', 'sec-ch-ua', 'sec-ch-ua-mobile', 'sec-ch-ua-platform', 'upgrade-insecure-requests', 'user-agent', 'accept', 'sec-fetch-site', 'sec-fetch-mode', 'sec-fetch-user', 'sec-fetch-dest', 'accept-encoding', 'accept-language'], 'header_priority': {'weight': 255, 'streamDep': 0, 'exclusive': True}}
session = tls_client.Session(**config)
headers = {
	"cache-control": "max-age=0",
	"sec-ch-ua": '".Not/A)Brand";v="99", "Google Chrome";v="103", "Chromium";v="103"',
	"sec-ch-ua-mobile": "?0",
	"sec-ch-ua-platform": '"Windows"',
	"upgrade-insecure-requests": "1",
	"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36",
	"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
	"sec-fetch-site": "none",
	"sec-fetch-mode": "navigate",
	"sec-fetch-user": "?1",
	"sec-fetch-dest": "document",
	# "accept-encoding": "gzip, deflate, br",
	"accept-encoding": "deflate",
	"accept-language": "zh-CN,zh;q=0.9"
}
session.headers.update(headers)
response = session.get(url="https://tls.peet.ws/api/all")
print(response.text)

Proxy not work

I am afraid but it seems like ip is not changing:

import tls_client

s = tls_client.Session(
    client_identifier="chrome_105"
)
res = s.get("http://httpbin.org/ip", proxy="http://6j80wk0dmeef:[email protected]:7492")

print(res.text)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.