Puppeteer Being Detected about tiktok-signature

[mitchhymel]

My guess is it's something with Puppeteer - maybe Tiktok figured out how to detect it. If you manually run through what the signer is doing, you can generate urls that are signed. But for some reason this doesn't work in the chromium instance Puppeteer launches.

For example

• go to https://www.tiktok.com/trending?lang=en
• in chrome dev tools console, run the following

1. var t = {}
2. webpackJsonp.filter(x => typeof x[1]['duD4'] === "function")[0][1].duD4(null, t)
3. var url = 'https://m.tiktok.com/api/challenge/detail/?verifyFP=&challengeName=meme&language=en'
4. var token = t.sign({url: url})
5. var signedUrl = url + '&_signature=' + token
6. signedUrl

This will print out a url that you can go to and see a non empty response

Edit: Updating sign call to use object instead of just the string

[mitchhymel]

I made a mistake with step 4 in my previous comment (and updated it to correct value)
Replace

var token = t.sign(url)

with

var token = t.sign({url: url})

and that should work. I tested with "https://m.tiktok.com/api/user/detail/?uniqueId=who&language=en&verifyFp="

[carcabot]

Hi David,

It seems that TikTok uses this structure for getting user data.

https://t.tiktok.com/api/user/detail/?uniqueId=worldeconomicforum&secUid=&verifyFp=

[davidteather]

Do you mind telling me where you find these api endpoints for tiktok?
I've just been using google chrome's network inspector and my browser never hit that end point. Mine just seemed to hit the ones that I specified in my post.

Here's an image of a request it makes for user data.

Thanks!

Edit: Just tried that end point and it doesn't seem to work for me.

[davidteather]

Hey there your workaround seems to only work for that url you sent. I've tested your theory on a few other endpoints which are the music & getting a user object endpoint from my initial post that don't seem to work in the google chrome console.

[davidteather]

Thanks!
It's strange that TikTok only detects puppeteer on specific endpoints.

[mitchhymel]

That's just my theory, and I'm not exactly an expert here, just wanted to share what I was seeing.

[davidteather]

Yeah that seems to be the issue. I'm also not an expert, but I'm fairly certain that's what's going on.

I'm basically porting this project to a python package, and I'll look into avoiding detection. If I find a JavaScript equivalent I'll update this issue.

[mitchhymel]

Swapping puppeteer out for playwright seems to be a fix. Though I don't know if there is an equivalent of the puppeteer-extra-plugin-stealth for playwright (or if it's still needed). Either way I'll send out a PR sometime this week.

[carcabot]

Hi guys,

It's good to see that people are interested for this repo and trying to fix things.
I have just tested some links with the current code and everything worked very well.

For example I've tried generating signature for:

https://m.tiktok.com/api/user/detail/?uniqueId=who&language=en&verifyFp=
https://m.tiktok.com/api/challenge/detail/?verifyFP=&challengeName=meme&language=en
https://t.tiktok.com/api/user/detail/?uniqueId=fotbal_romania_memes&secUid=&verifyFp=

All of them are working very well and I can't see any issue with the current code. Probably there is some other reason for empty result.

All tests used trending.py file included in repo. The user-agent used should match when requesting for signature and when requesting for response.

I need some more information to find out where exactly is the issue using puppeteer.

Greetings.

[Flo354]

Hello there,

We saw this issue too. It seems that tiktok started to use the parameter verifyFp.
Is it possible for you @carcabot to look at this verifyFp parameter to check if it's possible to get it the way you do to generate the signature?

I know that the purpose of this repo is to "generate the signature" and not generate the verifyFp. But it could be a nice addition for us.

Thanks,