pansidong(盘丝洞)is a proxy ips' store service. pansidong is another name of Cave of the Silken Web
. In the movie, There are lots of little spiders in pansidong. Sometimes we want to scrapy pages using web spiders. But it is a game between crawler and anti crawler.Proxy ips are the best solution to do with it.
Fortunatly,there are many free proxy ip provider on the internet. pansidong try to provide a store and standard api to help crawler developers focusing on developing instead of collection free ips.
You can set up pansidong as a microservice and allocate every request a single proxy ip.
I found this one (https://github.com/henson/proxypool), pretty cool !
This software is released under the Apache 2.0 license.