fake-useragent / fake-useragent

Up-to-date simple useragent faker with real world database

fake-useragent

Up-to-date simple useragent faker with real world database.

Features

• Data is pre-downloaded from https://user-agents.net/ and the data is part of the package
• The data consists of the current browser versions or one version lower
• Retrieves user-agent strings locally (both desktop and mobile UAs)
• Retrieve user-agent Python dictionary
• Supports Python 3.x

Installation

pip install fake-useragent

Or if you have multiple Python / pip versions installed, use pip3:

pip3 install fake-useragent

Usage

Simple usage examples below, see also next chapters in this readme for more advanced usages:

from fake_useragent import UserAgent
ua = UserAgent()

# Get a random browser user-agent string
print(ua.random)

# Or get user-agent string from a specific browser
print(ua.chrome)
# Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36
print(ua.google)
# Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.13 (KHTML, like Gecko) Chrome/24.0.1290.1 Safari/537.13
print(ua['google chrome'])
# Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
print(ua.firefox)
# Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0
print(ua.ff)
# Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0
print(ua.safari)
# Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15

Additional usage

Additional features that fake-useragent now offers since v1.2.0.

If you want to specify your own browser list, you can do that via the browsers argument (default is: ["chrome", "edge", "firefox", "safari"]).
This example will only return random user-agents from Edge and Chrome:

from fake_useragent import UserAgent
ua = UserAgent(browsers=['edge', 'chrome'])
ua.random

Note: Fakeuser-agent knowns about: Chrome, Edge, Firefox and Safari. Other browsers are not popular enough and aren't part of our dataset we use.

---

If you want to specify your own operating systems, you can do that via the os argument (default is: ["windows", "macos", "linux"]).
In this example you will only get Linux user-agents back:

from fake_useragent import UserAgent
ua = UserAgent(os='linux')
ua.random

---

You can also specify the type of platforms you want to use, you can do that via the platforms argument (default is ["pc", "mobile", "tablet"]. This example will only return random user-agents from a mobile device:

from fake_useragent import UserAgent
ua = UserAgent(platforms='mobile')
ua.random

---

If you want to return more recent user-agent strings, you can play with the min_version argument (default is: 0.0, meaning all user agents will match).
In this example you get only user agents that have a minimum version of 120.0:

from fake_useragent import UserAgent
ua = UserAgent(min_version=120.0)
ua.random

---

For backwards compatibility, a minimum usage percentage can still be specified with the min_percentage argument. However, the current list of user agents does not contain this statistic. Therefore all of the user-agents will match.

---

Hint: Of-course you can combine all those arguments to you liking!

User-agent Python Dictionary

Since version 1.3.0 we now also offer you the following "get" properties which return the whole Python dictionary of the UA, instead of only the user-agent string:

Warning Raw JSON objects (in a Python dictionaries) are returned "as is". Meaning, this data structure could change in the future!

Be aware that these "get" properties below might not return the same key/value pairs in the future. Use ua.random or alike as mentioned above, if you want to use a stable interface.

from fake_useragent import UserAgent
ua = UserAgent()

# Random user-agent dictionary (object)
ua.getRandom
# {'percent': 0.8, 'useragent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36 Edg/116.0.1938.76', 'system': 'Edge 116.0 Win10', 'browser': 'edge', 'version': 116.0, 'os': 'win10'}

# More get properties:
ua.getFirefox
# {'percent': 0.3, 'useragent': 'Mozilla/5.0 (Windows NT 10.0; rv:109.0) Gecko/20100101 Firefox/118.0', 'system': 'Firefox 118.0 Win10', 'browser': 'firefox', 'version': 118.0, 'os': 'win10'}
ua.getChrome
ua.getSafari
ua.getEdge

# And a method with an argument.
# This is exactly the same as using: ua.getFirefox
ua.getBrowser('firefox')

Notes

You can override the fallback string using the fallback parameter, in very rare cases something failed:

import fake_useragent

ua = fake_useragent.UserAgent(fallback='your favorite Browser')
# in case if something went wrong, one more time it is REALLY!!! rare case
ua.random == 'your favorite Browser'

If you will try to get unknown browser:

from fake_useragent import UserAgent
ua = UserAgent()
print(ua.unknown)
#Error occurred during getting browser: randm, but was suppressed with fallback.
#Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0

If you need to safe some attributes from overriding them in UserAgent by __getattr__ method use safe_attrs you can pass there attributes names. At least this will prevent you from raising FakeUserAgentError when attribute not found.

For example, when using fakeuseragent with injections <https://github.com/tailhook/injections> you need to:

import fake_useragent

ua = fake_useragent.UserAgent(safe_attrs=('__injections__',))

Please, do not use if you don't understand why you need this. This is magic for rarely extreme case.

Experiencing issues?

Make sure that you using latest version!

pip install --upgrade fake-useragent

Or if that isn't working, try to install the latest package version like this (1.5.1 is an example, check what the latest version is on PyPi):

pip install fake-useragent==1.5.1

Check version via the Python console:

import fake_useragent

print(fake_useragent.VERSION)

And you are always welcome to post issues.

Please do not forget to mention the version that you are using.

For Developers

Since GitHub Actions is unable to reach willshouse.com and has Cloudflare protection. We can run the script below to automatically scrape the user-agent strings from the external data source. The script will copy the JSONlines file to the src/fake_useragent/data directory. Execute:

./update_data_file.sh

The data JSON file is part of the Python package, see pyproject.toml. Read more about Data files support.

Python Virtual Environment

We encourage to use Python virtual environment before installing Pip packages, like so:

python -m virtualenv env
source env/bin/activate

Tests

pip install -r requirements.txt
tox

Linting

To fix imports using ruff:

pip install -r requirements.txt
ruff check --select="I" --fix .

Fix black code formatting errors:

pip install -r requirements.txt
black .

Note: When ruff v1.0 releases, we most likely move fully towards ruff instead of black.

Changelog

• 1.5.1 March 16, 2024

  • Remove trailing spaces in user agent strings

• 1.5.0 March 8, 2024

  • Migrated to new user-agent data source (thanks @BoudewijnZwart), backwards compatible API.
  • Update all pip package dependencies to latest stable versions

• 1.4.0 November 24, 2023

  • Update all PIP packages
  • Support Python 3.12 (thanks @vladkens)
  • Fix package conflict in cache scraper
  • Improve ruff CLI calls

• 1.3.0 October 2, 2023

  • Introducing new ua.getRandom, ua.getFirefox, ua.getChrome, ua.getSafari. And a generic method: ua.getBrowser(..) (eg. getBrowser('firefox'))
    • These new properties above allows you to retrieve the whole raw Python dictionary, instead of only the UA string.
    • These properties might return different key/values pairs in the future!
  • Fix the os argument 'windows' to check for both win10and win7 values (previously only checking on win10), thus returning more UAs
  • Improved user-agent scraper (now also containing Safari browser again)
  • Updated browsers.json data file

• 1.2.1 August 2, 2023

  • Small improvements in the min_percentage check
  • Update all Pip package dependencies

• 1.2.0 August 2, 2023

  • Updated browser useragent data
  • Allow filters on browser, OS and usage percentage
  • Update the cache scraper to scape the new data source for user-agent strings
  • Adapted the code to work with the new JSON data format
  • Parameter use_external_data=True and verify_ssl are removed. If you use those parameters, just remove it in your code!

• 1.1.3 March 20, 2023

  • Update dependencies

• 1.1.2 February 8, 2023

  • Security fixes

• 1.1.1 December 4, 2022

  • Remove whitespaces from user agent strings, this is a patch release

• 1.1.0 November 26, 2022

  • Add pkg_resource as fallback mechanism in trying to retrieve the local JSON data file

• 1.0.1 November 10, 2022

  • Add importlib-metadata & importlib-resources as dependencies
  • Check on specific Python version regarding the importlib resources (python v3.10 or higher) in order to have files() working
  • importlib_metadata should now also work on Python version before 3.8
  • Remove obsolete MANIFEST.in file

• 1.0.0 November 17, 2022

  • Make the JSON Lines data file part of the Python package, data is retrieved locally
    • Extend the myproject.toml file with package-data support
  • Remove centralized caching server implementation
  • Make real unit-tests which should run reliable, fast, independent and without Internet connection

• 0.1.14 November 5, 2022

  • Improve code quality standards using modern Python >=3.7 syntax
  • Migrated to pyproject.toml build system format + syntax check
  • Add additional classifiers to the toml file
  • Improved tox.ini file
  • Improved GitHub Actions job using pip cache
  • And various small fixes

• 0.1.13 October 21, 2022

  • Implement browsers argument, allowing you to override the browser names you want to use
  • Fix browser listing of Internet Explorer and Edge
  • Don't depend on w3schools.com anymore
  • Clean-up data (temp) file format
  • Update fallback cache server URL / use JSON Lines as file format
  • Move to GitHub Actions instead of Travis
  • Using black Python formatter in favour of Flake

• 0.1.12 March 31, 2022

  • forked

• 0.1.11 October 4, 2018

  • moved s3 + cloudfront fallback to heroku.com, cuz someone from Florida did ~25M requests last month

• 0.1.10 February 11, 2018

  • Minor fix docs cloudfront url

• 0.1.9 February 11, 2018

  • fix w3schools.com renamed IE/Edge to Edge/IE
  • moved heroku.com fallback to s3 + cloudfront
  • stop testing Python3.3 and pypy

• 0.1.8 November 2, 2017

  • fix useragentstring.com Can't connect to local MySQL server through socket

• 0.1.7 April 2, 2017

  • fix broken README.rst

• 0.1.6 April 2, 2017

  • fixes bug use_cache_server do not affected anything
  • w3schools.com <https://www.w3schools.com/browsers/browsers_stats.asp>_ moved to https
  • verify_ssl options added, by default it is True (urllib.urlopen ssl context for Python 2.7.9- and 3.4.3- is not supported)

• 0.1.5 February 28, 2017

  • added ua.edge alias to Internet Explorer
  • w3schools.com starts displaying Edge statistic
  • Python 2.6 is not tested anymore
  • use_cache_server option added
  • Increased fake_useragent.settings.HTTP_TIMEOUT to 5 seconds

• 0.1.4 December 14, 2016

  • Added custom data file location support
  • Added fallback browser support, in case of unavailable data sources
  • Added alias fake_useragent.FakeUserAgent for fake_useragent.UserAgent
  • Added alias fake_useragent.UserAgentError for fake_useragent.FakeUserAgentError
  • Reduced fake_useragent.settings.HTTP_TIMEOUT to 3 seconds
  • Started migration to new data file format
  • Simplified a lot 4+ years out of date code
  • Better thread/greenlet safety
  • Added verbose logging
  • Added safe_attrs for prevent overriding by __getattr__

• 0.1.3 November 24, 2016

  • Added hosted data file, when remote services is unavailable
  • Raises fake_useragent.errors.FakeUserAgentError in case when there is not way to download data
  • Raises fake_useragent.errors.FakeUserAgentError instead of None in case of unknown browser
  • Added gevent.sleep support in gevent patched environment when trying to download data

• X.X.X xxxxxxx xx, xxxx

  • xxxxx ?????

Authors

You can visit authors page.

fake-useragent's Issues

scrapy middleware - randomize useragent

Useragentstring.com change causing 404

Apparently the site isn't responding to requests to 'http://useragentstring.com/pages/%s/' anymore, causing an initialization of UserAgent to raise a 404 error.

Add capability when ssl module is not available

fake-useragent.herokuapp.com unreliable uptime

It seems the cache server is unreliably available, as my programs are failing. They are trying to load "https://fake-useragent.herokuapp.com/browsers/0.1.8". Sometimes when I visit this page in a browser it is available (loading "browsers": {"chrome":.....), most of the time I am getting a connection closed error:

Error occurred during loading data. Trying to use cache server https://fake-useragent.herokuapp.com/browsers/0.1.8
Traceback (most recent call last):
  File "C:\Python36\lib\site-packages\fake_useragent\utils.py", line 67, in get
    context=context,
  File "C:\Python36\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python36\lib\urllib\request.py", line 526, in open
    response = self._open(req, data)
  File "C:\Python36\lib\urllib\request.py", line 544, in _open
    '_open', req)
  File "C:\Python36\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "C:\Python36\lib\urllib\request.py", line 1346, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "C:\Python36\lib\urllib\request.py", line 1321, in do_open
    r = h.getresponse()
  File "C:\Python36\lib\http\client.py", line 1331, in getresponse
    response.begin()
  File "C:\Python36\lib\http\client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "C:\Python36\lib\http\client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

Configurable settings

I'd like to be able to change settings like BROWSERS_COUNT_LIMIT.
Maybe just as a parameter to the update method.

Index out of range

Seems that some parameters have changed and it is causing a bug:

>>> import fake_useragent
>>> fake_useragent.UserAgent()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/site-packages/fake_useragent/fake.py", line 10, in __init__
    self.data = load_cached()
  File "/usr/local/lib/python2.7/site-packages/fake_useragent/utils.py", line 140, in load_cached
    update()
  File "/usr/local/lib/python2.7/site-packages/fake_useragent/utils.py", line 135, in update
    write(load())
  File "/usr/local/lib/python2.7/site-packages/fake_useragent/utils.py", line 92, in load
    browsers_dict[browser_key] = get_browser_versions(browser)
  File "/usr/local/lib/python2.7/site-packages/fake_useragent/utils.py", line 55, in get_browser_versions
    html = html.split('<div id=\'liste\'>')[1]
IndexError: list index out of range

Useragentstring.com throws 404

Hi @hellysmile!

Thank you for this nice library, it is a really nice addition to the community! Today I tried to use the library but got stuck when it tried to connect to http://useragentstring.com/pages/Chrome/.

It seems as if that page throws a 404. I have checked through the Wayback Machine and it seems as if that page existed in the past. My guess is that useragentstring.com changed their website architecture.

If you believe the site was down temporarily and everything works as expected then please mark this issue as solved.

Thank you so much!

Service is down

Returning 404

http://pastebin.com/raw/LKpiWimh

Still IndexError: list index out of range

I've seen u have already fixed this problem but it still appears to me :(

root@vmi52271:~/# pip install --upgrade fake-useragent
Collecting fake-useragent
  Downloading fake-useragent-0.1.2.tar.gz
Building wheels for collected packages: fake-useragent
  Running setup.py bdist_wheel for fake-useragent ... done
  Stored in directory: /root/.cache/pip/wheels/be/63/15/f6e26846756da814630681d9fd98d53310426a8464289d7455
Successfully built fake-useragent
Installing collected packages: fake-useragent
Successfully installed fake-useragent-0.1.2
root@vmi52271:~/# pip install -U fake-useragent
Requirement already up-to-date: fake-useragent in /usr/local/lib/pypy2.7/dist-packages

root@vmi52271:~/# python
Python 2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from fake_useragent import UserAgent
>>> ua = UserAgent()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/fake_useragent/fake.py", line 10, in __init__
    self.data = load_cached()
  File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 140, in load_cached
    update()
  File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 135, in update
    write(load())
  File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 92, in load
    browsers_dict[browser_key] = get_browser_versions(browser)
  File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 55, in get_browser_versions
    html = html.split('<div id=\'liste\'>')[1]
IndexError: list index out of range

thanks for the great work

Tests is needed

Error occurred during getting browser

I have a package that depends on fake-useragent, and recently, it's automated unittests have been failing because fake-useragent has been timing out when it tries to retrieve agents.

 File "/home/travis/build/chrisspen/howdou/.tox/py27/lib/python2.7/site-packages/fake_useragent/fake.py", line 98, in __getattr__

    raise FakeUserAgentError('Error occurred during getting browser')  # noqa

What's causing this? Am I hitting some web resource too much, or is it a bug in fake-useragent? What can I do to minimize this or cache the results locally?

ua = UserAgent() error

Hi Guys,

I'm new to python so I apologize if this is basic. The first two lines of my script read:

from fake_useragent import UserAgent
ua = UserAgent()

I'm getting the following error:
File "Clicks.py", line 2, in
ua = UserAgent()
File "/Library/Python/2.7/site-packages/fake_useragent/fake.py", line 10, in init
self.data = load_cached()
File "/Library/Python/2.7/site-packages/fake_useragent/utils.py", line 140, in load_cached
update()
File "/Library/Python/2.7/site-packages/fake_useragent/utils.py", line 135, in update
write(load())
File "/Library/Python/2.7/site-packages/fake_useragent/utils.py", line 92, in load
browsers_dict[browser_key] = get_browser_versions(browser)
File "/Library/Python/2.7/site-packages/fake_useragent/utils.py", line 55, in get_browser_versions
html = html.split('<div id='liste'>')[1]
IndexError: list index out of range

I've used this script many times successfully in the past and nothing in my code has changed. This is the first time I've run it in about a week though. Thanks again for any help.

-Tim

Please add a __version__ method

A lot of modules have a version method that reports the version of the module itself. Could you add something that?

socket.timeout: timed out

When I try use ua = UserAgent() on a EC2 ubuntu instance, I get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/cncases/local/lib/python3.4/site-packages/fake_useragent/fake.py", line 13, in __init__
    self.load()
  File "/home/ubuntu/cncases/local/lib/python3.4/site-packages/fake_useragent/fake.py", line 17, in load
    self.data = load_cached()
  File "/home/ubuntu/cncases/local/lib/python3.4/site-packages/fake_useragent/utils.py", line 135, in load_cached
    update()
  File "/home/ubuntu/cncases/local/lib/python3.4/site-packages/fake_useragent/utils.py", line 130, in update
    write(load())
  File "/home/ubuntu/cncases/local/lib/python3.4/site-packages/fake_useragent/utils.py", line 96, in load
    browsers_dict[browser_key] = get_browser_versions(browser)
  File "/home/ubuntu/cncases/local/lib/python3.4/site-packages/fake_useragent/utils.py", line 61, in get_browser_versions
    html = get(settings.BROWSER_BASE_PAGE.format(browser=quote_plus(browser)))
  File "/home/ubuntu/cncases/local/lib/python3.4/site-packages/fake_useragent/utils.py", line 27, in get
    return urlopen(request, timeout=settings.HTTP_TIMEOUT).read()
  File "/usr/lib/python3.4/urllib/request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 463, in open
    response = self._open(req, data)
  File "/usr/lib/python3.4/urllib/request.py", line 481, in _open
    '_open', req)
  File "/usr/lib/python3.4/urllib/request.py", line 441, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 1210, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.4/urllib/request.py", line 1185, in do_open
    r = h.getresponse()
  File "/usr/lib/python3.4/http/client.py", line 1171, in getresponse
    response.begin()
  File "/usr/lib/python3.4/http/client.py", line 351, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.4/http/client.py", line 313, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.4/socket.py", line 374, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

The machine definitely have internet connection since I can ping google. Also, this only happens to a few instances I established this week. The ones from last month are still running properly. I have update fake-useragent to its most recent version 0.1.2.

Showing "list index out of range" again and cannot be fixed by reinstalling

Hi when I type in the commands:

from fake_useragent import UserAgent
ua = UserAgent()

It shows the following error:

Traceback (most recent call last):
File "scraper.py", line 287, in
ua = UserAgent()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/fake.py", line 10, in init
self.data = load_cached()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 140, in load_cached
update()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 135, in update
write(load())
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 92, in load
browsers_dict[browser_key] = get_browser_versions(browser)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 55, in get_browser_versions
html = html.split('<div id='liste'>')[1]
IndexError: list index out of range

I tried uninstalling and reinstalling using pip install fake-useragent, but the error still shows. Could you have a look and see what is the problem?
Thanks!

Installation guide?

How about a SIMPLE how to install guide for those of us who need it?

I don't mind copying & pasting Terminal commands, but I don't script.

Error occurred during fetching https://www.w3schools.com/browsers/default.asp

Hi,buddy,I found a bug like this:
DEBUG: Error occurred during fetching https://www.w3schools.com/browsers/default.asp
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/fake_useragent/utils.py", line 67, in get
context=context,
File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 1258, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib64/python2.7/urllib2.py", line 1214, in do_open
raise URLError(err)
Hope to be solved,thank you

Add random_browser & random_mobile

It would be useful to be able to use the very nice "random" behavior but ensure to always or never get a useragent corresponding to a mobile device

There is no option to use only cache server, do not try remote services

Browser stats page down [503]

http://www.w3schools.com/browsers/browsers_stats.asp

Page http://www.w3schools.com/browsers/browsers_stats.asp has changed.

Below is the traceback.

The page http://www.w3schools.com/browsers/browsers_stats.asp doesn't have a table with class="reference no translate".

I guess the layout has changed in the last 24 hours.

from fake_useragent import UserAgent
ua = UserAgent(cache=False)

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/fake_useragent/fake.py", line 12, in init
self.data = load()
File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 82, in load
for item in get_browsers():
File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 29, in get_browsers
html = html.split('

')[1]
IndexError: list index out of range

http://www.w3schools.com/browsers/browsers_stats.asp

Since the last update. All my programs crushsss due to: html = html.split('<div id=\'liste\'>')[1] IndexError: list index out of range

File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 2000, in call
return self.wsgi_app(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1991, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1567, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request
return self.view_functionsrule.endpoint
File "/root/appsflyer_pushs/main_with_stream.py", line 127, in listen
if valid_request(ip=request.remote_addr):
File "/root/appsflyer_pushs/main_with_stream.py", line 90, in valid_request
ua = UserAgent(cache=False)
File "/usr/local/lib/python2.7/dist-packages/fake_useragent/fake.py", line 12, in init
self.data = load()
File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 92, in load
browsers_dict[browser_key] = get_browser_versions(browser)
File "/usr/local/lib/python2.7/dist-packages/fake_useragent/utils.py", line 55, in get_browser_versions
html = html.split('<div id='liste'>')[1]
IndexError: list index out of range

ValueError: could not convert string to float: <a

Here is the whole traceback:

Traceback (most recent call last):
File "...\fake_useragent\fake.py", line 11, in init
self.data = load_cached()
File "...\fake_useragent\utils.py", line 133, in load_cached
update()
File "...\fake_useragent\utils.py", line 128, in update
write(load())
File "...\fake_useragent\utils.py", line 87, in load
for counter in range(int(float(percent))):
ValueError: could not convert string to float: <a

I tried to upgrade fake-useragent. It did not help.
I also tried pip uninstall fake-useragent and pip install fake-useragent again. It returns the same error message.

By the way, my OS is Windows 8.1.

I'll appreciate it if you help me with this issue.

Create some tag releases

Not everyone uses pip and in order to fetch the project besides pulling from master, it would be ideal to fetch from tags from a package manager perspective that is. Thanks in advanced

Один и тот же useragent

Добрый день. Хотелось бы видеть в новой версии возможность получения одного и того же useragent. К примеру, ua = UserAgent(last=True), после чего ua.chrome ВСЕГДА выдаёт самый свежий (для данной базы) useragent для этого браузера.

Для чего это надо - я ежедневно захожу на сайт, который при авторизации запрашивает и userkey, который высчитывается в том числе и на основе строки useragent. Если я буду ежедневно (по несколько раз) заходить с разным userkey, то это неизбежно вызовет подозрения.

ps: "ВСЕГДА" - в смысле не только для одной сессии, а при любом запуске скрипта.

IndexError: list index out of range

In [1]: from fake_useragent import UserAgent

In [2]: ua = UserAgent()
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-2-89a0eee92536> in <module>()
----> 1 ua = UserAgent()

/Users/josefson/virtualenvs/CNPQ/lib/python2.7/site-packages/fake_useragent/fake.pyc in __init__(self, cache)
      8     def __init__(self, cache=True):
      9         if cache:
---> 10             self.data = load_cached()
     11         else:
     12             self.data = load()

/Users/josefson/virtualenvs/CNPQ/lib/python2.7/site-packages/fake_useragent/utils.pyc in load_cached()
    138 def load_cached():
    139     if not exist():
--> 140         update()
    141
    142     return read()

/Users/josefson/virtualenvs/CNPQ/lib/python2.7/site-packages/fake_useragent/utils.pyc in update()
    133         rm()
    134
--> 135     write(load())
    136
    137

/Users/josefson/virtualenvs/CNPQ/lib/python2.7/site-packages/fake_useragent/utils.pyc in load()
     80     randomize_dict = {}
     81
---> 82     for item in get_browsers():
     83         browser, percent = item
     84

/Users/josefson/virtualenvs/CNPQ/lib/python2.7/site-packages/fake_useragent/utils.pyc in get_browsers()
     27     html = get(settings.BROWSERS_STATS_PAGE)
     28     html = html.decode('windows-1252')
---> 29     html = html.split('<table class="reference notranslate">')[1]
     30     html = html.split('</table>')[0]
     31

IndexError: list index out of range

Certain sites will not be accessible with out of date random user agents.

For me, certain sites will not be accessible with out of date random user agents using this library.

This might just be a limitation of this library, but I thought I would add my 2c in saying that adding an option for most popular user strings to prevent unable to access might be worthwhile.

E.g Taken From here-
https://developers.whatismybrowser.com/useragents/explore/software_name/chrome/

Might be of use. I use Chrome a lot so hence this example.

FakeUserAgentError on concurrent requests on Linux Server

Traceback (most recent call last):
  File "/usr/local/lib64/python2.7/site-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = g.send(result)
  File "/usr/local/lib/python2.7/site-packages/scrapy/core/downloader/middleware.py", line 37, in process_request
    response = yield method(request=request, spider=spider)
  File "/usr/local/lib/python2.7/site-packages/scrapy_fake_useragent/middleware.py", line 27, in process_request
    request.headers.setdefault('User-Agent', self.ua.random)
  File "/usr/local/lib/python2.7/site-packages/fake_useragent/fake.py", line 98, in __getattr__
    raise FakeUserAgentError('Error occurred during getting browser')  # noqa
FakeUserAgentError: Error occurred during getting browser

I keep getting this error on the Linux server when I run multiple spiders concurrently. What should I do to avoid that? Do I have to raise the RAM or something?

Browser list has no values, raising FakeUserAgentError in fake.py", line 140,

Hi there,

For whatever reason, the self.data['browsers']/self.data_browsers[browser] has only keys, no values,

{u'chrome': [], u'opera': [], u'firefox': [], u'internetexplorer': [], u'safari': []}

Therefore our production cron is broken due to this error

raise FakeUserAgentError('Error occurred during getting browser')  # noqa

So I am asking you to return a default browser if choice is empty.
i.e in line https://github.com/hellysmile/fake-useragent/blob/master/fake_useragent/fake.py#L136

if not self.data_browsers[browser]:
    return random.choice(['firefox', 'chrome', 'opera', 'safari']) # i.e a fallback list
return random.choice(self.data_browsers[browser])

Hope you can give a better solution.

fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

FakeUserAgentError('Error occurred during getting browser')

I'm getting this error with version 0.1.7 running on Mac OS X. It seems that the common suggestion to this is to update the version, but I think I have a version where this error should not come anymore? Any ideas?

Traceback (most recent call last): File "/Users/mikko/dev/norway/lib/python2.7/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks result = g.send(result) File "/Users/mikko/dev/norway/lib/python2.7/site-packages/scrapy/core/downloader/middleware.py", line 37, in process_request response = yield method(request=request, spider=spider) File "/Users/mikko/dev/norway/lib/python2.7/site-packages/scrapy_fake_useragent/middleware.py", line 28, in process_request self.proxy2ua[proxy] = get_ua() File "/Users/mikko/dev/norway/lib/python2.7/site-packages/scrapy_fake_useragent/middleware.py", line 23, in get_ua return getattr(self.ua, self.ua_type) File "/Users/mikko/dev/norway/lib/python2.7/site-packages/fake_useragent/fake.py", line 139, in __getattr__ raise FakeUserAgentError('Error occurred during getting browser') # noqa FakeUserAgentError: Error occurred during getting browser

Urllib timeout

Hello!
Thanks for cool library.
I have one issue though: on my hardware I see the exception "Error occurred during formatting data. Trying to use fallback server" too often. Hardcoding settings.HTTP_TIMEOUT to large value like 10 makes it go away though. Could you please increase it a bit (say, to 5) or maybe add a possibility to define particular timeout when creating UserAgent instance?

Distinguish between desktop and mobile (Android vs iOS)

I writes a spider to crawl the web from PC front with ua.random, but sometimes, it would redirect to mobile front , that's not what I expected.
I found it caused by ua.random would return a mobile ua as below:

import requests
res = requests.get("https://fake-useragent.herokuapp.com/browsers/0.1.5")
print res.json()['browsers']['safari'][41]
# Mozilla/5.0 (Android 2.2; Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4

From #10 (comment), there will be eventually added ua.desktop and ua.mobile, I'm really looking forward to it.

can't use fake-useragent

when I use fake-useragent in my scrapy spider.
When I type "scrapy crawl myspider", it give me an error "[fake_useragent] DEBUG: Error occurred during fetching https://www.w3schools.com/browsers/default.asp"
ConnectionRefusedError: [Errno 111] Connection refused.
Then I use command "scrapy fetch website", it give me the same error.
Finally, I get "https://www.w3schools.com/browsers/default.asp" with chrome browser. still failed.
So, I guess w3school going wrong.

Filter on date

Hello,

Thank you for the nice module. A nice addition to this library would be the ability to filter the user agent based on their release date (so that you dont get too old browser). Sadly, it seems http://useragentstring.com/ doesn't provide this kind of information. Another thing is that it seems most of the UA seems out of date on the website (exemple, for chrome, last version on useragentstring.com is 41 when in reality it is 52..). I tried to look around to find an up-to-date database of all user-agent but i kind of failed :( Does anyone know where we could gather/scrape this data ?

The best i could found is: https://techblog.willshouse.com/2012/01/03/most-common-user-agents/

user agent throws "ValueError: could not convert string to float" error - on both cached /uncached calls

from fake_useragent import UserAgent
on calling ua = UserAgent() or ua = UserAgent(cache=False)

Traceback (most recent call last):
File "<pyshell#17>", line 1, in
ua = UserAgent()
File "/usr/local/lib/python2.7/dist-packages/fake_useragent-0.0.6-py2.7.egg/fake_useragent/fake.py", line 12, in init
self.data = load_cached()
File "/usr/local/lib/python2.7/dist-packages/fake_useragent-0.0.6-py2.7.egg/fake_useragent/utils.py", line 134, in load_cached
update()
File "/usr/local/lib/python2.7/dist-packages/fake_useragent-0.0.6-py2.7.egg/fake_useragent/utils.py", line 129, in update
write(load())
File "/usr/local/lib/python2.7/dist-packages/fake_useragent-0.0.6-py2.7.egg/fake_useragent/utils.py", line 88, in load
for counter in range(int(float(percent))):
ValueError: could not convert string to float: <a

index out of range error on initialization

I am getting this when trying to init a new UserAgent()
Traceback (most recent call last):
File "", line 1, in
File "/Users/davidyu/.virtualenvs/webscraper/lib/python2.7/site-packages/fake_useragent/fake.py", line 10, in init
self.data = load_cached()
File "/Users/davidyu/.virtualenvs/webscraper/lib/python2.7/site-packages/fake_useragent/utils.py", line 140, in load_cached
update()
File "/Users/davidyu/.virtualenvs/webscraper/lib/python2.7/site-packages/fake_useragent/utils.py", line 135, in update
write(load())
File "/Users/davidyu/.virtualenvs/webscraper/lib/python2.7/site-packages/fake_useragent/utils.py", line 92, in load
browsers_dict[browser_key] = get_browser_versions(browser)
File "/Users/davidyu/.virtualenvs/webscraper/lib/python2.7/site-packages/fake_useragent/utils.py", line 55, in get_browser_versions
html = html.split('<div id='liste'>')[1]
IndexError: list index out of range

I believe it is possibly because internetexplorer's BROWSER_BASE_PAGE is not returning any usable values.

ua.chrome need upgrade

fake-useragent.UserAgent().chrome
return
'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36'
but the newest chrome is
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36'
Chrome/27.0 is too old, and when I request 'https://www.zhihu.com/sigup', I'm redirected to 'https://www.zhihu.com/compatibility/index.html'

twisted integration

replace urllib.request.urlopen by twisted.web.client.Agent

Error with Python3

I'm getting this error with version 0.1.7 running on Mac OS X with python3.6.1 when I try the code below:

from fake_useragent import UserAgent
ua = UserAgent()

which raises error:

Error occurred during loading data. Trying to use cache server https://fake-useragent.herokuapp.com/browsers/0.1.7
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1392, in connect
    super().connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 722, in create_connection
    raise err
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 67, in get
    context=context,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 150, in load
    for item in get_browsers(verify_ssl=verify_ssl):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 97, in get_browsers
    html = get(settings.BROWSERS_STATS_PAGE, verify_ssl=verify_ssl)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 84, in get
    raise FakeUserAgentError('Maximum amount of retries reached')
fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1392, in connect
    super().connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 722, in create_connection
    raise err
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/socket.py", line 713, in create_connection
    sock.connect(sa)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 67, in get
    context=context,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 150, in load
    for item in get_browsers(verify_ssl=verify_ssl):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 97, in get_browsers
    html = get(settings.BROWSERS_STATS_PAGE, verify_ssl=verify_ssl)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 84, in get
    raise FakeUserAgentError('Maximum amount of retries reached')
fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 67, in get
    context=context,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/fake.py", line 69, in __init__
    self.load()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/fake.py", line 78, in load
    verify_ssl=self.verify_ssl,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 246, in load_cached
    update(path, use_cache_server=use_cache_server, verify_ssl=verify_ssl)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 241, in update
    write(path, load(use_cache_server=use_cache_server, verify_ssl=verify_ssl))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 185, in load
    verify_ssl=verify_ssl,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/fake_useragent/utils.py", line 84, in get
    raise FakeUserAgentError('Maximum amount of retries reached')
fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached

But the same code works fine with python2.7. Any ideas?

urllib2.URLError: <urlopen error timed out>

from fake_useragent import UserAgent
ua = UserAgent()

Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/fake.py", line 17, in init
self.load()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/fake.py", line 21, in load
self.data = load_cached()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 138, in load_cached
update()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 133, in update
write(load())
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 99, in load
browsers_dict[browser_key] = get_browser_versions(browser)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 64, in get_browser_versions
html = get(settings.BROWSER_BASE_PAGE.format(browser=quote_plus(browser)))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/fake_useragent/utils.py", line 29, in get
return urlopen(request, timeout=settings.HTTP_TIMEOUT).read()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1227, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open
raise URLError(err)
urllib2.URLError:

Maximum amount of retries reached

I can not access "www.w3schools.com" in China,so I can't call get_browsers() in utils.py.

Add proxy support

Urllib usage does not account for proxies

If I have time, I will write a pull request, but I cannot promise. Not sure if this is something the author is aware of, so issue created.

Cant run it on QPython on Android

I get this out of the errors.log

WARNING:root:Error occurred during loading data. Trying to use cache server https://fake-useragent.herokuapp.com/browsers/0.1.8
Traceback (most recent call last):
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 154, in load
for item in get_browsers(verify_ssl=verify_ssl):
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 97, in get_browsers
html = get(settings.BROWSERS_STATS_PAGE, verify_ssl=verify_ssl)
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 84, in get
raise FakeUserAgentError('Maximum amount of retries reached')
FakeUserAgentError: Maximum amount of retries reached
WARNING:root:Error occurred during loading data. Trying to use cache server https://fake-useragent.herokuapp.com/browsers/0.1.8
Traceback (most recent call last):
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 154, in load
for item in get_browsers(verify_ssl=verify_ssl):
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 97, in get_browsers
html = get(settings.BROWSERS_STATS_PAGE, verify_ssl=verify_ssl)
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 84, in get
raise FakeUserAgentError('Maximum amount of retries reached')
FakeUserAgentError: Maximum amount of retries reached

And this out of executing the script

/data/user/0/org.qpython.qpy/files/bin/qpython-android5-root.sh
"/storage/emulated/0/qpython/instabot.py-master/main.py" && exit
/instabot.py-master/main.py" && exit <
Traceback (most recent call last):
File "/storage/emulated/0/qpython/instabot.py-master/main.py", line 77, in
'yuki_nishitani8','contemporary.paintings','danteslens','ramontrotman','voidcrack',])
File "/storage/emulated/0/qpython/instabot.py-master/src/instabot.py", line 162, in init fake_ua = UserAgent()
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/fake.py", line 69, in init self.load()
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/fake.py", line 78, in load
verify_ssl=self.verify_ssl,
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 250, in load_cached
update(path, use_cache_server=use_cache_server, verify_ssl=verify_ssl)
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 245, in update write(path, load(use_cache_server=use_cache_server, verify_ssl=verify_ssl))
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 189, in load
verify_ssl=verify_ssl,
File "/data/user/0/org.qpython.qpy/files/lib/python2.7/site-packages/fake_useragent-0.1.8-py2.7.egg/fake_useragent/utils.py", line 84, in get
raise FakeUserAgentError('Maximum amount of retries reached')
fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached
1|bullhead:/ $

Can someone tell me what is wrong? Have no issus on Windows or Mac

Cannot choose from an empty sequence

I have been having a lot of problems today regarding fake_useragent. In a real world example of using selenium, but i cant even get it to pull the string at all either.

Python 3.6.1 (default, Jun  8 2017, 06:36:16) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from fake_useragent import UserAgent as UA
>>> ua = UA()
>>> ua.random
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 136, in __getattr__
    return random.choice(self.data_browsers[browser])
  File "/usr/local/lib/python3.6/random.py", line 257, in choice
    raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 139, in __getattr__
    raise FakeUserAgentError('Error occurred during getting browser')  # noqa
fake_useragent.errors.FakeUserAgentError: Error occurred during getting browser
>>> ua.VERSION
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 136, in __getattr__
    return random.choice(self.data_browsers[browser])
KeyError: 'version'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 139, in __getattr__
    raise FakeUserAgentError('Error occurred during getting browser')  # noqa
fake_useragent.errors.FakeUserAgentError: Error occurred during getting browser
>>> ua.chrome
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 136, in __getattr__
    return random.choice(self.data_browsers[browser])
  File "/usr/local/lib/python3.6/random.py", line 257, in choice
    raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 139, in __getattr__
    raise FakeUserAgentError('Error occurred during getting browser')  # noqa
fake_useragent.errors.FakeUserAgentError: Error occurred during getting browser
>>> ua['google chrome']
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 136, in __getattr__
    return random.choice(self.data_browsers[browser])
  File "/usr/local/lib/python3.6/random.py", line 257, in choice
    raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 119, in __getitem__
    return self.__getattr__(attr)
  File "/usr/local/lib/python3.6/site-packages/fake_useragent/fake.py", line 139, in __getattr__
    raise FakeUserAgentError('Error occurred during getting browser')  # noqa
fake_useragent.errors.FakeUserAgentError: Error occurred during getting browser
>>> 

another traceback with selenium at
#41

Self health check

Each time when fake-useragent downloaded and combined data, data needs to be verified

Meaningful error message if get() reaches HTTP_RETRIES

I'm using fake-useragent in combination with scrapy-fake-useragent and due to useragentstring.com downtimes this library's get() method yields a socket.timeout: timed out.

It would be nice to have a distinct exception type, e.g. FakeUserAgentError, which says something like Url {0} unreachable instead.

Add android as option

Would be great to add android as an option, for example

ua.android

Thanks

useragentstring.com (and as a result fake-useragent) is Out of Date

The service this library relies on to get user agent data is woefully out of date.

This library pulls from this page to get firefox User Agent strings, but it looks like it only goes up to version 40.1, which was released in 2015.

Similarly, chrome versions are pulled from here and only go up to version 41.

useragentstring.com uptime is low

list of alternatives
http://user-agent-string.info/list-of-ua