Coder Social home page Coder Social logo

useragentparser's Introduction

UserAgentParser

Build Status Code Coverage Scrutinizer Code Quality

Latest Stable Version Latest Unstable Version License Total Downloads

User agent parsing is, was and will always be a painful thing.

The target of this package is to make it less painful, by providing an abstract layer for many user agent parsers.

Currently 11 local providers and 6 HTTP providers are available! See the comparison list here

So you can

  • use multiple providers at the same time with the Chain provider
  • use local and/or HTTP API providers at the same time
  • switch between different parsers, without changing your code
  • compare the result of the different parsers
  • get always the same result model, regardless of which parser you use currently

The quality of this package is currently covered by

  • unit tests (373 tests, 746 assertions)
  • integration tests (86 tests, 310 assertions)
  • regular real result testing (the results of over 33.000 user agents are compared here)

Try it out

LIVE test

Compare the detection results of the parsers

Installation

Using composer is currently the only supported way to install this package.

composer require thadafinser/user-agent-parser

Note: to use local providers you need to install additional packages, which are listed inside the composer suggests section

Getting started

You need to register an API key or install an additional package (listed in the section suggest of composer.json)

use UserAgentParser\Exception\NoResultFoundException;
use UserAgentParser\Provider\WhichBrowser;

$provider = new WhichBrowser();

try {
    /* @var $result \UserAgentParser\Model\UserAgent */
    $result = $provider->parse('Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.73 Safari/537.36');
} catch (NoResultFoundException $ex){
    // nothing found
}

if($result->isBot() === true) {
  // if one part has no result, it's always set not null
  $result->getBot()->getName();
  $result->getBot()->getType();
} else {
  // if one part has no result, it's always set not null
  $result->getBrowser()->getName();
  $result->getBrowser()->getVersion()->getComplete();

  $result->getRenderingEngine()->getName();
  $result->getRenderingEngine()->getVersion()->getComplete();

  $result->getOperatingSystem()->getName();
  $result->getOperatingSystem()->getVersion()->getComplete();

  $result->getDevice()->getModel();
  $result->getDevice()->getBrand();
  $result->getDevice()->getType();
  $result->getDevice()->getIsMobile();
  $result->getDevice()->getIsTouch();
}

Use cases

Bot or human

// initialisation see Getting started
if($result->isBot() === true) {
    // do something special with the bot
}

Mobile detection

// initialisation see Getting started
if($result->isMobile() === true) {
    // redirect to the the mobile optimized page or suggest the other to download your app
    // NOTE mobile means not "phone". It can be any moveable device, e.g. tablet, media player, watch, ...
}

Providers

UserAgentParser comes with local and http providers

See detailed documenation here

local providers

http providers

Name Type Browser Engine Operating system Device model Device brand Device type Is mobile Is bot Bot name Bot type Comment
BrowscapFull local x x x x x x x x x x
BrowscapLite local x x x x
BrowscapPhp local x x x x x x
DonatjUAParser local x
Endorphin local x x x x x x
HandsetDetection local x x x x
JenssegersAgent local x x x x x Based on MobileDetect
PiwikDeviceDetector local x x x x x x x x x x
SinergiBrowserDetector local x x x x x
UAParser local x x x x x x
WhichBrowser local x x x x x x x x
Woothee local x x x x
Zsxsoft local x x x x
DeviceAtlasCom http x x x x free available
FiftyOneDegreesCom http x x x x x x x x free unlimited
NeutrinoApiCom http x x x x x x x x 25/day free
UdgerCom http x x x x x 500/month free (API key only for one month valid!)
UserAgentApiCom http x x x x x 1000/day free
WhatIsMyBrowserCom http x x 500/month free

Local providers

Local providers are (most time) faster then HTTP providers and dont require a working internet connection. But you need to update them yourself from time to time, to make sure you detect the latest UAs

  • BrowscapFull
  • BrowscapLite
  • BrowscapPhp
  • DonatjUAParser
  • Endorphin
  • HandsetDetection
  • JenssegersAgent
  • PiwikDeviceDetector
  • SinergiBrowserDetector
  • UAParser
  • WhichBrowser
  • Woothee
  • Zsxsoft

HTTP providers (API)

HTTP providers are simple to use, since you need only an API key to get started. But they require (always) a working internet connection.

  • Http\DeviceAtlasCom
  • Http\FiftyOneDegreesCom
  • Http\NeutrinoApiCom
  • Http\UdgerCom
  • Http\UserAgentApiCom
  • Http\WhatIsMyBrowserCom

Comparison matrix

Here is a comparison matrix, with many analyzed UserAgent strings, to help you device which provider fits your needs. Every provider has it's strengh and weakness, so it will depend on your need, which one you should use.

Go to the comparison

Overview

Single provider

require 'vendor/autoload.php';

use UserAgentParser\Provider;

$userAgent = 'Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5';

$provider = new Provider\PiwikDeviceDetector();

/* @var $result \UserAgentParser\Model\UserAgent */
$result = $provider->parse($userAgent);
// optional add all headers, to improve the result further
// $result = $provider->parse($userAgent, getallheaders());

$result->getBrowser()->getName(); // Mobile Safari
$result->getOperatingSystem()->getName(); // iOS
$result->getDevice()->getBrand(); // iPod Touch
$result->getDevice()->getBrand(); // Apple
$result->getDevice()->getType(); // portable media player

$resultArray = $result->toArray();

Chain provider

This is very useful to improve your results. The chain provider starts with the first provider and checks if there is a result, if not it takes the next one and so on. If none of them have a result, it will throw a NoResultException like a single provider.

require 'vendor/autoload.php';

use UserAgentParser\Provider;

$userAgent = 'Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5';

$chain = new Provider\Chain([
    new Provider\PiwikDeviceDetector(),
    new Provider\WhichBrowser(),
    new Provider\UAParser(),
    new Provider\Woothee(),
    new Provider\DonatjUAParser()
]);

/* @var $result \UserAgentParser\Model\UserAgent */
$result = $chain->parse($userAgent);
// optional add all headers, to improve the result further (used currently only by WhichBrowser)
//$result = $chain->parse($userAgent, getallheaders());

$result->getBrowser()->getName(); // Mobile Safari

$result->getOperatingSystem()->getName(); // iOS

$result->getDevice()->getBrand(); // iPod Touch
$result->getDevice()->getBrand(); // Apple
$result->getDevice()->getType(); // portable media player

$resultArray = $result->toArray();

useragentparser's People

Contributors

machou avatar nielsleenheer avatar photodude avatar thadafinser avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

useragentparser's Issues

PHP 8 support

Currently requires PHP ~5.6|~7.0. It'd be nice if we could use this with the latest PHP 8 release. There ain't many breaking changes, shouldn't be complicated (if this project is still alive).

TBD

The target of this project is wherever one or more of the following:

  • create an overview/matrix:
    • which Providers have a good quality (generation time, valid data, ...)
    • use different UA databases to generate those matrixes
  • maybe release this as a counter part for UA parsing, like https://github.com/geocoder-php/Geocoder

...for now it's just a prototype:

  • install it with composer
  • execute php tests/test.php for the matrix

//cc @sgiehl could i use the Piwik-DeviceDetector fixtures (https://github.com/piwik/device-detector/tree/master/Tests/fixtures) to generate matrix with all provider results?

More generics for various Providers

BrowscapPhp:

  • model: "Windows Desktop"
  • model: "Linux Desktop"

SinergiBrowserDetector:

  • model: "Windows Phone"

Wurfl:

  • model: "SmartTV"
  • browser: "Java Applet"

Final check Browscap

Hello @mimmi20 (sorry to ping again)

i start now to finalize the BrowscapPhp provider.

Could you maybe check if i missed here something, or done something wrong in the implementation?
https://github.com/ThaDafinser/UserAgentParser/blob/master/src/Provider/BrowscapPhp.php

For testing i create a small browscap.ini file and used it in the tests.
https://github.com/ThaDafinser/UserAgentParser/blob/master/tests/fixtures/browscap.ini

Is this fine or do i need to mock i further?
https://github.com/ThaDafinser/UserAgentParser/blob/master/tests/unit/Provider/BrowscapPhpTest.php#L41

composer autoload issue

If the running script is placed in a a subdirectory, it will break the loading of the installed providers.

For ex., including autoload like this
require __DIR__.'../vendor/autoload.php';

produce a fatal error:

PHP Fatal error: Uncaught exception 'UserAgentParser\\Exception\\PackageNotLoadedException' with message 'You need to install the package piwik/device-detector to use this provider' in ..

Replace generics

To solve the generics problem from ThaDafinser/UserAgentParserComparison#19

we should create something abstract to filter default/generic values.

Currently we have those variations:
https://github.com/ThaDafinser/UserAgentParser/blob/master/src/Provider/AbstractProvider.php
https://github.com/ThaDafinser/UserAgentParser/blob/master/src/Provider/Wurfl.php#L64-L66

https://github.com/ThaDafinser/UserAgentParser/blob/master/src/Provider/BrowscapPhp.php#L103-L115
https://github.com/ThaDafinser/UserAgentParser/blob/master/src/Provider/Wurfl.php#L142-L166

Possible solution

We change the possible values of $defaultValues to

  • multi dimensional array
    • general for all fields
    • specific filter for a single field (e.g browser -> name)
  • do it with regex and not only direct string comparison

Example current

namespace UserAgentParser\Provider;

class PiwikDeviceDetector extends AbstractProvider
{
    protected $defaultValues = [
        DeviceDetector::UNKNOWN,

        // bot names
        'Bot',
        'Generic Bot',
    ];
}

Example new

namespace UserAgentParser\Provider;

class PiwikDeviceDetector extends AbstractProvider
{
    protected $defaultValues = [

        'general' => [
            '/^' . DeviceDetector::UNKNOWN . '$/'
        ],

        'bot' => [
            'name' => [
                '/^Bot$/',
                '/^Generic Bot$/'
            ]
        ]
    ];
}

New Provider: crossjoin/Browscap

crossjoin/Browscap is a UA parser, who uses the browscap.ini too, but is faster and requires lower amout of RAM. Actually only a file cache is supported. The browscap-php library has adopted a lot of features from it.

Versions with non-digits are not interpreted correctly by UserAgentParser

WhichBrowser adds the nickname of the OS release before the version number. So the version is "Yosemite 10.10" and not just "10.10". UserAgentParser interprets this as "0.10".

See for example:
http://useragent.mkf.solutions/?userAgent=Kodi%2F14.0+%28Macintosh%3B+Intel+Mac+OS+X+10_10_3%29+App_Bitness%2F64+Version%2F14.0-Git%3A2014-12-23-ad747d9-dirty

This also happens with many other libraries with versions like "XP" for Windows XP. It is interpreted by UserAgentParser as "0".

See for example:
http://useragent.mkf.solutions/?userAgent=Microsoft+Office%2F14.0+%28Windows+NT+5.1%3B+Microsoft+Outlook+14.0.4536%3B+Pro%3B+MSOffice+14%29

The function getParts() should probably skip over any non digit characters in the version string before splitting it into parts.

Also when setComplete() is called, it first stores the original string as the $complete property, then hydrates all the parts. But the hydrateVersionParts() function calls setMajor(), setMinor() and setPatch() which in turn call calculateComplete() which overwrites the original $complete property that was initially set. This means the original string calculated by the parser is lost.

Change isRealResult() to getRealResult()

Instead of doing this

if ($this->isRealResult($deviceRaw->model, 'device', 'model') === true) {
    $device->setModel($deviceRaw->model);
}

Do this and return null if it's not a valid value

$device->setModel($this->getRealResult($deviceRaw->model, 'device', 'model'));

There are some providers where the value is not always available - there we still need a isset() if check before that

Ua cutted

When a UA is tested on the web, the UA is cutted if the UA conains a ".

Sample UA: Mozilla/5.0 (Linux; Android 5.1; Alba 7" Build/LMY47I) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.93 Safari/537.36

Provider dependencies detection

In order to be able to deploy the library in various environments, it would be better to check if the primitive functions/classes of the different providers are present instead of checking if the package has been deployed in a specific vendor directory.

Even if it's a best practice to put external libraries in a vendor directory, all projects don't respect it and can't deploy your great library.

I created a pull request for the DonatjUAParser Provider : #60

Let me know if it makes sense to you

Cloud / HTTP Provider

There are a lot of free and paid cloud services around.

Implement at least 1 or 2 of them to get started

Browscap-php result not corectly implemented

The used properties win32 and win64 are true only for Windows. If tyou want to detect also other platforms, use the platform_bits property.
The platform version can be detected with the platform_version property.
The device brand can be detected with the device_brand_name property.
The device model can be detected with the device_name property.

To detect the device properties you need the full_php_browscap.ini file.

Installation fails with composer error

$ composer require thadafinser/user-agent-parser
Using version ^0.1.0 for thadafinser/user-agent-parser
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for thadafinser/user-agent-parser ^0.1.0 -> satisfiable by thadafinser/user-agent-parser[v0.1.0].
    - thadafinser/user-agent-parser v0.1.0 requires browscap/browscap-php dev-master -> no matching package found.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details.

WhatIsMyBrowserCom update

Add the type + bot detection

field: user_agent_type

browser
mobile
ebook_reader
crawler
analyser
tool
tv
mediaplayer
gamesystem
library
unknown

PiwikDeviceDetector Provider error

Hi,
I a using the latest version of this library. Now theres an issue. Not sure if its a bug.
I already have this library installed PiwikDeviceDetector but i keep getting the following exception error. You need to install the package piwik/device-detector to use this provider. I have done all i could to debug but i can seem to find a solution. The only one is to comment out the following:
public function __construct(DeviceDetector $parser = null)

    {
        if ($parser === null && ! file_exists('vendor/' . $this->getPackageName() . '/composer.json')) {
         Icommented this line - // throw new PackageNotLoadedException('You need to install the package ' . $this->getPackageName() . ' to use this provider');
        }

        $this->parser = $parser;
    }

I moved the requested library into the vendor/ directory as its supposed to, but it didn't solve either.

I am on windows and it doesn't allow this kind of file naming structure piwik/device-detector, because i believe this is the directory that is being looked, and since it can not be located, an error is thrown.

Any suggestions?

Add tests from WhichBrowser

I've created a separate Composer package the tests I run for WhichBrowser. The package is called whichbrowser/testrunner and includes the runner.php script and all the yaml files. It is automatically updated whenever there is a new release of WhichBrowser.

If you want I can probably create a pull request for the actual extracting of the UA strings.

But it is also possible to extract it by using the runner.php script from the command line:

php vendor/whichbrowser/testrunner/runner.php list

or

php vendor/whichbrowser/testrunner/runner.php list television

or

php vendor/whichbrowser/testrunner/runner.php list mobile tablet television gaming

Provide an option, to disable generic remove

We remove currently some generics in various providers, to prevent the user from "false positive" detection results.
This plays also really well with the chain, since only then the next provider has the chance to give "good results"

But maybe some need to disabled this and want the original results at all time?

Add integration tests

We have true 100% code coverage in case of unittests, but the providers are all mocked.

So we need to add some tests, which check with the lowest and highest supported version of the provider, if they still work.

Any good ideas, how to implement this right?

Abstraction of device type?

Each provider has it's own device types..

BrowscapPhp

https://github.com/browscap/browscap/wiki/Resource:-Device-Database#device_type

Console
TV Device
Tablet
Mobile Phone
Mobile Device
Desktop
Ebook Reader
Car Entertainment System
Digital Camera

PiwikDeviceDetector

https://github.com/piwik/device-detector/blob/master/Parser/Device/DeviceParserAbstract.php#L42-L54

    protected static $deviceTypes = array(
        'desktop'               => self::DEVICE_TYPE_DESKTOP,
        'smartphone'            => self::DEVICE_TYPE_SMARTPHONE,
        'tablet'                => self::DEVICE_TYPE_TABLET,
        'feature phone'         => self::DEVICE_TYPE_FEATURE_PHONE,
        'console'               => self::DEVICE_TYPE_CONSOLE,
        'tv'                    => self::DEVICE_TYPE_TV,
        'car browser'           => self::DEVICE_TYPE_CAR_BROWSER,
        'smart display'         => self::DEVICE_TYPE_SMART_DISPLAY,
        'camera'                => self::DEVICE_TYPE_CAMERA,
        'portable media player' => self::DEVICE_TYPE_PORTABLE_MEDIA_PAYER,
        'phablet'               => self::DEVICE_TYPE_PHABLET
    );

UAParser

Can be some of those: https://raw.githubusercontent.com/ua-parser/uap-core/master/regexes.yaml

Generic Tablet
Generic Smartphone
Generic Feature Phone
Spider

WhichBrowser

https://github.com/WhichBrowser/WhichBrowser/blob/master/libraries/whichbrowser.php#L30-L48

define ('TYPE_DESKTOP', 'desktop');
define ('TYPE_MOBILE', 'mobile');
define ('TYPE_DECT', 'dect');
define ('TYPE_TABLET', 'tablet');
define ('TYPE_GAMING', 'gaming');
define ('TYPE_EREADER', 'ereader');
define ('TYPE_MEDIA', 'media');
define ('TYPE_HEADSET', 'headset');
define ('TYPE_WATCH', 'watch');
define ('TYPE_EMULATOR', 'emulator');
define ('TYPE_TELEVISION', 'television');
define ('TYPE_MONITOR', 'monitor');
define ('TYPE_CAMERA', 'camera');
define ('TYPE_SIGNAGE', 'signage');
define ('TYPE_WHITEBOARD', 'whiteboard');
define ('TYPE_GPS', 'gps');
define ('TYPE_CAR', 'car');
define ('TYPE_POS', 'pos');
define ('TYPE_BOT', 'bot');

Woothee

class DataSet
{
const DATASET_CATEGORY_PC          = 'pc';
const DATASET_CATEGORY_SMARTPHONE  = 'smartphone';
const DATASET_CATEGORY_MOBILEPHONE = 'mobilephone';
const DATASET_CATEGORY_CRAWLER     = 'crawler';
const DATASET_CATEGORY_APPLIANCE   = 'appliance';
const DATASET_CATEGORY_MISC        = 'misc';
}

Wurfl

http://web.wurfl.io/

form_factor
Desktop
App
Tablet
Smartphone
Feature Phone
Smart-TV
Robot
Other non-Mobile
Other Mobile

Yzalis

https://github.com/yzalis/UAParser/blob/master/src/UAParser/Result/DeviceResult.php#L111-L113

mobile
tablet
desktop

First results - comparing some vendors

Hello together,

i create this prototype project in order to try out the different UserAgent parsers available for PHP.

First i wanted to try if it's currently possible to create an abstract result set, like it's done for GeoIP already https://github.com/geocoder-php/Geocoder
Second i wanted to have a Chain Adapter on top it to get results from different vendors to get better results https://github.com/ThaDafinser/UserAgentParser/blob/master/bin/generateMatrix.php#L45-L53

Last but not least i wanted to compare the different implementations - how complete and good are they?

At this stage the results of the comparison are not completed, since some vendor adapter will be not complete or have errors.
But i still wanted to share those results to you all, to get (hopefully) some feedback about your Adapter implementation and get improvements https://github.com/ThaDafinser/UserAgentParser/tree/master/src/Provider

See the (current not correct or complete) results

Download the HTML files from here and open it just in a browser https://github.com/ThaDafinser/UserAgentParser/tree/master/data/results

It should look like this screenshot.
Click on "toggle" to see the original data and the result detail

image

Note about used datasets to generate the matrix

I currently did not used datasets from each vendor, so the comparison is not fair currently. (and of course missing convertions or errors)

Better would be an additional independent list...need to find a good source first

Next steps

  • get some feedback, about the provider adapter
  • complete and correct the provider adapters to get better results and a true comparison
  • refactor all code...it's ugly written...just a prototype
  • seperate UserAgentParser and the Matrix generation
  • release initial UserAgentParser
  • add more test datasets for the matrix
  • create a better comparison markup...the current on is not so "nice"
  • release a Matrix generator website

I hope it's ok to ping you all ...if not just tell me and i wont do it again ๐Ÿ‘
@mimmi20
@donatj
@sgiehl
@lstrojny
@NielsLeenheer
@yuya-takeyama
@blaugueux

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.