stegschreck / rats

Movie Ratings Synchronization with Python

RatS

This project serves for analyzing, and transferring your ratings from one movie tracking / rating website to another.

The goal of this project is to have a universal tool which can transfer your ratings from any site to another without the need of any manual steps like configuring an API access or whatever. Just configure your credentials (see steps below), start the tool and relax.

This also works if your lists are marked as private, as this tool uses a browser to login and get the content.

How do I use this?

Natively on the command line

This project is currently still under development. Please be patient, as I'm only working on this every once in a while.

1. Make sure you have Python3, Firefox and Xvfb installed on your system. This project is designed to run on Linux.

2. Checkout the project git clone https://github.com/StegSchreck/RatS.git && cd RatS

3. Install the requirements with pip for Python3 pip3 install -r requirements.txt

4. Install Geckodriver

   • Use your system's package manager (if it contains Geckodriver)
     • Arch Linux: pacman -S geckodriver
     • MacOS: brew install geckodriver
   • Or execute sudo ./InstallGeckodriver.sh. For this you will need to have tar, wget and curl installed.

5. Set your credentials

   1. Copy the credentials.cfg.orig file to credentials.cfg and insert your credentials for the sites you need there (without any quotation marks etc.).

      Copying the file will conserve the possibility to do a git pull later on without overwriting your credentials.

   2. Alternatively, you can set the credentials via environment variables. For example, if you want to set the credentials for your IMDB and Trakt accounts you would need to set these environment variables:

      export [email protected]
      export IMDB_PASWORD=def
      export TRAKT_USERNAME=abc
      export TRAKT_PASWORD=def

      It is important to use the variable names completely in uppercase!

   3. The third way is to set the environment variables when running the transfer script, like this: [email protected] IMDB_PASWORD=def TRAKT_USERNAME=abc TRAKT_PASWORD=def python3 transfer_ratings.py --source trakt --destination imdb

   The credentials in environment variables are overruling the ones in the credentials.cfg file.

6. Execute the script with Python3 python3 transfer_ratings.py --source trakt --destination movielens

   This will first parse your ratings in Trakt, save them in a JSON file for later use and then try to find those movies in Movielens an put your rating there. Notice: This will also overwrite rating you already did set there before.

   This script will take some minutes. Relax. You can follow the progress in console output.

   For more information about how to use the script, you can call python3 transfer_ratings.py --help

7. At the end, the script will print out how many movies were successfully posted. Afterward, all the movies which couldn't be found are printed out, so you can check them manually. The failed movie are also exported to a JSON file, so you can easily try them again (see below).

Inside a Docker container

Please note: This is currently not working on Windows. See issue #19

1. Create a credentials configuration in your home folder, e.g.: touch ~/.RatS.cfg

2. Configure your credentials in the file you just created, e.g.:

   [Trakt]
   USERNAME = abc
   PASSWORD = def
   

   Please see the RatS/credentials.cfg.orig in this repository as a template. Please enter your credentials as they are, without any quotation marks.

3. Get the Docker image: docker pull ghcr.io/stegschreck/rats/rats:latest

4. Run the script: docker run -it -v ~/.RatS.cfg:/RatS/RatS/credentials.cfg stegschreck/rats python3 transfer_ratings.py --source trakt --destination movielens

   The -v ~/.RatS.cfg:/RatS/RatS/credentials.cfg option will load the credentials file you just created from your home directory into the docker container, so that the script can use it.

   You will see the progress in your console. If you want to run this in the background, you can add the option -d to the docker run command to hide the output.

5. If you want to run the command again, simply run docker start -ai <container-id>. You can find the container id using docker ps -a or by running docker ps -q -l, if you haven't started any other containers in the meanwhile.

6. After the successful run of the transfer script, you may remove the docker container using docker rm <container-id>.

Command line call parameters

1. the first argument (--source) is the site where the ratings are parsed from (see Available Parsers)
2. the second argument (--destination) is the site where the ratings should be posted (inserted) to (see Available Inserters)

You can also omit the destination argument in order to just save the parsing results to a JSON file. You might insert the saved results anytime later. (see below)

Furthermore, you can define multiple destinations, e.g. like this: python3 transfer_ratings.py --source trakt --destination movielens --destination imdb in order to use the same data from the parser in this run for multiple destinations.

Mandatory

-s / --source: the site where the ratings are parsed from (see Available Parsers). This parameter must be given in even if you use the -f option.

Optional

-d / --destination: the site where the ratings should be posted (inserted) to (see Available Inserters). If you don't specify any destination, the ratings are only parsed and saved locally.

-D / --all-destinations: try to insert the parsed ratings to all configured sites for which valid credentials can be found

-f / --file: read ratings from a previous run instead of parsing them

-v / --verbose: increase output verbosity (i.e. print the generated predictions in the console)

-x / --show_browser: show the browser doing his work (this might help for debugging))

-h / --help: Display the help, including all possible parameter options

Currently Available Parsers

• AlloCine
• Criticker
• FilmAffinity
• Filmtipset
• Flixster deactivated until new account feature is implemented at Flixster
• iCheckMovies
  • please configure the conversion value for the like/dislike into a numeric rating yourself in the credentials.cfg
• IMDB (with IMDB account)
• Letterboxd
• Listal
• Movielens
• MoviePilot
• Plex (locally hosted server)
• Rotten Tomatoes
• TMDB (The Movie Database)
• Trakt

Currently Available Inserters

• AlloCine
• Criticker
• FilmAffinity
• Filmtipset
• Flixster deactivated until new account feature is implemented at Flixster
• iCheckMovies
  • please configure the boundaries for the ratings conversion yourself in the credentials.cfg
• IMDB (with IMDB account)
• Letterboxd
• Listal
• Metacritic
• Movielens
• MoviePilot
• Plex (locally hosted server)
• Rotten Tomatoes
• TMDB (The Movie Database)
• Trakt

Trying again with former export data

Depending on what you used in the first run:

... using the command line

You can re-use the data you parsed before, without parsing again. This will help you, if you want to distribute from one source to multiple destinations. The parser tells you in which file he saved his result, the folder is ./RatS/exports. You can use this data by calling the script for example this way:

python3 transfer_ratings.py --source trakt --destination movielens --file 20170721191143_Trakt.json

Please notice, that the --source argument is still needed in order to identify which data to use from the given file for the inserter.

Furthermore: This section is meant to be used with the native command line version. Docker containers work differently.

... using Docker

1. After the first run of the Docker container: (assuming you didn't start any other Docker containers in the meanwhile)

   • save the state of the container (including the saved data): docker commit <container-id> user/RatS
   • lookup the JSON file the script saved into the exports folder: docker run -it user/RatS ls -l RatS/exports/ (take the latest JSON file)

2. Run the container again, e.g. with a different destination docker run -it -v ~/.RatS.cfg:/RatS/RatS/credentials.cfg user/RatS python3 transfer_ratings.py --source trakt --destination imdb --file 20170721191143_Trakt.json

Switching between environments

You can copy the exported JSON from/to the Docker container if you like. I don't recommend this if you have no experience with docker. If you have, you already know what to do here ;)

Problem shooting

Script aborts because of Timeouts

It might occasionally happen, that the script runs into errors caused by the page loading too slow. In the past, this was quite an issue. I now tried to build a mechanism with dynamically increasing timeouts. Therefore, the script sleeps for one second after failing the first try (effective for multiple problematic locations in the code), waiting two seconds after the second try failed, and so on. The maximum number of tries is ten. If this is reached, the error gets re-raised and the script will fail.

Script aborts with WebDriverException

If you recently updated your Firefox, you might encounter the following exception during the login attempt of the parser:

selenium.common.exceptions.WebDriverException: Message: Expected [object Undefined] undefined to be a string

This can be fixed by installing the latest version of Mozilla's Geckodriver by running again the Install Geckodriver command mentioned above.

Script aborts with MoveTargetOutOfBoundsException while inserting to IMDB

IMDB shows special promotional banners for certain movies. If they are too big, they will cause the ratings component to be out of the viewport of the browser and the script fails with something like:

selenium.common.exceptions.MoveTargetOutOfBoundsException: Message: (620, 629) is out of bounds of viewport width (720) and height (538)

For now, the easiest solution is to run the script on your console with the -x flag. This will display the browser to you in a maximized window which then has a bigger viewport and prevent the error.

Login attempt does not work

This can have multiple explanations. One is, that you are using a password which starts or ends with a space character. RatS is currently not capable of dealing with that. If your credentials have a space character in the middle though, it will work fine.

Trakt aborts with AttributeError

While tying to login to Trakt, the script aborts with this error message:

===== Trakt: performing login
Traceback (most recent call last):
[...]
AttributeError: 'NoneType' object has no attribute 'find'

This was observed to happen when the credentials were configured to use the email address for login. Try to set your username instead.

You are missing a feature or noticed something is wrong?

Before submitting a bug ticket, I would ask you to check first whether all dependencies are up-to-date, especially geckodriver.

Please write a ticket, I will have a look as soon as I can.

Where does the name come from?

The name for this project comes from the first letters of "rating synchronisation". It is that simple. It is also a reference to the movie "Departed".

Acknowledgments

This project uses a free license for PyCharm by benefiting from JetBrains' open source community support program.

rats's Issues

Parse error

Hi. Just installed everything as wrote in the README. When i run the command i get the error written above.

System

• Version: Latest commit ed9f288
• Platform: Ubuntu 16.04.3 LTS
• Firefox version: 58.0 (x86)
• Python version: 3.5.2
• PIP version: 9.0.1
• Running in a virtualenv?: no
• Executed command: python3 /opt/RatS/transfer_ratings.py --source trakt --destination imdb

Stacktrace

:~$ python3 /opt/RatS/transfer_ratings.py --source trakt --destination imdb
Traceback (most recent call last):
  File "/opt/RatS/transfer_ratings.py", line 158, in <module>
    main()
  File "/opt/RatS/transfer_ratings.py", line 65, in main
    execute(args)
  File "/opt/RatS/transfer_ratings.py", line 110, in execute
    parser = get_parser_from_arg(args.source)(args)
  File "/opt/RatS/RatS/trakt/trakt_ratings_parser.py", line 9, in __init__
    super(TraktRatingsParser, self).__init__(Trakt(args), args)
  File "/opt/RatS/RatS/trakt/trakt_site.py", line 11, in __init__
    super(Trakt, self).__init__(args)
  File "/opt/RatS/RatS/base/base_site.py", line 35, in __init__
    self._init_browser()
  File "/opt/RatS/RatS/base/base_site.py", line 84, in _init_browser
    log_path="{timestamp}_geckodriver.log".format(timestamp=TIMESTAMP)
  File "/home/sugar/.local/lib/python3.5/site-packages/selenium/webdriver/firefox/webdriver.py", line 148, in __init__
    self.service.start()
  File "/home/sugar/.local/lib/python3.5/site-packages/selenium/webdriver/common/service.py", line 76, in start
    stdin=PIPE)
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error

Geckodriver log

0 byte, empty geckodriver logfiles created as the error occures.

Time prediction for parse / insert

print to console alongside with the progress bar

Improve Letterboxd Inserter progress feedback

• parse read progress
• parse process progress

print progress bar according to above values

Can't perform login to PLEX

Hello again, I'm trying to use Rats to transfer data from PLEX to TMDB but when executing the command

python3 transfer_ratings.py --source plex --destination tmdb

give me the following error

AttributeError: 'Plex' object has no attribute 'browser'
.

System

• Version: Latest
• Platform: Linux Ubuntu 16.04 (64 bits)
• Firefox version: 58.0.1 (64-bits)
• Geckdrover version: geckodriver 0.19.1
• Python version: Python 2.7.12
• Python3 version: Python 3.5.2
• PIP version: pip 9.0.1
• Running in a virtualenv?: no
• Running in Docker?: no
• Executed command: python3 transfer_ratings.py -s plex -d tmdb

Stacktrace

===== Plex: determine movie sectionTraceback (most recent call last):
  File "transfer_ratings.py", line 158, in <module>
    main()
  File "transfer_ratings.py", line 65, in main
    execute(args)
  File "transfer_ratings.py", line 110, in execute
    parser = get_parser_from_arg(args.source)(args)
  File "/home/miguel/RatS/RatS/plex/plex_ratings_parser.py", line 10, in __init__
    super(PlexRatingsParser, self).__init__(Plex(args), args)
  File "/home/miguel/RatS/RatS/plex/plex_site.py", line 15, in __init__
    super(Plex, self).__init__(args)
  File "/home/miguel/RatS/RatS/base/base_site.py", line 33, in __init__
    self._parse_configuration()
  File "/home/miguel/RatS/RatS/plex/plex_site.py", line 20, in _parse_configuration
    self.MOVIE_SECTION_ID = self._determine_movies_section_id()
  File "/home/miguel/RatS/RatS/plex/plex_site.py", line 32, in _determine_movies_section_id
    self.browser.get('http://{base_url}/library/sections'.format(base_url=self.BASE_URL))
AttributeError: 'Plex' object has no attribute 'browser'

refactor string formatings

• replace usages of "%s" % variable with "{}".format(variable)
• if multiple variables are needed, used named placeholders

see https://pyformat.info/ for more informations

Docker Image

Hi,

Thank you for that tool!
It will be great if you can create a docker image for that tool, so i can use it in my NAS.

www.icheckmovies.com ratings inserter

requested by @robot2xl

Handle captcha challenge in IMDB login

During the login to IMDB, it might happen, that a captcha is required.

• Research what causes this
  • maybe something in the login url?
• Enable user to enter this captcha in order to continue normal processing

Adjust the filename of the data parsed

You say: "You can use the data you parsed before again without parsing again. The parser tells you in which file he saved his results, the folder is ./RatS/exports. You can use this data by commenting out the parsing in the transfer_ratings.py script and comment in the file loader part, where you just have to adjust the filename."

To make use of a json saved file I understand the text should be like this:

def execute(args):
# PARSING DATA
#parser = get_parser_from_arg(args.source)(args)
#movies = parse_data_from_source(parser)
# FILE LOADING
movies = load_data_from_file(filename)
# POSTING THE DATA
inserter = get_inserter_from_arg(args.destination)(args)
insert_movie_ratings(inserter, movies, type(parser.site).name)

My question is, where in the text should I put the file name?

My file is called: 20170809172059_Criticker.json

Hopefully you can tell me where in the text.

Thank you.

plausibility check for iCheckMovies config

ensure the following constraints:

• INSERT_DISLIKE_UPPER_BOUND < INSERT_LIKE_LOWER_BOUND
• PARSE_DISLIKE_TRANSLATION < PARSE_LIKE_TRANSLATION

on failure:

• tell the user that he should reconsider the configuration
• abort further actions

GUI

To make this tool more useful for all users.

Possible solutions:

• Browser-based with e.g. flask as REST framework?
  • use WebComponents (Polymer)?
• PyGame application?

detect failed login (invalid credentials)

• tell user what happened
• abort further actions

Trakt: handle missing IMDB id

otherwise, the IMDB id might look like this:
find?q=Doctor+Who%3A+The+Snowmen&s=tt

• don't save this as id
• use regex for IMDB movie id extraction
• adjust IMDB inserter to handle those urls (or leave this field empty as well)

Let user enter credentials via command line

possible ways:

• as argument
• prompt for it when needed
• prompt for all needed credentials before starting

MoviePilot Inserter

https://www.moviepilot.de/

Use standard progress bar

in order to reduce project's complexity, just use an existing solution for displaying the progress, ETA etc.

possible alternatives:

• https://progressbar-2.readthedocs.io/en/latest/
• https://pypi.python.org/pypi/tqdm

UAC

• visual representation of progress (the actual progress bar)
• percentage finished so far
• ETA to finish
• (optional) processed items / total items
• (optional) time taken so far

Plex ratings inserter

Transfer tracking ("watched status")

possible ways:

• own transfer_trackings.py
• choose mode via command line argument
  e.g.
  python3 transfer.py -s trakt -d movielens -m tracking

error transfer_ratings.py

hello, when I execute the command "python3 transfer_ratings.py --source trakt --destination movielens" it returns the following error without the possibility of continuing

File "transfer_ratings.py", line 8, in
from RatS.criticker.criticker_ratings_inserter import CritickerRatingsInserter
File "/home/user/RatS/RatS/criticker/criticker_ratings_inserter.py", line 4, in
from bs4 import BeautifulSoup
ImportError: No module named 'bs4'

How to use on a headless server?

I want this so badly, and it is awesome, but how can one use it on a headless no gui server environment where nor xvfb nor xorg is installed?

Plex: Adapt to newer URL schema

http://{base_url}:{port}/library/sections gives an 401 HTTP error in Plex version 1.11.3.4803

Coverage data in CodeClimate

see:

• https://docs.codeclimate.com/docs/configuring-test-coverage
• https://docs.codeclimate.com/docs/travis-ci-test-coverage

Unable to locate element: [id="csv_file"]

Hey!

I finally had some luck with the Ubuntu Bash in W10 (while docker continues to fail).
I don't know what changed but now it parses (trakt) ratings correctly and generates the JSON, but then fails to pass them on:

python3 transfer_ratings.py --source trakt --destination tmdb
===== Trakt: Parsing 19 pages with 1134 movies in total
Trakt [#######################################################################################################] 100.0%

===== Trakt: saved 1134 parsed movies to /*/RatS/RatS/exports/20170818090253_Trakt.json
===== TMDB: posting 1134 movies
===== saving movies to CSV
Traceback (most recent call last):
  File "transfer_ratings.py", line 135, in <module>
    main()
  File "transfer_ratings.py", line 58, in main
    execute(args)
  File "transfer_ratings.py", line 110, in execute
    insert_movie_ratings(inserter, movies, type(parser.site).__name__)
  File "transfer_ratings.py", line 131, in insert_movie_ratings
    inserter.insert(movies, source)
  File "/mnt/c/Users/Sara/RatS/RatS/tmdb/tmdb_ratings_inserter.py", line 23, in insert
    self.upload_csv_file()
  File "/mnt/c/Users/Sara/RatS/RatS/tmdb/tmdb_ratings_inserter.py", line 37, in upload_csv_file
    self.site.browser.find_element_by_id('csv_file').send_keys(os.path.join(self.exports_folder, CSV_FILE_NAME))
  File "/home/ubu/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 289, in find_element_by_id
    return self.find_element(by=By.ID, value=id_)
  File "/home/ubu/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 791, in find_element
    'value': value})['value']
  File "/home/ubu/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
    self.error_handler.check_response(response)
  File "/home/ubu/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: [id="csv_file"]

Thanks for your efforts, I hope this can help you!

Plex ratings parser

local urls!

Evaluate Chrome headless instead of Firefox / Geckodriver

Handle missing credentials

if the user didn't configure credentials, don't do anything but tell the user to configure the credentials.

Can't perform login to IMDB

Cant export trakt to imdb

System

• Version: Latest commit 246ad94 4 days ago
• Platform: Ubuntu 17.10
• Firefox version: 58.0.1
• Geckdrover version: geckodriver 0.19.1
• Python version: Python 3.6.3
• PIP version: pip 9.0.1
• Running in a virtualenv?: yes
• Running in Docker?: no
• Executed command: python3 transfer_ratings.py --source trakt --destination imdb --file 20180129140242_Trakt.json -v

Stacktrace

python3 transfer_ratings.py --source trakt --destination imdb --file 20180129140242_Trakt.json -v
===== Trakt: performing login
===== loaded 586 movies from /home/rootz/RatS/RatS/exports/20180129140242_Trakt.json
===== IMDB: performing loginTraceback (most recent call last):
  File "transfer_ratings.py", line 158, in <module>
    main()
  File "transfer_ratings.py", line 65, in main
    execute(args)
  File "transfer_ratings.py", line 123, in execute
    inserter = get_inserter_from_arg(dest)(args)
  File "/home/rootz/RatS/RatS/imdb/imdb_ratings_inserter.py", line 14, in __init__
    super(IMDBRatingsInserter, self).__init__(IMDB(args), args)
  File "/home/rootz/RatS/RatS/imdb/imdb_site.py", line 15, in __init__
    super(IMDB, self).__init__(args)
  File "/home/rootz/RatS/RatS/base/base_site.py", line 35, in __init__
    self._init_browser()
  File "/home/rootz/RatS/RatS/base/base_site.py", line 98, in _init_browser
    self.login()
  File "/home/rootz/RatS/RatS/base/base_site.py", line 110, in login
    self._click_login_button()
  File "/home/rootz/RatS/RatS/base/base_site.py", line 142, in _click_login_button
    login_button.click()
  File "/home/rootz/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/home/rootz/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute
    return self._parent.execute(command, params)
  File "/home/rootz/.local/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
    self.error_handler.check_response(response)
  File "/home/rootz/.local/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 237, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: Element <input id="audio-captcha-play-pause-button" class="a-button-input" type="submit"> could not be scrolled into view

Geckodriver log

<!--
There is a {TIMESTAMP}_geckdrover.log file. Most of the time, this is not needed.
-->

Delete or document environment variables credentials

Problem with UTF8 from imdb to trakt

I follow the instruction and the program start. But I geht

System

• Version: commit 6314f86
• Platform: Ubuntu 18.04 LTS
• Firefox version: 59.0.2 (64-Bit)
• Geckdrover version: 0.20.1
• Python version: 2.7.15rc1
• PIP version: pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
• Running in a virtualenv?: no
• Running in Docker?: no
• Executed command: python3 transfer_ratings.py --source IMDB --destination trakt

Stacktrace

===== IMDB: CSV downloaded to /home/puchrojo/repositories/RatS/RatS/exports/20180510211929_IMDB.csv
===== getting movies from CSV
Traceback (most recent call last):
File "transfer_ratings.py", line 168, in
main()
File "transfer_ratings.py", line 65, in main
execute(args)
File "transfer_ratings.py", line 111, in execute
movies = execute_parsing(args, parser)
File "transfer_ratings.py", line 133, in execute_parsing
movies = parse_data_from_source(parser)
File "transfer_ratings.py", line 138, in parse_data_from_source
movies = parser.parse()
File "/home/puchrojo/repositories/RatS/RatS/base/base_ratings_parser.py", line 30, in parse
self._parse_ratings()
File "/home/puchrojo/repositories/RatS/RatS/imdb/imdb_ratings_parser.py", line 16, in _parse_ratings
self.movies = file_impex.load_movies_from_csv(os.path.join(self.exports_folder, self.csv_filename))
File "/home/puchrojo/repositories/RatS/RatS/utils/file_impex.py", line 44, in load_movies_from_csv
next(reader, None) # ignore csv header
File "/usr/lib/python3.6/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc1 in position 4999: invalid start byte

---

My rating list on IMDB ist public: https://www.imdb.com/user/ur5211117/ratings?ref_=uspf_rt

Vielen Dank!

Issue parsing non-ascii characters from CSV

Non-ascii character in movielens csv causes crash

The Naked Gun 33⅓: The Final Insult (1994)

System

• Version: Docker image ID: 3ec95fa75ecc
• Platform: Linux
• Running in Docker?: yes
• Executed command: python3 transfer_ratings.py --source movielens -v --destination trakt

Stacktrace

Traceback (most recent call last):
  File "transfer_ratings.py", line 158, in <module>
    main()
  File "transfer_ratings.py", line 65, in main
    execute(args)
  File "transfer_ratings.py", line 118, in execute
    movies = parse_data_from_source(parser)
  File "transfer_ratings.py", line 128, in parse_data_from_source
    movies = parser.parse()
  File "/RatS/RatS/base/base_ratings_parser.py", line 30, in parse
    self._parse_ratings()
  File "/RatS/RatS/movielens/movielens_ratings_parser.py", line 17, in _parse_ratings
    self.movies = self._parse_movies_from_csv(os.path.join(self.exports_folder, self.csv_filename))
  File "/RatS/RatS/base/base_ratings_downloader.py", line 76, in _parse_movies_from_csv
    return [self._convert_csv_row_to_movie(row) for row in reader]
  File "/RatS/RatS/base/base_ratings_downloader.py", line 76, in <listcomp>
    return [self._convert_csv_row_to_movie(row) for row in reader]
  File "/RatS/RatS/movielens/movielens_ratings_parser.py", line 30, in _convert_csv_row_to_movie
    sys.stdout.write(r + '\r\n')
UnicodeEncodeError: 'ascii' codec can't encode character '\u2153' in position 16: ordinal not in range(128)

AttributeError: 'NoneType' object has no attribute 'find_all'

Hi, I've been trying to get the script to work with no luck. I was able to circumnavigate certain errors by forcing the bash to use python3, otherwise this error qould appear:
TypeError: super() argument 1 must be type, not classobj

But in the end I got stuck here:

python3 transfer_ratings.py --source trakt --destination tmdb
===== Trakt: performing loginTraceback (most recent call last):
  File "./RatS/RatS/parsers/base_parser.py", line 21, in parse
    self._parse_ratings()
  File "./RatS/RatS/parsers/base_parser.py", line 31, in _parse_ratings
    pages_count = self._get_pages_count(movie_ratings_page)
  File "./RatS/RatS/parsers/trakt_parser.py", line 22, in _get_pages_count
    return int(movie_ratings_page.find(id='rating-items').
AttributeError: 'NoneType' object has no attribute 'find_all'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "transfer_ratings.py", line 120, in <module>
    main()
  File "transfer_ratings.py", line 53, in main
    execute(args)
  File "transfer_ratings.py", line 90, in execute
    movies = parse_data_from_source(parser)
  File "transfer_ratings.py", line 99, in parse_data_from_source
    movies = parser.parse()
  File "./RatS/RatS/parsers/base_parser.py", line 24, in parse
    self._parse_ratings()
  File "./RatS/RatS/parsers/base_parser.py", line 31, in _parse_ratings
    pages_count = self._get_pages_count(movie_ratings_page)
  File "./RatS/RatS/parsers/trakt_parser.py", line 22, in _get_pages_count
    return int(movie_ratings_page.find(id='rating-items').
AttributeError: 'NoneType' object has no attribute 'find_all'

And, since I'm no pro, I can't figure out any more fixes... Any idea what's happening?

www.icheckmovies.com ratings parser

Implement ratings parser for https://www.icheckmovies.com.
requested by @robot2xl

Create GitHub page

• usage documentation
• create custom domain?

CSV retrieval ends in Timeout, despite file being downloaded successfully

affecting at least Letterboxd, Criticker, Movielens

python3 transfer_ratings.py -s letterboxd  -d imdb -x                                               ✘  RatS 
===== Letterboxd: Retrieving ratings CSV fileTraceback (most recent call last):
  File "/home/developer/repos/RatS/RatS/base/base_ratings_downloader.py", line 30, in _download_ratings_csv
    self._call_download_url()
  File "/home/developer/repos/RatS/RatS/letterboxd/letterboxd_ratings_parser.py", line 39, in _call_download_url
    self.site.browser.get('https://letterboxd.com/data/export/')
  File "/home/developer/virtual_env/RatS/lib64/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 268, in get
    self.execute(Command.GET, {'url': url})
  File "/home/developer/virtual_env/RatS/lib64/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
    self.error_handler.check_response(response)
  File "/home/developer/virtual_env/RatS/lib64/python3.4/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timeout loading page after 10000ms


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "transfer_ratings.py", line 149, in <module>
    main()
  File "transfer_ratings.py", line 65, in main
    execute(args)
  File "transfer_ratings.py", line 118, in execute
    movies = parse_data_from_source(parser)
  File "transfer_ratings.py", line 128, in parse_data_from_source
    movies = parser.parse()
  File "/home/developer/repos/RatS/RatS/base/base_ratings_parser.py", line 22, in parse
    self._parse_ratings()
  File "/home/developer/repos/RatS/RatS/letterboxd/letterboxd_ratings_parser.py", line 17, in _parse_ratings
    self._download_ratings_csv()
  File "/home/developer/repos/RatS/RatS/base/base_ratings_downloader.py", line 33, in _download_ratings_csv
    self._call_download_url()
  File "/home/developer/repos/RatS/RatS/letterboxd/letterboxd_ratings_parser.py", line 39, in _call_download_url
    self.site.browser.get('https://letterboxd.com/data/export/')
  File "/home/developer/virtual_env/RatS/lib64/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 268, in get
    self.execute(Command.GET, {'url': url})
  File "/home/developer/virtual_env/RatS/lib64/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
    self.error_handler.check_response(response)
  File "/home/developer/virtual_env/RatS/lib64/python3.4/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timeout loading page after 10000ms

Implement version check on startup

• check the latest release under https://github.com/StegSchreck/RatS/releases/latest
• notify user via WARN log

MoviePilot Parser

https://www.moviepilot.de/

CritickerRatingsInserter: CSV upload

use CSV import instead of "manual" mode for speed improvement

handle TV show ratings

IMDB Inserter: out of bounds of viewport

System

• Version: commit 8bd88b9
• Platform: Manjaro Arch Linux
• Firefox version: 58.0.1
• Geckdrover version: 0.19.0
• Python version: 3.6.4
• PIP version: 9.0.1
• Running in a virtualenv?: yes
• Running in Docker?: no
• Executed command: python3 transfer_ratings.py -s trakt -d imdb

Stacktrace

Traceback (most recent call last):
  File "transfer_ratings.py", line 158, in <module>
    main()
  File "transfer_ratings.py", line 65, in main
    execute(args)
  File "transfer_ratings.py", line 124, in execute
    insert_movie_ratings(inserter, movies, type(parser.site).__name__)
  File "transfer_ratings.py", line 154, in insert_movie_ratings
    inserter.insert(movies, source)
  File "/home/sebastian/repos/RatS/RatS/base/base_ratings_inserter.py", line 41, in insert
    self._post_movie_rating(movie[source.lower()]['my_rating'])
  File "/home/sebastian/repos/RatS/RatS/base/base_ratings_inserter.py", line 129, in _post_movie_rating
    self._click_rating(my_rating)
  File "/home/sebastian/repos/RatS/RatS/imdb/imdb_ratings_inserter.py", line 49, in _click_rating
    .move_to_element(stars[star_index]).click(stars[star_index])\
  File "/home/sebastian/.virtualenvs/RatS/lib/python3.6/site-packages/selenium/webdriver/common/action_chains.py", line 80, in perform
    self.w3c_actions.perform()
  File "/home/sebastian/.virtualenvs/RatS/lib/python3.6/site-packages/selenium/webdriver/common/actions/action_builder.py", line 76, in perform
    self.driver.execute(Command.W3C_ACTIONS, enc)
  File "/home/sebastian/.virtualenvs/RatS/lib/python3.6/site-packages/selenium/webdriver/remote/webdriver.py", line 311, in execute
    self.error_handler.check_response(response)
  File "/home/sebastian/.virtualenvs/RatS/lib/python3.6/site-packages/selenium/webdriver/remote/errorhandler.py", line 237, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.MoveTargetOutOfBoundsException: Message: (620.5, 721) is out of bounds of viewport width (720) and height (545)

Merge multiple sources before inserting

parse from multiple sources, merge results (merge strategy?), then insert somewhere

e.g.
python3 transfer_ratings.py -s trakt -s imdb -d movielens

Listal export not saving anything/Letterboxd import ends in error

Hi. I'm running this on a VM using Ubuntu. That might have something to do with the issue. Anyhow. I can't seem to get the Listal->Letterboxd-conversion to work. As the log below notes the export doesn't save anything to the JSON-file though it seems to log in just fine, and the import ends in "raise exception_class(message, screen, stacktrace) selenium.common.exceptions.ElementNotInteractableException: Message: Element is not reachable by keyboard

EDIT: OK Now I did the -x using the command and it definitely scrolls through all the movies correctly, then logs in to Letterboxd and does nothing. As it doesn't actually save the movies it scrolls through.

I filled out all the details below to my best ability.

System

• Version: Latest as of 7/2/2018
• Platform: Ubuntu 16.04.03 X64
• Firefox version: Firefox 58.0.1
• Geckdrover version: 0.19.1
• Python version: 3.5.2
• PIP version: PIP 3 is 8.1.1 PIP is 9.0.1
• Running in a virtualenv?: Yes
• Running in Docker?: No
• Executed command: python3 transfer_ratings.py --source listal --destination letterboxd

Stacktrace

miikka@miikka-VirtualBox:~/RatS$ python3 transfer_ratings.py --source listal --destination letterboxd
===== Listal: Parsing 54 pages with 3188 movies in total
 
===== Listal: saved 0 parsed movies to /home/miikka/RatS/RatS/exports/20180207174432_Listal.json
===== Letterboxd: posting 0 movies
===== saving movies to CSV
Traceback (most recent call last):
  File "transfer_ratings.py", line 158, in <module>
    main()
  File "transfer_ratings.py", line 65, in main
    execute(args)
  File "transfer_ratings.py", line 124, in execute
    insert_movie_ratings(inserter, movies, type(parser.site).__name__)
  File "transfer_ratings.py", line 154, in insert_movie_ratings
    inserter.insert(movies, source)
  File "/home/miikka/RatS/RatS/letterboxd/letterboxd_ratings_inserter.py", line 33, in insert
    self.upload_csv_file(len(movies))
  File "/home/miikka/RatS/RatS/letterboxd/letterboxd_ratings_inserter.py", line 50, in upload_csv_file
    self.site.browser.find_element_by_id('upload-imdb-import').send_keys(os.path.join(filename))
  File "/home/miikka/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 479, in send_keys
    'value': keys_to_typing(value)})
  File "/home/miikka/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute
    return self._parent.execute(command, params)
  File "/home/miikka/.local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 312, in execute
    self.error_handler.check_response(response)
  File "/home/miikka/.local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: Element <input id="upload-imdb-import" name="file" type="file"> is not reachable by keyboard

check if phantomJS can be used for speed improvements

Transfer Collection

possible ways:

• own transfer_collection.py
• choose mode via command line argument
  e.g.
  python3 transfer.py -s trakt -d movielens -m collection

transfer watch list

Hi,

First, thank you for the Docker image!

Second, when i try to transfer my watch-list from IMDB to Trakt, the tool can't parse the movies from IMDB.
Is your code transferring also watch-lists? or only ranking movies?

I have 75 movies in my IMDB watch-list, here is the log:

ash-4.3# docker run -it -v /volume1/docker/rats/.RatS.cfg:/RatS/RatS/credentials.cfg stegschreck/rats python3 transfer_ratings.py --source imdb --destination trakt
===== IMDB: CSV downloaded to /RatS/RatS/exports/20170722183808_IMDB.csv
===== getting movies from CSV

===== IMDB: saved 0 parsed movies to /RatS/RatS/exports/20170722183808_IMDB.json
===== Trakt: posting 0 movies

===== Trakt: sucessfully posted 0 of 0 movies

Docker solution not working on Windows 10

reported by: @michomuchomacho:

I tried to import (with docker) to imdb and tmdb but got these errors:

TRAKT TO TMDB: docker run -it -v /.RatS.cfg:/RatS/RatS/credentials.cfg stegschreck/rats python3 transfer_ratings.py --source trakt --destination imdb

===== Trakt: Parsing 1 pages with 0 movies in total
===== Trakt: Parsing 1 pages with 0 movies in total 
Traceback (most recent call last): File "/RatS/RatS/base/base_ratings_parser.py", line 21, in parse self._parse_ratings() File "/RatS/RatS/base/base_ratings_parser.py", line 42, in _parse_ratings self._parse_movie_listing_page(movie_listing_page) File "/RatS/RatS/base/base_ratings_parser.py", line 56, in _parse_movie_listing_page movies_tiles = self._get_movie_tiles(movie_listing_page) File "/RatS/RatS/trakt/trakt_ratings_parser.py", line 30, in _get_movie_tiles return movie_listing_page.find(class_='row posters').find_all('div', attrs={'data-type': 'movie'}) AttributeError: 'NoneType' object has no attribute 'find_all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "transfer_ratings.py", line 125, in <module> main() File "transfer_ratings.py", line 56, in main execute(args) File "transfer_ratings.py", line 95, in execute movies = parse_data_from_source(parser) File "transfer_ratings.py", line 104, in parse_data_from_source movies = parser.parse() File "/RatS/RatS/base/base_ratings_parser.py", line 24, in parse self._parse_ratings() File "/RatS/RatS/base/base_ratings_parser.py", line 42, in _parse_ratings self._parse_movie_listing_page(movie_listing_page) File "/RatS/RatS/base/base_ratings_parser.py", line 56, in _parse_movie_listing_page movies_tiles = self._get_movie_tiles(movie_listing_page) File "/RatS/RatS/trakt/trakt_ratings_parser.py", line 30, in _get_movie_tiles return movie_listing_page.find(class_='row posters').find_all('div', attrs={'data-type': 'movie'}) AttributeError: 'NoneType' object has no attribute 'find_all'

TRAKT TO TMDB: docker run -it -v /.RatS.cfg:/RatS/RatS/credentials.cfg stegschreck/rats python3 transfer_ratings.py --source trakt --destination tmdb

===== Trakt: Parsing 1 pages with 0 movies in total 
===== Trakt: Parsing 1 pages with 0 movies in total 
Traceback (most recent call last): File "/RatS/RatS/base/base_ratings_parser.py", line 21, in parse self._parse_ratings() File "/RatS/RatS/base/base_ratings_parser.py", line 42, in _parse_ratings self._parse_movie_listing_page(movie_listing_page) File "/RatS/RatS/base/base_ratings_parser.py", line 56, in _parse_movie_listing_page movies_tiles = self._get_movie_tiles(movie_listing_page) File "/RatS/RatS/trakt/trakt_ratings_parser.py", line 30, in _get_movie_tiles return movie_listing_page.find(class_='row posters').find_all('div', attrs={'data-type': 'movie'}) AttributeError: 'NoneType' object has no attribute 'find_all' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "transfer_ratings.py", line 125, in <module> main() File "transfer_ratings.py", line 56, in main execute(args) File "transfer_ratings.py", line 95, in execute movies = parse_data_from_source(parser) File "transfer_ratings.py", line 104, in parse_data_from_source movies = parser.parse() File "/RatS/RatS/base/base_ratings_parser.py", line 24, in parse self._parse_ratings() File "/RatS/RatS/base/base_ratings_parser.py", line 42, in _parse_ratings self._parse_movie_listing_page(movie_listing_page) File "/RatS/RatS/base/base_ratings_parser.py", line 56, in _parse_movie_listing_page movies_tiles = self._get_movie_tiles(movie_listing_page) File "/RatS/RatS/trakt/trakt_ratings_parser.py", line 30, in _get_movie_tiles return movie_listing_page.find(class_='row posters').find_all('div', attrs={'data-type': 'movie'}) AttributeError: 'NoneType' object has no attribute 'find_all' 

Logging to file

create the possibility to log events to a log file

• configurable log levels?

consider using the standard logging lib from python.

configparser.InterpolationSyntaxError when importing to IMDb

Describe the bug
I get an error when trying to import from Trakt JSON file to IMDb

To Reproduce
Steps to reproduce the behavior:

1. I have obtained the JSON file from trakt using the script
2. I want to import the results from 1. to IMDb. I run the following:
   python3 transfer_ratings.py -s trakt -f 20180910123347_Trakt.json -d imdb -x

Expected behavior
I expect the information from the file to be imported in IMDb

Desktop (please complete the following information):

• Version: commit 0f62a35
• Platform OS: MacOS 10.13.6 (High Sierra)
• Firefox version: 62.0 (64-bit)
• Geckdrover version: 0.21.0
• Python version: 3.6.5
• PIP version: 18.0
• Running in a virtualenv?: no
• Running in Docker?: no
• Executed command: python3 transfer_ratings.py -s trakt -f 20180910123347_Trakt.json -d imdb -x

Stacktrace

$ python3 transfer_ratings.py -s trakt -f 20180910123347_Trakt.json -d imdb -x 
===== Trakt: performing login
===== loaded 285 movies from /Users/gaspar/RatS/RatS/exports/20180910123347_Trakt.json
Traceback (most recent call last):
  File "transfer_ratings.py", line 168, in <module>
    main()
  File "transfer_ratings.py", line 65, in main
    execute(args)
  File "transfer_ratings.py", line 112, in execute
    execute_inserting(args, movies, parser)
  File "transfer_ratings.py", line 122, in execute_inserting
    inserter = get_inserter_from_arg(dest)(args)
  File "/Users/gaspar/RatS/RatS/imdb/imdb_ratings_inserter.py", line 14, in __init__
    super(IMDBRatingsInserter, self).__init__(IMDB(args), args)
  File "/Users/gaspar/RatS/RatS/imdb/imdb_site.py", line 16, in __init__
    super(IMDB, self).__init__(args)
  File "/Users/gaspar/RatS/RatS/base/base_site.py", line 29, in __init__
    self._parse_credentials()
  File "/Users/gaspar/RatS/RatS/base/base_site.py", line 47, in _parse_credentials
    self.PASSWORD = self.config[self.site_name]['PASSWORD']
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/configparser.py", line 1234, in __getitem__
    return self._parser.get(self._name, key)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/configparser.py", line 800, in get
    d)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/configparser.py", line 394, in before_get
    self._interpolate_some(parser, option, L, value, section, defaults, 1)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/configparser.py", line 444, in _interpolate_some
    "found: %r" % (rest,))
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%'

Geckodriver log

1536744043951	geckodriver	INFO	geckodriver 0.21.0
1536744043959	geckodriver	INFO	Listening on 127.0.0.1:54295
1536744045183	addons.manager	DEBUG	Application has been upgraded
1536744045190	addons.manager	DEBUG	Loaded provider scope for resource://gre/modules/addons/XPIProvider.jsm: ["XPIProvider", "XPIInternal"]
1536744045194	addons.manager	DEBUG	Loaded provider scope for resource://gre/modules/LightweightThemeManager.jsm: ["LightweightThemeManager"]
1536744045206	addons.manager	DEBUG	Loaded provider scope for resource://gre/modules/addons/GMPProvider.jsm
1536744045207	addons.manager	DEBUG	Loaded provider scope for resource://gre/modules/addons/PluginProvider.jsm
1536744045208	addons.manager	DEBUG	Starting provider: XPIProvider
1536744045208	addons.xpi	DEBUG	startup
1536744045209	addons.xpi	INFO	SystemAddonLocation directory is missing
1536744045217	addons.xpi	INFO	Removing all system add-on upgrades.
1536744045218	addons.xpi	DEBUG	checkForChanges
1536744045218	addons.xpi	DEBUG	Loaded add-on state: ${}
1536744045218	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045219	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045220	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045220	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045221	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045221	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045221	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045222	addons.xpi	DEBUG	New add-on [email protected] in app-system-defaults
1536744045222	addons.xpi	DEBUG	scanForChanges changed: true, state: {}
1536744045226	addons.xpi-utils	DEBUG	Error: Synchronously loading the add-ons database (resource://gre/modules/addons/XPIDatabase.jsm:1329:15) JS Stack trace: [email protected]:1329:15
[email protected]:2533:9
[email protected]:2116:25
[email protected]:206:12
[email protected]:654:5
[email protected]:813:9
[email protected]:2811:5
[email protected]:66:9
1536744045227	addons.xpi-utils	DEBUG	Starting async load of XPI database /var/folders/rp/h562wd515yl5k7j43tyk9phr0000gn/T/rust_mozprofile.59u8KYqVEzeZ/extensions.json
1536744045334	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
*** Blocklist::_preloadBlocklistFile: blocklist is disabled
1536744045354	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
1536744045358	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
1536744045361	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
1536744045365	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
1536744045367	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
1536744045370	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
1536744045373	addons.xpi-utils	DEBUG	New add-on [email protected] installed in app-system-defaults
1536744045460	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045464	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045465	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 2018.08.22.1219-93becf29
1536744045465	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045466	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{dd982bd7-0c06-f545-a789-616c955cf94e}","version":"2018.08.22.1219-93becf29","type":"extension","defaultLocale":{"name":"Activity Stream","description":"A rich visual history feed and a reimagined home page make it easier than ever to find exactly what you're looking for in Firefox."},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994921,"updateDate":1536743994921,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045466	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045466	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045467	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 2.0
1536744045467	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045467	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{9a4685a1-cb89-d34c-82f2-d4405e38b6ad}","version":"2.0","type":"extension","defaultLocale":{"name":"Application Update Service Helper","description":"Sets value(s) in the update url based on custom checks."},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994901,"updateDate":1536743994901,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045467	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045467	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045471	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 1.0.5
1536744045471	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045471	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{3ddb1a15-2f3a-8b49-9d46-9bbb06b71b5e}","version":"1.0.5","type":"extension","defaultLocale":{"name":"Pocket","description":"When you find something you want to view later, put it in Pocket."},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994900,"updateDate":1536743994900,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045471	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045471	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045472	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 1.0
1536744045472	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045472	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{dc955496-81d6-734d-afdf-4ce5de6893f8}","version":"1.0","type":"extension","defaultLocale":{"name":"Form Autofill","description":"Autofill forms with saved profiles"},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994893,"updateDate":1536743994893,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045472	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045472	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045474	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 1.0
1536744045474	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045474	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{2696d0ee-7720-6a4e-8aee-8b794fd64e90}","version":"1.0","type":"extension","defaultLocale":{"name":"Photon onboarding","description":"Photon onboarding"},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994888,"updateDate":1536743994888,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045474	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045474	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045475	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 33.0.0
1536744045475	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045476	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{2ee8589c-8b0a-5240-a7a8-6522c7acd2dc}","version":"33.0.0","type":"extension","defaultLocale":{"name":"Firefox Screenshots","homepageURL":"https://screenshots.firefox.com/"},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994882,"updateDate":1536743994882,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"57.0a1","maxVersion":"*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045476	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045476	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045476	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 1.0.0
1536744045476	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045477	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{162b5d05-562d-e047-b9a4-9424ca37f86d}","version":"1.0.0","type":"extension","defaultLocale":{"name":"WebCompat Reporter","description":"Report site compatibility issues on webcompat.com."},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994875,"updateDate":1536743994875,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045477	addons.manager	DEBUG	Registering startup change 'installed' for [email protected]
1536744045477	addons.xpi	DEBUG	Loading bootstrap scope from /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045477	addons.xpi	DEBUG	Calling bootstrap method install on [email protected] version 2.0
1536744045478	addons.xpi-utils	DEBUG	Make addon app-system-defaults:[email protected] visible
1536744045478	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{8f5dc9c6-e191-6c4c-9699-c6d2cfa9f308}","version":"2.0","type":"extension","defaultLocale":{"name":"Web Compat","description":"Urgent post-release fixes for web compatibility."},"visible":true,"active":false,"userDisabled":false,"appDisabled":false,"installDate":1536743994873,"updateDate":1536743994873,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"},{"id":"{aa3c5121-dab2-40e2-81ca-7ea25febc110}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":true,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045478	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{dd982bd7-0c06-f545-a789-616c955cf94e}","version":"2018.08.22.1219-93becf29","type":"extension","defaultLocale":{"name":"Activity Stream","description":"A rich visual history feed and a reimagined home page make it easier than ever to find exactly what you're looking for in Firefox."},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994921,"updateDate":1536743994921,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045478	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{9a4685a1-cb89-d34c-82f2-d4405e38b6ad}","version":"2.0","type":"extension","defaultLocale":{"name":"Application Update Service Helper","description":"Sets value(s) in the update url based on custom checks."},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994901,"updateDate":1536743994901,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045478	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{3ddb1a15-2f3a-8b49-9d46-9bbb06b71b5e}","version":"1.0.5","type":"extension","defaultLocale":{"name":"Pocket","description":"When you find something you want to view later, put it in Pocket."},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994900,"updateDate":1536743994900,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045478	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{dc955496-81d6-734d-afdf-4ce5de6893f8}","version":"1.0","type":"extension","defaultLocale":{"name":"Form Autofill","description":"Autofill forms with saved profiles"},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994893,"updateDate":1536743994893,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045478	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{2696d0ee-7720-6a4e-8aee-8b794fd64e90}","version":"1.0","type":"extension","defaultLocale":{"name":"Photon onboarding","description":"Photon onboarding"},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994888,"updateDate":1536743994888,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045478	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{2ee8589c-8b0a-5240-a7a8-6522c7acd2dc}","version":"33.0.0","type":"extension","defaultLocale":{"name":"Firefox Screenshots","homepageURL":"https://screenshots.firefox.com/"},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994882,"updateDate":1536743994882,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"57.0a1","maxVersion":"*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045479	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{162b5d05-562d-e047-b9a4-9424ca37f86d}","version":"1.0.0","type":"extension","defaultLocale":{"name":"WebCompat Reporter","description":"Report site compatibility issues on webcompat.com."},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994875,"updateDate":1536743994875,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":false,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045479	addons.xpi	DEBUG	Updating XPIState for {"id":"[email protected]","syncGUID":"{8f5dc9c6-e191-6c4c-9699-c6d2cfa9f308}","version":"2.0","type":"extension","defaultLocale":{"name":"Web Compat","description":"Urgent post-release fixes for web compatibility."},"visible":true,"active":true,"userDisabled":false,"appDisabled":false,"installDate":1536743994873,"updateDate":1536743994873,"applyBackgroundUpdates":1,"path":"/Applications/Firefox.app/Contents/Resources/browser/features/[email protected]","skinnable":false,"sourceURI":null,"releaseNotesURI":null,"softDisabled":false,"foreignInstall":false,"strictCompatibility":false,"locales":[],"targetApplications":[{"id":"{ec8030f7-c20a-464f-9b0e-13a3a9e97384}","minVersion":"62.0","maxVersion":"62.*"},{"id":"{aa3c5121-dab2-40e2-81ca-7ea25febc110}","minVersion":"62.0","maxVersion":"62.*"}],"targetPlatforms":[],"seen":true,"dependencies":[],"hasEmbeddedWebExtension":true,"userPermissions":null,"icons":{},"blocklistState":0,"blocklistURL":null,"startupData":null,"location":"app-system-defaults"}
1536744045479	addons.xpi-utils	DEBUG	Updating add-on states
1536744045481	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045481	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 2018.08.22.1219-93becf29
1536744045482	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045482	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 2.0
1536744045482	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045483	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 1.0.5
1536744045487	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045487	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 1.0
1536744045487	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045487	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 1.0
1536744045488	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045488	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 33.0.0
1536744045488	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045488	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 1.0.0
1536744045488	addons.xpi	DEBUG	Registering manifest for /Applications/Firefox.app/Contents/Resources/browser/features/[email protected]
1536744045489	addons.xpi	DEBUG	Calling bootstrap method startup on [email protected] version 2.0
1536744045490	addons.manager	DEBUG	Registering shutdown blocker for XPIProvider
1536744045490	addons.manager	DEBUG	Provider finished startup: XPIProvider
1536744045490	addons.manager	DEBUG	Starting provider: LightweightThemeManager
1536744045490	addons.manager	DEBUG	Registering shutdown blocker for LightweightThemeManager
1536744045491	addons.manager	DEBUG	Provider finished startup: LightweightThemeManager
1536744045491	addons.manager	DEBUG	Starting provider: GMPProvider
1536744045492	addons.manager	DEBUG	Registering shutdown blocker for GMPProvider
1536744045492	addons.manager	DEBUG	Provider finished startup: GMPProvider
1536744045492	addons.manager	DEBUG	Starting provider: PluginProvider
1536744045492	addons.manager	DEBUG	Registering shutdown blocker for PluginProvider
1536744045492	addons.manager	DEBUG	Provider finished startup: PluginProvider
1536744045493	addons.manager	DEBUG	Completed startup sequence
1536744045774	addons.repository	DEBUG	No addons.json found.
2018-09-12 11:20:46.156 plugin-container[18327:2631180] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x7f3f, name = 'com.apple.tsm.portname'
See /usr/include/servers/bootstrap_defs.h for the error codes.
2018-09-12 11:20:46.255 plugin-container[18327:2631180] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x7f1f, name = 'com.apple.coredrag'
See /usr/include/servers/bootstrap_defs.h for the error codes.
2018-09-12 11:20:46.280 plugin-container[18328:2631210] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x8d43, name = 'com.apple.tsm.portname'
See /usr/include/servers/bootstrap_defs.h for the error codes.
2018-09-12 11:20:46.364 plugin-container[18328:2631210] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x8d1f, name = 'com.apple.coredrag'
See /usr/include/servers/bootstrap_defs.h for the error codes.
1536744046909	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046909	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046909	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046909	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046909	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046910	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046910	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046910	addons.xpi	DEBUG	Existing add-on [email protected] in app-system-defaults
1536744046910	addons.xpi	DEBUG	scanForChanges changed: false, state: {}
1536744047040	addons.xpi-utils	DEBUG	XPI Database saved, setting schema version preference to 27
1536744047140	Marionette	WARN	TLS certificate errors will be ignored for this session
1536744047197	Marionette	DEBUG	[4294967297] Frame script loaded
1536744047198	Marionette	DEBUG	[4294967297] Frame script registered
1536744047240	Marionette	DEBUG	[4294967297] Received DOM event beforeunload for about:blank
1536744047485	Marionette	DEBUG	[4294967297] Received DOM event pagehide for about:blank
2018-09-12 11:20:47.601 plugin-container[18329:2631350] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x8803, name = 'com.apple.tsm.portname'
See /usr/include/servers/bootstrap_defs.h for the error codes.
2018-09-12 11:20:47.751 plugin-container[18329:2631350] *** CFMessagePort: bootstrap_register(): failed 1100 (0x44c) 'Permission denied', port = 0x889f, name = 'com.apple.coredrag'
See /usr/include/servers/bootstrap_defs.h for the error codes.
1536744047989	Marionette	DEBUG	[4294967297] Received DOM event DOMContentLoaded for https://trakt.tv/auth/signin
1536744049245	Marionette	DEBUG	[4294967297] Received DOM event pageshow for https://trakt.tv/auth/signin
JavaScript error: https://trakt.tv/assets/application-c70888efc837f390a33ab997ea1c4e69.js, line 1: TypeError: $(...).val(...) is undefined
1536744050367	Marionette	DEBUG	[4294967297] Received DOM event beforeunload for https://trakt.tv/auth/signin
1536744051021	Marionette	DEBUG	[4294967297] Received DOM event pagehide for https://trakt.tv/auth/signin
1536744051022	Marionette	DEBUG	[4294967297] Received DOM event unload for https://trakt.tv/auth/signin
1536744051297	Marionette	DEBUG	[4294967297] Received DOM event DOMContentLoaded for https://trakt.tv/dashboard
1536744051525	Marionette	DEBUG	[4294967297] Received observer notification outer-window-destroyed for 4294967302
1536744053192	Marionette	DEBUG	[4294967297] Received DOM event pageshow for https://trakt.tv/dashboard
1536744055213	Marionette	DEBUG	[4294967297] Received DOM event beforeunload for https://trakt.tv/dashboard
1536744055493	Marionette	DEBUG	[4294967297] Received DOM event pagehide for https://trakt.tv/dashboard
1536744055494	Marionette	DEBUG	[4294967297] Received DOM event unload for https://trakt.tv/dashboard
1536744055915	Marionette	DEBUG	[4294967297] Received DOM event DOMContentLoaded for https://trakt.tv/users/gasbi/ratings/movies/all/added
1536744056386	Marionette	DEBUG	[4294967297] Received observer notification outer-window-destroyed for 4294967303
1536744057384	Marionette	DEBUG	[4294967297] Received DOM event pageshow for https://trakt.tv/users/gasbi/ratings/movies/all/added
JavaScript error: https://trakt.tv/users/gasbi/ratings/movies/all/added, line 1: TypeError: top is null
JavaScript error: https://trakt.tv/users/gasbi/ratings/movies/all/added, line 1: TypeError: top is null
1536744057443	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 2.0
1536744057443	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 1.0.0
1536744057444	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 33.0.0
1536744057448	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 1.0
1536744057448	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 1.0
1536744057449	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 1.0.5
1536744057449	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 2.0
1536744057450	addons.xpi	DEBUG	Calling bootstrap method shutdown on [email protected] version 2018.08.22.1219-93becf29
1536744057528	addons.manager	DEBUG	shutdown
1536744057528	addons.manager	DEBUG	Calling shutdown blocker for XPIProvider
1536744057529	addons.xpi	DEBUG	shutdown
1536744057529	addons.xpi-utils	DEBUG	shutdown
1536744057529	addons.manager	DEBUG	Calling shutdown blocker for LightweightThemeManager
1536744057529	addons.manager	DEBUG	Calling shutdown blocker for GMPProvider
1536744057529	addons.manager	DEBUG	Calling shutdown blocker for PluginProvider
1536744057534	addons.manager	DEBUG	Async provider shutdown done

Additional context
I am using the -x option because otherwise the error requesting Xvfb pops out. I haven't found a way to install Xvfb in MacOS High Sierra, so I just opted to view the firefox window.

Insert to multiple destinations

Give the possibility to define multiple destinations in the command line arguments in order to insert to them during the same script run

e.g.
python3 transfer_ratings.py -s trakt -d movielens -d tmdb

Exception handling - increasing timeouts

Implement a exception handling, that does not only try again once, but increases timeouts on every failed try.
There should be a maximum and exit strategy.

example code:

    result = False
    iteration = 0
    while not result:
        try:
            result = perform_action()
        except:
            iteration += 1
            sys.stdout.write('.')
            sys.stdout.flush()
            time.sleep(iteration * 1)
            continue
    sys.stdout.write('\n')

error transfer from trakt to imdb

hello, i can't get the script to work. i already reinstall everything, no luck. this is the error message :

===== Trakt: saved 558 parsed movies to /home/rudiecantfail/RatS/RatS/exports/20171129050104_Trakt.json
===== IMDB: posting 558 movies
===== IMDB: posted The Other End of the Line (2008)
Traceback (most recent call last):
File "/home/rudiecantfail/RatS/RatS/base/base_ratings_inserter.py", line 104, in _post_movie_rating
self._click_rating(my_rating)
File "/home/rudiecantfail/RatS/RatS/imdb/imdb_ratings_inserter.py", line 45, in _click_rating
stars[star_index].click()
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webelement.py", line 501, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/rudiecantfail/RatS/transfer_ratings.py", line 149, in
main()
File "/home/rudiecantfail/RatS/transfer_ratings.py", line 65, in main
execute(args)
File "/home/rudiecantfail/RatS/transfer_ratings.py", line 124, in execute
insert_movie_ratings(inserter, movies, type(parser.site).name)
File "/home/rudiecantfail/RatS/transfer_ratings.py", line 145, in insert_movie_ratings
inserter.insert(movies, source)
File "/home/rudiecantfail/RatS/RatS/base/base_ratings_inserter.py", line 35, in insert
self._post_movie_rating(movie[source.lower()]['my_rating'])
File "/home/rudiecantfail/RatS/RatS/base/base_ratings_inserter.py", line 107, in _post_movie_rating
self._click_rating(my_rating)
File "/home/rudiecantfail/RatS/RatS/imdb/imdb_ratings_inserter.py", line 45, in _click_rating
stars[star_index].click()
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webelement.py", line 80, in click
self._execute(Command.CLICK_ELEMENT)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webelement.py", line 501, in _execute
return self._parent.execute(command, params)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.5/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: