mickaelwalter / wp-json-scraper Goto Github PK

$ python wp-json-scraper/WPJsonScraper.py https://somepage.com/ \
--export-posts somepage/posts \
--export-pages somepage/pages \
--export-comments somepage/comments \
--download-media somepage/media

[*] Testing connectivity with the server
[+] Connection OK
Total number of entries: 2205
 |██████████████████████████████████████████████████████████████████████| 100.0% 
Total number of entries: 64
 |██████████████████████████████████████████████████████████████████████| 100.0% 
Total number of entries: 27
 |██████████████████████████████████████████████████████████████████████| 100.0% 
Total number of entries: 2
 |██████████████████████████████████████████████████████████████████████| 100.0% 

[+] Exported 2205 posts to somepage/posts
Total number of entries: 78
 |██████████████████████████████████████████████████████████████████████| 100.0% 

[+] Exported 78 pages to somepage/pages
Total number of entries: 2205
 |██████████████████████████████████████████████████████████████████████| 100.0% 
Traceback (most recent call last):
  File "wp-json-scraper/WPJsonScraper.py", line 394, in <module>
    main()
  File "wp-json-scraper/WPJsonScraper.py", line 365, in main
    post_list = scanner.get_posts(True)
  File "wp-json-scraper/lib/wpapi.py", line 394, in get_posts
    self.posts = self.update_cache(self.posts, posts, total_entries, start, num)
  File "wp-json-scraper/lib/wpapi.py", line 345, in update_cache
    if start >= total_entries:
TypeError: '>=' not supported between instances of 'NoneType' and 'int'

Traceback (most recent call last):
  File "WPJsonScraper.py", line 31, in <module>
    from lib.wpapi import WPApi
  File "/Users/Nirawit/Desktop/wp-json-scraper-0.4/lib/wpapi.py", line 26, in <module>
    from urllib.parse import urlencode
ImportError: No module named parse

Traceback (most recent call last):
  File ".\WPJsonScraper.py", line 365, in <module>
    main()
  File ".\WPJsonScraper.py", line 231, in main
    basic_info = scanner.get_basic_info()
  File "C:\Users\User\Desktop\wp-json-scraper-master\wp-json-scraper-master\lib\wpapi.py", line 89, in get_basic_info
    self.basic_info = req.json().read().decode('utf-8-sig')
  File "C:\python3\lib\site-packages\requests\models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\python3\lib\json\__init__.py", line 337, in loads
    raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",
json.decoder.JSONDecodeError: Unexpected UTF-8 BOM (decode using utf-8-sig): line 1 column 1 (char 0)