zoranpandovski / bookingscraper

:earth_americas: :hotel: Scrape Booking.com :hotel: :earth_americas:

bookingscraper's Issues

Add tests

We don't have any tests. Contributions are welcome

Fix Travis Build

https://travis-ci.org/ZoranPandovski/BookingScraper

Output refers to .xlsx file even when .txt file is generated

Regardless of the specified output format, the save_data function prints:

You can find them in hotels-in-macedonia.xlsx file

BookingScraper/booking.py

Lines 99 to 102 in 2206fe9

	wb.save('hotels-in-macedonia.xlsx')
	print('All accommodations are saved.')
	print('You can find them in hotels-in-macedonia.xlsx file')

Following the setup instructions leads to error

Traceback (most recent call last):
  File "booking.py", line 214, in <module>
    get_data(args.rooms, args.country, args.out_format, args.startdate, args.enddate)
  File "booking.py", line 153, in get_data
    save_data(hotels_list, out_format=out_format, country=country)
  File "booking.py", line 164, in save_data
    file = writer.output_file()
  File "C:\Projects\bookingscraper\file_writer.py", line 22, in output_file
    json.dump(list(self.data), outfile, indent=2, ensure_ascii=False)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\json\__init__.py", line 180, in dump
    fp.write(chunk)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u01c0' in position 23: character maps to <undefined>

Got this after python booking.py. Does it require any arguments or config changes?

Add option for choosing the number of rooms

TypeError: 'NoneType' object is not subscriptable

Hi there, Excellent work, thanks a lot!
I face an issue when running it though:

The error is:

Exception in thread Thread-18:
Traceback (most recent call last):
  File "C:\Anaconda3\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Anaconda3\Projects\Scraper\Booking.com\bthread.py", line 15, in run
    self.process_hotels(self.session, self.offset, self.rooms, self.country)
  File "booking.py", line 56, in process_hotels
    **name = ho.find('a', {'class': 'jq_tooltip'})['title']**
  File "C:\Anaconda3\lib\site-packages\bs4\element.py", line 1071, in __getitem__
    return self.attrs[key]
KeyError: 'title'

This error is thrown out 11 times, for different threads.
The line throwing the error seems to be: name = ho.find('a', {'class': 'jq_tooltip'})['title']

I ran the script booking.py
It produces however a file with a list of hotels in the country.

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Add option for choosing which country should be scrapped

Currently, booking URL is hardcoded, we need an option for choosing which country should be scrapped.

Bugged

/Documents/GitHub/BookingScraper/booking.py' && echo Exit status: $? && exit 1
Traceback (most recent call last):
File "/Documents/GitHub/BookingScraper/booking.py", line 214, in
get_data(args.rooms, args.country, args.out_format, args.startdate, args.enddate)
File "/Documents/GitHub/BookingScraper/booking.py", line 152, in get_data
hotels_list: list = prep_data(rooms, country, out_format, start_date, end_date)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/pascal/Documents/GitHub/BookingScraper/booking.py", line 122, in prep_data
parsed_html.find_all("li", {"class": "sr_pagination_item"})[-1]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
IndexError: list index out of range

Add option for choosing file format

Currently, all of the accommodations are saved inside as JSON inside a file, we need more options e.g(CSV, excel)

Traceback (most recent call last):

I'm trying to use it on Mac but I got the next errors

any solution? Thank you

Track more data points

Right now the scraper just gets names of the accommodations.

Would it be possible to start tracking more data points? E.g. price per night, amenities, location, ratings, reviews etc.

Add more tests

We need more tests. Contributions are welcome!

Help wanted

Congratulation for the nice work.
I was wondering how can it be added the city to the searching arguments?
Thank you in advance.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• Update dependency argcomplete to v1.12.3

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update dependency argcomplete to v3

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• Update dependency openpyxl to v2.6.4
• Update dependency openpyxl to v3

Detected dependencies

pip_requirements

requirements.txt

• openpyxl ==2.6.2
• argcomplete ==1.12.2

---

• Check this box to trigger a request for Renovate to run again on this repository