harry-s-grewal / mls-real-estate-scraper-for-realtor.ca Goto Github PK

Hey there,

I'm new to scrapping and would like to learn how to run both the queries.py and realtorca.py.

I've run the queries.py first then realtorca.py, but a HTTPError came up... any pointers on how I can get this up and running to export to a csv file?

get_property_list_by_city("Calgary, AB")
Traceback (most recent call last):

Cell In[3], line 1
get_property_list_by_city("Calgary, AB")

File c:\users\kelvi\onedrive\desktop\web scrapper\mls-real-estate-scraper-for-realtor.ca-main\realtorca.py:13 in get_property_list_by_city
coords = get_coordinates(city) # Creates bounding box for city

File ~\OneDrive\Desktop\Web Scrapper\mls-real-estate-scraper-for-realtor.ca-main\queries.py:10 in get_coordinates
response.raise_for_status()

File ~\anaconda3\Lib\site-packages\requests\models.py:1021 in raise_for_status
raise HTTPError(http_error_msg, response=self)

HTTPError: 400 Client Error: Bad Request for url: https://nominatim.openstreetmap.org/search?q=Calgary,%20AB&format=json&country=Canada

I am working with your API locally.

When I first ran data = get_property_list_by_city("Toronto, Ontario"), it returned an error,

HTTPError: 403 Client Error: Forbidden for url: https://nominatim.openstreetmap.org/search?q=Toronto,%20Ontario&format=json

I reviewed the url by openstreetmap, it did provide me with the correct information each time I ran it, although it had returned a client error.

I manually copied the json results from the website to get the boundingbox values and saved it as coords, I ran

get_property_list( coords[0], coords[1], coords[2], coords[3], current_page=1)

Error 403: Rate limited

and as well as

When i ran get_property_list for the first time, I got a

Is this an active API that can be used by anyone?

Hi, thanks for this. Haven't tried this, but I'm using the RapidApi/ApiDojo endpoint for this data.

It looks like the CurrentPage value gets reset to 0 after 50 pages.

And the maximum records per page is 50. Anything above 50 is ignored. Anything below 50 is respected.

Is your experience the same as well?

please add json.dumps

As I was looking for a way to access realtor.ca data, I stumbled upon this awesome repo! I work for the Nature Conservancy of Canada where one of our goals is to purchase private land for conservation. Apologies for the specific question, but with this scraper you built, is it possible to pull data by municipality name, property type and land size? This API has serious potential to help us improve how we prioritize available private lands.

Thanks!

Hi I was unable to install it from pip due to it not having a setup.py file so I just ran it from my IDE.
This is my error:
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://nominatim.openstreetmap.org/search?q=Calgary,%20AB&format=json (I am running the example in the readme)

I have run it for the first time, all my versions match, do I need to register for the realtor.ca and get some kind of authorization ?
Not sure what to do lol.

Thanks

HTTPError: 403 Client Error: Forbidden for url: https://api2.realtor.ca/Listing.svc/PropertySearch_Post
Any idea how to fix this? Is this api from realtor.ca still working?