ryansherby / redfinscraper

Scrapes Redfin data.

An idea that has gone unseized,
Is a forest without any trees,
It exists as a ghost,
With no one to host,
Lost like a breath to the breeze,

An idea that rests in the hand,
Is a seed to be sown in the land,
As a sprout it will grow,
With a spark melting snow,
Shone brightly from your lantern of gold,

Soon trees will dawn,
Bark thick and of brawn,
A forest built to be old

redfinscraper's Issues

API Results Limited to 350 Entries Regardless of Time Span

`
scraper = RedfinScraper()
scraper.setup(zip_database_path, multiprocessing=False)
zip_codes = ['85255']

for zip_code in zip_codes:
        data = scraper.scrape(zip_codes=[zip_code], sold=True, sale_period='1yr') # change to 3yr
        data.to_csv(os.path.join(output_dir, f'scraped_data_{zip_code}.csv'), index=False)`

--
Steps to Reproduce:

1. Set up the scraper with a specific zip code.
2. Scrape data for a 1-year time span.
3. Observe the number of results (350 in my case).
4. Increase the time span to 3 years.
5. Scrape data again for the same zip code.
6. Observe the number of results (still 350).

---

Any insights or workarounds would be appreciated. Thank you!

Time On Market

Great project. For Sold properties, is is possible to obtain List Price?

Is there documentation for the Stingray API anywhere?

inability to list pending sales

Pending sales are not included in either sold=False or sold=True lists. Other projects working with stingray suggest there is also a pending status. Can this be catered for?

id	status
1	active
130	pending
131	active + pending

https://github.com/alientechsw/RedfinPlus/blob/master/docs/REDFIN.md#status

Additional fields

Is it possible to have this additionally scrape the listing text?

Downloads partial data:

Sample code block where RedfinScraper does not work:

from redfin_scraper import RedfinScraper
scraper = RedfinScraper()

scraper.setup('zip_code_database.csv', False)
print(scraper.scrape(zip_codes=['78704']))
print(scraper.scrape(city_states=['Austin, TX']))

Config changes

Can the base URLs in the rsc config be changed? Meaning, if we wanted to view sold properties vs just for sale properties, could we change the base url? I think it needs to be of the shape https://www.redfin.com/zipcode/<zip_code>/filter/include=sold-<some_number>mo

reddit API limit? (building a stream)

the smallest time unit supported by this app is 1mo...is that a limit built into the reddit API?

for a class, I'm trying to find examples of real data coming in a stream; so, it would be grand if I could grab lots of zip codes, but for only a short period of time (day, hour, ??).