arshka / linkedin-job-scraper Goto Github PK

When there's too many job postings from the same company at the time of job searching, that company will be added multiple times when executing details retriever.

1. There should be a check to prevent adding it again if the time is too short, or
2. We could use a separate script file to remove duplicated data and reorder the primary key, with time (e.g. in days) as the parameter. This should be only executed after everything's done since it's expensive if executed every time a data is received.

See screenshot below:

Hi @ArshKA,

I am from Proxycurl, we came across your repo and we would like to sponsor you.

Are you keen?

When running search_retriever.py,
A browser is opened and it logins automatically.
A few seconds after logging in, the program prompts me to press enter to continue.
After pressing enter, the browser loads the linked job search page, and then closes itself.

Traceback (most recent call last):
  File "LinkedIn-Job-Scraper/search_retriever.py", line 24, in <module>
    all_results = job_searcher.get_jobs()
  File "LinkedIn-Job-Scraper/fetch.py", line 62, in get_jobs
    raise Exception('Status code {} for search\nText: {}'.format(results.status_code, results.text))
Exception: Status code 400 for search

Would you mind helping me with this please? Thank you.