The openinsiderdata from sd3v

sd3v / openinsiderdata Goto Github PK

View Code? Open in Web Editor NEW

An Openinsider Scraper written in Python. Get the full Database behind the infamous SEC site :)

Python 93.49% Dockerfile 6.51%

openinsiderdata's People

Contributors

Stargazers

Watchers

openinsiderdata's Issues

If date has no data, scripts exits with error

See:

Traceback (most recent call last):
  File "/daily.py", line 76, in <module>
    get_openinsider_data()
  File "/daily.py", line 59, in get_openinsider_data
    all_data.extend(future.result())
                    ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/daily.py", line 35, in get_data_for_date
    rows = soup.find('table', {'class': 'tinytable'}).find('tbody').findAll('tr', recursive=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'find'

This can be fixed if this line is split into two and a None check is made:

    # Find all the rows in the table on the website
    rows = soup.find('table', {'class': 'tinytable'}).find('tbody').findAll('tr', recursive=False)

Like this:

    tinytable = soup.find('table', {'class': 'tinytable'})

    if tinytable is None
        print(f"No data for {date_string}")
    return data

    # Find all the rows in the table on the website
    rows = tinytable.find('tbody').findAll('tr', recursive=False)

Recommend Projects