Python library for working with SEC Edgar
License: MIT License
Email me at [email protected] or find me on LinkedIn
it shows error "cannot import name 'set_identity' from 'edgar'" I tried using this on both jupter notebook & google colab. My python version on google colab is Python 3.10.12. & on jupyter is Python 3.11.7 . Please let me know how to resolve this. Edit: working now
When displaying html sometimes after a heading there is no line break so the header text continues into the next line.
This happens in a Press Release but probably also with html()
To detect issues earlier and across a wide range of filings we need to sample filings across years and filing times.
Implement a series of GitHub actions that sample filings and run test.
I am trying to run Company(ticker).get_filings(form=["10-K","10-Q"]).filter(date=f'{start_date}:{end_date}') for around 1000 stocks using ProcessPoolExecutor. I seem to be getting error 429 even by setting the max_workers = 8 (SEC says it allows for 10 requests every second). I was wondering if 1. form=["10-K","10-Q"] = 1 or 2 requests and if 2. there was a better way to send the requests i.e accessing a list of company fillings AFTER importing all of the SEC filings at once (one request). Any help would be appreciated!
Hello I'm back 😅
I seem to have found another edge case for strip_ixbrl_tags.
edgartools version: 2.8.1
import edgar
edgar.set_identity("id")
company = edgar.Company("ABNB") # Ticker is important, not all filings run into this issue
filings = company.get_filings(form=["10-K"])
if filings:
print(filings[0].text()) Stacktrace
Traceback (most recent call last):
File "/workspace/test.py", line 10, in <module>
print(filings[0].text())
^^^^^^^^^^^^^^^^^
File "/workspace/.venv/lib/python3.11/site-packages/edgar/_filings.py", line 1512, in text
return html_to_text(html_content, ignore_tables=ignore_tables, sep=sep)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/.venv/lib/python3.11/site-packages/edgar/htmltools.py", line 87, in html_to_text
html_str = try_to_strip_ixbrl_tags(html_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/.venv/lib/python3.11/site-packages/edgar/htmltools.py", line 176, in try_to_strip_ixbrl_tags
return strip_ixbrl_tags(html_content)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/.venv/lib/python3.11/site-packages/edgar/htmltools.py", line 191, in strip_ixbrl_tags
if parent.tag == '{http://www.w3.org/1999/xhtml}div' and 'display:inline' in parent.get('style'):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable
Finish and publish edgartools documentation.
The WIP in progress is in mkdocs hosted on the gh-pages branch
I think press releases are typically encoded as EX 99.1, which is an attachment of 8-K. This worked for me in ~200 out of 500 companies, by iterating through every 8-K and checking for that - but the other ones were missing
Is there an easier way to get the latest press release per company?
There are functions to get the latest 8-K but this doesn't always have the correct attachment
users will have authors rate limiting applied to their own use It should use the user's user agent from set_identity
edgar/_filings.py has line 81 has the authors user-agent identity hard coded.
same issue is in tests/test_filing.py on line 214
When trying to get the data from the income statement in the latest 10-K form from AMD, I am getting an error. I am using the following commands:
filing = Company("AMD").get_filings(form="10-K").latest(1)
income_statement = filing.obj().income_statement
Traceback (most recent call last):
File "", line 1, in
File "/opt/homebrew/lib/python3.11/site-packages/edgar/company_reports.py", line 56, in financials
return Financials.from_xbrl(xbrl)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/edgar/financials.py", line 301, in from_xbrl
balance_sheet = BalanceSheet(xbrl.facts)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/edgar/financials.py", line 174, in init
super().init(facts, end_date)
File "/opt/homebrew/lib/python3.11/site-packages/edgar/financials.py", line 66, in init
self.end_date: str = end_date or self.facts.period_end_date
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/edgar/_xbrl.py", line 126, in period_end_date
return self.get_dei('DocumentPeriodEndDate')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/edgar/_xbrl.py", line 122, in get_dei
return res.value.item()
^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/pandas/core/base.py", line 418, in item
raise ValueError("can only convert an array of size 1 to a Python scalar")
ValueError: can only convert an array of size 1 to a Python scalar
edgartools runs into this XML parsing exception when name is missing in the XML
Repro simple
import edgar
edgar.set_identity("id")
company = edgar.Company("HON")
filings = company.get_filings(form=["10-K"])
filings[0].text() <-- throws here
Python: 3.11.6
Edgartools version: 2.7.2
Hello!
I really appreciate this Python package and thank you heaps to the authors and contributors of this package, you have made my life so much easier!
I hope it is okay for me to ask a question about the usage of the package. I am now in a situation that I would like to download only the attachments (i.e., not the primary document) of a filing. Would that be possible? Any suggestions would be appreciated.
Best regards,
G
Hello,
is there a way to retrieve the information on interest expenses (needed to calculate EBITDA for a company)?
Or is a synonym used?
Regards,
Peter
When you use the .download() method, the attachments such as images aren't downloaded (and therefore, not rendered) in the downloaded HTML
Example:
cik = 'TSLA'
after = '2013-01-01'
filings = Company(cik).get_filings(form="8-K").filter(date=f"{after}:")
filings[0].attachments[1].url
This leads to https://www.sec.gov/Archives/edgar/data/1318605/000095017024046895/tsla-ex99_1.htm which is a slide-deck.
The .download() method returns the HTML but the images are hosted relative to the file URL (e.g. https://www.sec.gov/Archives/edgar/data/1318605/000095017024046895/tsla-ex99_1s2.jpg).
We can also find these images in the filings[0].attachments field, under GRAPHIC tag.
What do you think would be the best pattern for the .download() method to include this? Or are there other alternatives you propose?
This one is quite strange and may be related to #20
It looks like only NEW filings are throwing this error, though I have not yet confirmed. When taking one of the most recent filing accession numbers from https://www.sec.gov/cgi-bin/browse-edgar?company=&CIK=&type=8-K&owner=include&count=100&action=getcurrent&start=2 and passing it to find the following error is thrown:
accession_no = '0001493152-24-001619'
f = find(accession_no)
WARNING _filings.py:626
Provide a year between 1994 and 2024 and optionally a quarter (1-4) for which the SEC has
filings.
e.g. filings = get_filings(2023) OR
filings = get_filings(2023, 1)
(You specified the year 2024 and quarter 2)
WARNING _filings.py:626
Provide a year between 1994 and 2024 and optionally a quarter (1-4) for which the SEC has
filings.
e.g. filings = get_filings(2023) OR
filings = get_filings(2023, 1)
(You specified the year 2024 and quarter 3)
WARNING _filings.py:626
Provide a year between 1994 and 2024 and optionally a quarter (1-4) for which the SEC has
filings.
e.g. filings = get_filings(2023) OR
filings = get_filings(2023, 1)
(You specified the year 2024 and quarter 4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/blahblah/python3.11/site-packages/edgar/__init__.py", line 77, in find
return get_by_accession_number(search_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/blahblah/python3.11/site-packages/edgar/_filings.py", line 2050, in get_by_accession_number
return filings.get(accession_number)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/blahblah/python3.11/site-packages/edgar/_filings.py", line 812, in get
with Status(f"[bold deep_sky_blue1]Searching through the most recent filings for {accession_number}...",
File "/blahblah/python3.11/site-packages/rich/status.py", line 97, in __enter__
self.start()
File "/blahblah/python3.11/site-packages/rich/status.py", line 87, in start
self._live.start()
File "/blahblah/python3.11/site-packages/rich/live.py", line 113, in start
self.console.set_live(self)
File "/blahblah/python3.11/site-packages/rich/console.py", line 836, in set_live
raise errors.LiveError("Only one live display may be active at once")
rich.errors.LiveError: Only one live display may be active at once
>>> and if attempted again...
>>> f = find('0001493152-24-001619')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/blahblah/python3.11/site-packages/edgar/__init__.py", line 77, in find
return get_by_accession_number(search_id)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/blahblah/python3.11/site-packages/edgar/_filings.py", line 2050, in get_by_accession_number
return filings.get(accession_number)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/blahblah/python3.11/site-packages/edgar/_filings.py", line 812, in get
with Status(f"[bold deep_sky_blue1]Searching through the most recent filings for {accession_number}...",
File "/blahblah/python3.11/site-packages/rich/status.py", line 97, in __enter__
self.start()
File "/blahblah/python3.11/site-packages/rich/status.py", line 87, in start
self._live.start()
File "/blahblah/python3.11/site-packages/rich/live.py", line 113, in start
self.console.set_live(self)
File "/blahblah/python3.11/site-packages/rich/console.py", line 836, in set_live
raise errors.LiveError("Only one live display may be active at once")
rich.errors.LiveError: Only one live display may be active at onceAfter running the following code
from edgar import get_filings, Filings, Filing, get_company, set_identity, Company
set_identity('Isla Sthiss [email protected]')
forms = ['10-Q',"10-K","10-K/A","10-Q/A"]
filings = get_filings(range(2013, 2024), form=forms)
file_10k = filings.filter(form="10-K", amendments=True)
filing = filings[3]
filing.homepage
filing_xbrl = filing.xbrl()
TypeError: unhashable type: 'Series'
TypeError: unhashable type: 'Series'
Here you have an interactive notebook to reproduce the error: https://colab.research.google.com/drive/12m_ohER2LGLe0khppGeKyqoqUkMLh_Ks?usp=sharing
NVDA 10-K "Item 1" does not seem to provide the full item text.
February 24, 2023 - 10-K: Annual report for year ending January 29, 2023
NVDA 10-K filing "Item 1" text spans from page 4 to page 15. However below code provides only part of "Item 1" text page 4 through middle of page 11.
tenk = Company("NVDA").get_filings(form="10-K").latest(1).obj()
print(f"NVDA item 1 text:\n{tenk['Item 1']}")
Is there another way to get the full "Item 1" text correctly?
First, excellent library. Well done! 👏 Thanks for creating this....
I'm curious to know if you plan to add text parsing support for 10-Ks and 10-Qs and other filings, much like you provided for 8-Ks?
Like the following:
On a side note, is there an easy way to convert the results above to JSON?
Cheers!
Python version: 3.12.2 (main, Mar 12 2024, 08:01:18) [GCC 12.2.0]
edgartools version: 2.21.0
Minimum Reproducible Example
from edgar import Company, set_identity
if __name__ == "__main__":
set_identity("Ilias Antonopoulos ([email protected])")
filings = Company("AAPL").get_filings(form="10-K", filing_date="2023-01-01:2023-12-31")
filing = filings.get(0).obj()
print(filing.financials.income_statement)
print(filing.financials.income_statement.to_dataframe())I have noticed that, although the facts present in filing.financials.income_statement are correct, when compared with the corresponding income statement table from SEC, the facts:
are missing from the .to_dataframe() version of it (see screenshot below).
Implement parsing of Ownership XML documents for Form 3,4,5
How much do you know about filings 144, for some reason AAPL only has two but maybe I am missing something. In anycase, here is some code that can help with parsing if you are looking into adding it: https://colab.research.google.com/drive/1VyH7hQ__W1Ab-HWRmPS2gywDHHZgsbrl?usp=sharing
Hi,
does an internal mechanism exist to track the number of requests to match the rate limit imposed by data.sec.gov?
The limit is currently 10 requests/second.
https://www.sec.gov/oit/announcement/new-rate-control-limits
Thanks!
Hi!
More of a feature request, although this may already be supported; I may just be missing it.
If I have a Filing, it would be great to be able to get the URL links of the filing (screenshot) programmatically, as well:
How can I programmatically get the URL https://sec.gov/Archives/edgar/data/789019/0000950170-23-035122-index.html from the filing?
Here is the code I used to get the filing:
company = Company("MSFT")
company.get_filings(accession_number="0000950170-23-035122")[0]
NFLX 10-K filing income statement appears to be using different gaap tags and missing multiple facts from income statement after parsing.
https://www.sec.gov/ix?doc=/Archives/edgar/data/1065280/000106528023000035/nflx-20221231.htm#id2a4f0e6b6dd43c49a69044f9260d065_115
"Total Revenue" tag used in 10-K filing: us-gaap:Revenues
incdf = Company("NFLX").get_filings(form="10-K").latest(1).obj().income_statement.income_statement_dataframe
print(f"{incdf.to_markdown()}")
above code produces only the below three facts from NFLX 10-K filing.
| Fact | Value | |
|---|---|---|
| 0 | Operating Income or Loss | $5,632,831,000 |
| 1 | Net Income | $4,491,924,000 |
| 2 | Interest Expense | $706,212,000 |
Python version: 3.12.2 (main, Mar 12 2024, 08:01:18) [GCC 12.2.0]
edgartools version: 2.21.1
Minimum Reproducible Example
from edgar import Company, set_identity
if __name__ == "__main__":
set_identity("Ilias Antonopoulos ([email protected])")
filings = Company("AAPL").get_filings(form="10-K", filing_date="2016-01-01:2016-12-31")
filing = filings.get(0).obj()
print(filing.financials.income_statement)
print(filing.financials.income_statement.to_dataframe())yields the following:
i.e. apart from the missing Net Sales, one can also identify some incorrect numerical values for:
Gross ProfitNet IncomeEarnings Per ShareUpon closer examination, it seems that the parser may have "picked up" the values not from the CONSOLIDATED STATEMENTS OF OPERATIONS but from Note 12 – Selected Quarterly Financial Information (Unaudited) (see screenshot below).
p.s. May also happen to 10Ks from other fiscal years prior to 2021 (based on my findings, 2021 and beyond is correct).
p.s.#2 Keep up the great work, this library is a true gift to the community <3
Really appreciate you have created such an awesome library.
Here is one issue we came up with:
After I fetched attachment
I need to download last text file, here is my code:
with open("my_file.txt", "wb") as binary_file:
submission_txt = filing.attachments[14].download()
# Write bytes to file
binary_file.write(submission_txt)
It showed error, can I resolve this issue?
I recently updated my Conda environment to version 24.1.2 and then updated Python to 3.11.8. Using pip, I installed EDGAR to get version 5.4.3.
However, upon trying to import EDGAR and run setup, I got the following error:
What could be the cause of this? Thanks.
I have a particular filing and was looking at the items and noticed it cuts off after the dollar sign. This particular example is an 8-K.
# get the company
c = Company('0001588272')
# get the filing
f = c.get_filings().filter(form='8-K')[0]
# display the item text
print(f.text())Notice the cut-off at:
which the Company issues shares under its distribution reinvestment plan (the “DRP”) at $
I get the expected result when I run
filings = get_filings(2023, 1)
It finishes in about 7~8 secs on my laptop. However, when I run
filings = get_filings(2023)
the code runs indefinitely. I have to write a for loop to get the filings for each quarter
filings_list = []
for i in range(1,5):
filings_list.append(get_filings(2023, i))
it takes 24~25 secs to finish. My Python version is 3.10.14 and I am using edgartools==2.19.2. I am not sure if this is a bug or if I am doing something wrong. I would appreciate any help. Thank you.
I am trying to print financial statements of quarterly or annual reports. However, the API allows ti print only one at a time. Is there a way to dump the information of last say 3 statements in single table?
For example
Company("AAPL").get_filings(form="10-K", date="2021-01-01:").latest(1).obj()
The above one prints only one statement. Can the API consolidate all the requested filings and dump in a table format?
Hi,
I‘m recently trying to see if it is possible to fetch latest data via edgartools.
eg. I was looking at https://www.sec.gov/cgi-bin/browse-edgar?company=&CIK=&type=4&owner=only&count=40&action=getcurrent
and try to get the latest records with exact accepted date time. However, it seems edgartools' most recent records only went up to "yesterday" not "today". I also tried edgar.get_by_accession_number but cannot find a specific "today's" record.
Would it be possible to achieve "live" records fetching using edgartools?
Thank you for the great tool.
import pandas as pd
from edgar import *
set_identity("Sample [email protected]")
to_drop = ['ticker', 'maturity_date', 'annualized_rate','desc_other_units',
'is_default', 'cash_collateral','non_cash_collateral', 'restricted']
fund = Company("MDY")
latest_holdings = Company("MDY").get_filings(form="NPORT-P")[0]
fund_report = latest_holdings.obj().investment_data()
invest_table = fund_report.drop(to_drop, axis=1)
print(invest_table)
#all the ISINs are the same:
name title
0 Hubbell Inc Hubbell Inc
1 Builders FirstSource Inc Builders FirstSource Inc
2 Reliance Steel & Aluminum Co Reliance Steel & Aluminum Co
3 Graco Inc Graco Inc
4 Jabil Inc Jabil Inc
lei cusip isin balance units \
0 54930088VDQ6840Y6597 443510607 US3719011096 453329.00000000 NS
1 549300W0SKP6L3H7DP63 12008R107 US3719011096 1084228.00000000 NS
2 549300E287ZOFT3C5Z56 759509102 US3719011096 497354.00000000 NS
3 4T5VJ4S81BRT6Q7GGT78 384109104 US3719011096 1424046.00000000 NS
4 5493000CGCQY2OQU7669 466313103 US3719011096 1122419.00000000 NS
Can we easily aggregate the 13F-HR holdings per ticker to know which companies are associated with a ticker ? For example, similar to how SEC-API does here
https://sec-api.io/sandbox/13f-filings-holding-tesla
I got to the point where I have all the issuers, but need to map this to a ticker
In the spirit of ETL, it would be beneficial for downstream applications of edgartools to be able to discretize company filings persistence (e.g. in a storage backend like S3) and company filings parsing (e.g. extracting the balance sheet data) in order to engineer more flexible data pipelines.
The library right now provides the ability to persist a company filing (e.g. through .full_text_submission() on filing or through .download() on attachments) but - based on my experience as an application user - there doesn't seem to be a straightforward way to load a saved company filing and continue the parsing (e.g. filing.obj()) from there.
I suspect that this might not be the vision and philosophy of edgartools (and i totally respect it), just pitching the angle in order to discuss whether this resonates with you and the community.
p.s. I really, really like how the library is designed! Kudos for the effort, this is monumental work! <3
p.s.2 happy to help with the design / implementation of said feature (if it first passes feedback ofc)
2024-04-14 15:25:01,042 - root - INFO - Attachment for 0001047469-02-007674.txt -> EX-99.1.txt downloaded.
Traceback (most recent call last):
File "/Users/test.py", line 94, in
download_filings_and_attachments(fillings10K, dir_path_10K)
File "/Users/test.py", line 57, in download_filings_and_attachments
f.write(filing.text())
File "/Users/sumithkumars/Library/Python/3.9/lib/python/site-packages/edgar/_filings.py", line 1671, in text
return HtmlDocument.from_html(html_content).text
File "/Users/sumithkumars/Library/Python/3.9/lib/python/site-packages/edgar/documents.py", line 422, in from_html
root: Tag = cls.get_root(html)
File "/Users/sumithkumars/Library/Python/3.9/lib/python/site-packages/edgar/documents.py", line 412, in get_root
if "" in html[:500]:
TypeError: a bytes-like object is required, not 'str'
The current implementation of the edgar.core.get_bool function returns False, if the string field value is String 'true', therefore returning wrong reporting_relationship for some filings, e.g.
this filing will be parsed to is_officer = False, but should be is_officer = True
Per README documentation:
company = Company("SNOW")
company_facts = company.get_facts_for_namespace()is expected to work - however get_facts_for_namespace returns an AttributeError.
# e.g. tested it in Python 3.12.2
pip install edgartools==2.9.0and then:
from edgar import *
set_identity("Ilias Antonopoulos [email protected]") # enter valid email here
company = Company("SNOW")
company_facts = company.get_facts_for_namespace()
p.s. happy to assist with its resolution if it's source code issue and not simply a documentation one :)
I've been running some tests on "Item 1" extraction across multiple symbols and have found a few that don't get picked up in the code blow.
Code:
symbol = "PR" #EE also has this behavior if you need another test
filing = Company(symbol).get_filings(form="10-K").latest(1)
tenk = filing.obj()
output = tenk['Item 1']
Hi I'm getting an error at the initial import statement
from edgar import *
error is:
ImportError Traceback (most recent call last)
Cell In[5], line 1
----> 1 from edgar import *
File ~\AppData\Roaming\Python\Python311\site-packages\edgar\muniadvisors.py:14
11 from rich.table import Table
12 from rich.text import Text
---> 14 from edgar import Filing
15 from edgar._party import Name, Address
16 from edgar._rich import repr_rich
ImportError: cannot import name 'Filing' from 'edgar' (C:\Users\Geoffrey\AppData\Roaming\Python\Python311\site-packages\edgar_init_.py)
I came across a curious situation where the find method does not appear to work when passing a fairly new filing accession number.
However, finding the filing via Company method seems to fetch the filing just fine.
Example:
accession_no = '0001193125-23-300021'
# not working as of 12/20/21 @ 20:43 ET
find(accession_no')
cik = '0001588272'
# vs this which works fine
Company(cik).get_filings().filter(form='8-K')[0]Nothing critical just thought I'd log it! Thanks :)
Primary document and full submission text not working if you click the links. There is info on homepage. Cik 1145255 for example
Think this is a fantastic library! Is it possible to pass in a http proxy in the request?
First of all, This is really amazing! I am enjoying using it and makes it much more easier to find filings than the edgar sec website.
Is there a way to convert the latest 10-Q filing data to a pandas dataframe?
from edgar import *
set_identity("first last [email protected]")
company = Company(909832)
last10q = company.get_filings(form="10-Q").latest()
last10q.xbrl()One of the nicest looking EDGAR libraries out there, thanks for your open source contribution.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.