zhaolewen / py-sec-xbrl

SEC EDGAR Parser based on Python 3

py-sec-xbrl

SEC EDGAR Parser based on Python 3

Intruduction

This is a tool intended to parse XBRL files from SEC. Thus, the focus is to parse XBRL XML files so that data is more easily accessible. The idea is to provide a tool for you to code you want instead of a tool that implements a workflow but is rigid.

In addition, it's not intended to be a tool to scrap SEC EDGAR as it varies a lot as to how you want to do the scrapping and it's relatively easier. (though it can be added later if you want)

The repository is originally forked from https://github.com/tooksoi/ScraXBRL, but I soon find out that we have very different approaches and objectives, so soon afterwards the code in the 2 repositories are completely different and nothing is taken from ScraXBRL.

features

• Parsing of the main XBRL XML file to extract data
• Identify the main XBRL file within its XBRL package

How to install

Current verion: v0.2

Dependencies: in the requirements.txt file, currently only the lxml library

Installation: pip install py-sec-xbrl

How to start

1. get some XBRL XML files (see documentation if you don't have one yet)
2. see test-parse.py, modify the path to the XML file and it's really easy

Documentation

More detailed documentation can be found here: doc

Development roadmap

2 priorities for the moment:

1. Better formatting of extracted XBRL data
2. More advanced extracting functionalities (notably on the segments & calculations)

py-sec-xbrl's Issues

KeyError: 'id' for older filings

Hello,

I was using the package with the xbrl.zip file from:

https://www.sec.gov/Archives/edgar/data/320193/000119312510238044

Afterwards I tested the code from the "test-parse.py" file.

However, the last parser.parseMainXBRL(txt) throws the following error:

---

KeyError Traceback (most recent call last)
in
1 with open(os.path.join(folder, main_data_f)) as f:
2 txt = f.read()
--> 3 ctx_elems, data_elems = parser.parseMainXBRL(txt)
4
5 print("parsing result: ")

c:\users\edgar api\src\py-sec-xbrl\pysecxbrl\parsing.py in parseMainXBRL(self, text)
14 #main_elems = root.findall(".//dei:DocumentType",namespaces=root.nsmap)
15 # print(len(main_elems))
--> 16 data_elems = self.getDataElementsAsDict(root)
17 ctx_elems = self.getContextElementsAsDict(root)
18

c:\users\edgar api\src\py-sec-xbrl\pysecxbrl\parsing.py in getDataElementsAsDict(self, tree)
43 item[attr] = elem.attrib[attr]
44
--> 45 res[elem.attrib["id"]] = item
46
47 return res

src\lxml\etree.pyx in lxml.etree._Attrib.getitem()

KeyError: 'id'