csingley / ibflex Goto Github PK
View Code? Open in Web Editor NEWPython parser for Interactive Brokers Flex XML statements
License: MIT License
Python parser for Interactive Brokers Flex XML statements
License: MIT License
Hello,
I have a simple code retrieving my IB Flex trading activity
token = 'myvalidtoken'
query_id = '583460'
response = client.download(token, query_id)
response = parser.parse(response)
stmt = response.FlexStatements[0]
The reponse looks good I receive a correct XML structure but the parser is not happy with it apparently and going through this problem below (hiding my accountid for privacy consideration)
ParserError: Unknown string format: b'<FlexQueryResponse queryName="Trades YTD" type="AF">\n<FlexStatements count="1">\n<FlexStatement accountId="UXXXXXXXX" fromDate="20220103" toDate="20220520" period="YearToDate" whenGenerated="20220523;103059">\n<Trades>\n<Trade symbol="AAOI" tradeDate="20220107" fifoPnlRealized="0" buySell="SELL" quantity="-100" tradePrice="4.8" currency="USD" />\n<Trade symbol="AAOI" tradeDate="20220119" fifoPnlRealized="233.97086" buySell="BUY" quantity="200" tradePrice="4.05" currency="USD" />\n<Trade symbol="AAOI" tradeDate="20220202" fifoPnlRealized="0" buySell="SELL" quantity="-125" tradePrice="4.15" currency="USD" />\n<Trade symbol="AAOI" tradeDate="20220222" fifoPnlRealized="0" buySell="SELL" quantity="-125" tradePrice="3.99" currency="USD" />\n<Trade symbol="AAOI" tradeDate="20220307" fifoPnlRealized="27.592442" buySell="BUY" quantity="50" tradePrice="3.57" currency="USD" />\n<Trade symbol="AAOI" tradeDate="20220307" fifoPnlRealized="94.519868" buySell="BUY" quantity="200" tradePrice="3.568" currency="USD" />\n<Trade symbol="AAON" tradeDate="20220202" fifoPnlRealized="0" buySell="SELL" quantity="-4" tradePrice="63.95" currency="USD" />\n<Trade symbol="AAON" tradeDate="20220209" fifoPnlRealized="0" buySell="SELL" quantity="-4" tradePrice="59.93" currency="USD" />\n<Trade symbol="AAON" tradeDate="20220215" fifoPnlRealized="11.252432" buySell="BUY" quantity="8" tradePrice="60.158" currency="USD" />\n<Trade symbol="AAPL" tradeDate="20220124" fifoPnlRealized="0" buySell="BUY" quantity="2" tradePrice="159.88" currency="USD" />\n<Trade symbol="AAPL" tradeDate="20220211" fifoPnlRealized="22.91798"
...
Any idea what can be wrong? I am attaching a XML sample of my response in a .txt format if it can help in the debugging,
Thanks in advance for your help!
I got this error, changed the code and now seems to be working
ValueError: 'RI' is not a valid Code
Pr for fix:
#66
Hi,
I see a lot of changes have been made since the last release in 2021.
My app depends on the old version in pypi (A pypi package must have dependencies in pypi).
So I suggest to update version and upload it to pypi.
Thanks!
Hi,
Since yesterday, I've been having an issue with getting flex queries via the client.download(...) method.
I just wanted to confirm if this is the case for everyone or if I'm the only one getting this issue.
I also tried entering the URL manually via browser and got a "This site can’t be reached" message.
Here's the link template:
https://gdcdyn.interactivebrokers.com/Universal/servlet/FlexStatementService.GetStatement?q=REFERENCE_CODE&t=TOKEN&v=VERSION
(Source: https://www.interactivebrokers.com/en/software/am/am/reports/flex_web_service_version_3.htm)
Can anyone please confirm if this is an issue on IB's side or something to do with my proxy/network settings?
Thank you!
hey. (I edited the questions that why the topic isn't right)
I was wondering if there is a way to arrange trades by the open and close.
Sometimes a trade is open few days ago meanwhile other trade activity happens, only then a the trade was closed. so I can not find a way to attach the open of a trade with there close.
====================================
O 0.0 BUY MBTV2 2022-10-03 18:05:41
O 0.0 BUY BITO 2022-10-03 18:12:20
O 0.0 BUY BITO 2022-10-03 18:12:20
O 0.0 BUY BITO 2022-10-03 18:12:20
O 0.0 BUY BITO 2022-10-03 18:12:20
O 0.0 BUY BITO 2022-10-03 18:12:40
O 0.0 SELL NQZ2 2022-10-04 06:52:08
C 145.7 BUY NQZ2 2022-10-04 07:10:05
====================================
O 0.0 SELL NQZ2 2022-10-04 07:25:09
C -99.3 BUY NQZ2 2022-10-04 07:26:44
C 1328.9 SELL BITO 2022-10-04 11:05:10
====================================
here I tried to sort the trades by dates the were executed but it seems the order isn't right (open BITO at 10-3-22 closed at 10-4-22 but the close of NQZ2 was before.)
hope that make sense.
thanks!
Wow. It's great to find this repo. Exactly what I need. Thank you so much.
Sorry if this issue doesn't belong here. Maybe it should be a question for the IBKR platform. As indicated in the title, is it supported to specify custom start and end date to download flex query reports? I want to download three years of trading confirmation and I am not sure if it's possible with the flex query api. If I can specify the date, then I can first download the report for year 2019, then for year 2020 and then year to date in 2021.
Thanks!
I was able to download/parse the Activity Flex Query outputs, as XML.
When I set up my Activity Flex Query output format to CSV, the code throws a BadResponseError.
Can you support CSV output format?
Thanks for working on this!
Hey @csingley.
Found this issue recently while testing the package from master branch.
>>> res = parser.parse(response)
Traceback (most recent call last):
File "/root_folder/ibflex/ibflex/parser.py", line 303, in convert
prepped_value = prep(value)
^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 176, in prep_date
date_format = DATE_FORMATS[len(value)][value.count('/')]
~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: 5
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root_folder/ibflex/ibflex/parser.py", line 159, in parse_element_attr
converted = ATTRIB_CONVERTERS[Type](value=value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 324, in optional_convert
return None if value in ("", "-", "--", "N/A") else func(value)
^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 314, in convert
raise FlexParserError(
ibflex.parser.FlexParserError: Can't convert 'MULTI' to <class 'datetime.date'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/root_folder/ibflex/ibflex/parser.py", line 53, in parse
parsed = parse_element(root)
^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 85, in parse_element
return parse_data_element(elem)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 80, in parse_element
return parse_element_container(elem)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 83, in parse_element
return parse_element_container(elem)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 115, in parse_data_element
attrs = dict(
^^^^^
File "/root_folder/ibflex/ibflex/parser.py", line 116, in <genexpr>
parse_element_attr(Class, k, v)
File "/root_folder/ibflex/ibflex/parser.py", line 166, in parse_element_attr
raise FlexParserError(msg)
ibflex.parser.FlexParserError: SymbolSummary.tradeDate - Can't convert 'MULTI' to <class 'datetime.date'>
For now, I'll just remove this attribute from the query configuration.
Best regards.
Hi, Could you please add CFD to the AssetClass Enum? Thanks.
Hi,
I've been experimenting with ibflex and found 2 issues:
Here's the XML file:
sample.zip
First, this is how I generated the XML file:
token = "..."
query_id = "..."
response = client.download(token, query_id)
with open('sample.xml', 'w') as outfile:
outfile.write(response.decode('utf-8'))
response_p = parser.parse("sample.xml") #errors here
Second, here's the flex query configuration:
Note that under trades, all boxes are checked.
Issues:
FlexParserError: Trade.notes - Can't convert 'FP;P' to <class 'tuple'>
response1 = parser.parse("sample.xml")
Traceback (most recent call last):
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 312, in convert
return Type(prepped_value)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 280, in <genexpr>
enums.Code(v)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\enum.py", line 339, in __call__
return cls.__new__(cls, value)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\enum.py", line 662, in __new__
raise ve_exc
ValueError: 'FP' is not a valid Code
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 159, in parse_element_attr
converted = ATTRIB_CONVERTERS[Type](value=value)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 314, in convert
raise FlexParserError(
FlexParserError: Can't convert 'FP;P' to <class 'tuple'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<ipython-input-29-c6dab1bf5206>", line 1, in <module>
response1 = parser.parse("sample.xml")
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 53, in parse
parsed = parse_element(root)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 85, in parse_element
return parse_data_element(elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in <dictcomp>
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 80, in parse_element
return parse_element_container(elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in <dictcomp>
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 83, in parse_element
return parse_element_container(elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 115, in parse_data_element
attrs = dict(
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 116, in <genexpr>
parse_element_attr(Class, k, v)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 166, in parse_element_attr
raise FlexParserError(msg)
FlexParserError: Trade.notes - Can't convert 'FP;P' to <class 'tuple'>
I tried fixing it by doing:
response = response.decode('utf-8').replace('FP;P','').replace('P;RP','')
and it works. But I'm not sure why the error was caused in the first place.
FlexParserError: Trade has no attribute 'serialNumber'
response = client.download(token, query_id)
response = response.decode('utf-8').replace('FP;P','').replace('P;RP','')
#with open('sample.xml', 'w') as outfile:
# outfile.write(response.decode('utf-8'))
parsed_response = parser.parse(bytes(response, 'utf-8'))
Traceback (most recent call last):
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 115, in parse_data_element
attrs = dict(
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 116, in <genexpr>
parse_element_attr(Class, k, v)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 156, in parse_element_attr
Type = Class.__annotations__[name]
KeyError: 'serialNumber'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<ipython-input-30-4183a17da92d>", line 5, in <module>
parsed_response = parser.parse(bytes(response, 'utf-8'))
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 53, in parse
parsed = parse_element(root)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 85, in parse_element
return parse_data_element(elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in <dictcomp>
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 80, in parse_element
return parse_element_container(elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 125, in <dictcomp>
contained_elements = {child.tag: parse_element(child) for child in elem}
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 83, in parse_element
return parse_element_container(elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
File "F:\Dev\WPy64-3870\python-3.8.7.amd64\lib\site-packages\ibflex\parser.py", line 121, in parse_data_element
raise FlexParserError(msg)
FlexParserError: Trade has no attribute 'serialNumber'
I'm not able to solve this error. I manually check the XML file and did find 'serialNumber` field for all entries, but still, the error occurs.
Edit 1:
If I uncheck the following fields under "Trade" section, then I'm able to solve the 2nd issue - but why does this happen?
Firstly, thanks for making this! I found this through the ghostfolio-sync which uses this package to import into ghostfolio.
I followed the steps to create the flex report but it seems to not handle open positions with empty figi value
I think most Canadian stocks have no figi values so should we maybe not enforce this?
Thanks!
I don't know if it is a feature request, but I receive results for last business day only (client.dowload).
Can it be changed? Is it determined by IB ?
Hi,
I've just started to play around with this package. One thing I encountered (as many others it seems) is that the parser fails violently if the flex query results contains unknown fields. I was wondering if there would be a way to change this behavior, maybe optionally, to simply ignore unknown fields? The IBKR web interface allows to very easily add all available fields to a query, and manually figuring out which to enable for the parser and which to disable appears a bit tricky. Many of the unsupported fields seem to be details that are maybe not interesting in most use cases, so it probably would make sense for the code to fail only once the user tries to access such an attribute.
Any thoughts on this? I guess this is more of a general feature request than a specific issue.
AttributeError Traceback (most recent call last)
Cell In[57], line 1
----> 1 stmt = response.FlexStatements[0]
AttributeError: 'bytes' object has no attribute 'FlexStatements'
Hello,
I have jsut started using Python and am new to programming. I am struggling to make use of this project. Could anyone explain how to get started/set up this thing? I have the. token + an xml, however I have no idea what to do with makefile, setup, etc.
I have tried: pythong ibflex build and install but the sample code does not work for me (its in Jypiter format and I am using pycharm.
Sorry for the beginer questions!
Thanks a lot in advance.
Trying to parse my report, and always get this error.
Since very recently, my flexqueries do not parse anymore. one month ago, it still worked fine.
Python 3.7.10
ibflex 0.15 (same with 0.14)
WSL Ubuntu 18.04.4 LTS
sample code:
import yaml
credsfile="../ibkr.yaml"
with open(credsfile, 'r') as f:
config = yaml.safe_load(f)
token = config['token']
queryId = config['queryId']
from ibflex import client, parser, Types
response = client.download(token, queryId)
print('received valid FlexQuery information:\n' + str(response[:270]))
print('failing parsing of the response:')
parser.parse(response)
received valid FlexQuery information:
b'<FlexQueryResponse queryName="my monthly flexquery" type="AF">\n<FlexStatements count="1">\n<FlexStatement accountId="U*******" fromDate="2022-06-01" toDate="2022-06-30" period="LastMonth" whenGenerated="2022-07-02;10:12:05">\n<CashReport>\n<CashReportCurrency currency="BASE_S'...
failing parsing of the response:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_data_element(elem)
116 parse_element_attr(Class, k, v)
--> 117 for k, v in elem.attrib.items()
118 )
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in <genexpr>(.0)
116 parse_element_attr(Class, k, v)
--> 117 for k, v in elem.attrib.items()
118 )
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_element_attr(Class, name, value)
155 # in parse_data_element(), instead accepting it as a function arg here.
--> 156 Type = Class.__annotations__[name]
157
KeyError: 'endingCashIBUKL'
During handling of the above exception, another exception occurred:
FlexParserError Traceback (most recent call last)
<ipython-input-4-4e26de39b5b6> in <module>
11 print('received valid FlexQuery information:\n' + str(response[:270]))
12 print('failing parsing of the response:')
---> 13 parser.parse(response)
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse(source)
51 if root.tag != "FlexQueryResponse":
52 raise FlexParserError("Not a FlexQueryResponse")
---> 53 parsed = parse_element(root)
54 assert isinstance(parsed, Types.FlexQueryResponse)
55 return parsed
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_element(elem)
83 return parse_element_container(elem)
84
---> 85 return parse_data_element(elem)
86
87
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_data_element(elem)
123 # FlexQueryResponse & FlexStatement are the only data elements
124 # that contain other data elements.
--> 125 contained_elements = {child.tag: parse_element(child) for child in elem}
126 if contained_elements:
127 assert elem.tag in ("FlexQueryResponse", "FlexStatement")
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in <dictcomp>(.0)
123 # FlexQueryResponse & FlexStatement are the only data elements
124 # that contain other data elements.
--> 125 contained_elements = {child.tag: parse_element(child) for child in elem}
126 if contained_elements:
127 assert elem.tag in ("FlexQueryResponse", "FlexStatement")
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_element(elem)
78 raise FlexParserError(msg)
79
---> 80 return parse_element_container(elem)
81
82 if not elem.attrib:
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_element_container(elem)
99 return tuple(itertools.chain.from_iterable(fxlots))
100
--> 101 instances = tuple(parse_data_element(child) for child in elem)
102 return instances
103
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in <genexpr>(.0)
99 return tuple(itertools.chain.from_iterable(fxlots))
100
--> 101 instances = tuple(parse_data_element(child) for child in elem)
102 return instances
103
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_data_element(elem)
123 # FlexQueryResponse & FlexStatement are the only data elements
124 # that contain other data elements.
--> 125 contained_elements = {child.tag: parse_element(child) for child in elem}
126 if contained_elements:
127 assert elem.tag in ("FlexQueryResponse", "FlexStatement")
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in <dictcomp>(.0)
123 # FlexQueryResponse & FlexStatement are the only data elements
124 # that contain other data elements.
--> 125 contained_elements = {child.tag: parse_element(child) for child in elem}
126 if contained_elements:
127 assert elem.tag in ("FlexQueryResponse", "FlexStatement")
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_element(elem)
81
82 if not elem.attrib:
---> 83 return parse_element_container(elem)
84
85 return parse_data_element(elem)
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_element_container(elem)
99 return tuple(itertools.chain.from_iterable(fxlots))
100
--> 101 instances = tuple(parse_data_element(child) for child in elem)
102 return instances
103
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in <genexpr>(.0)
99 return tuple(itertools.chain.from_iterable(fxlots))
100
--> 101 instances = tuple(parse_data_element(child) for child in elem)
102 return instances
103
~/miniconda3/envs/IBenv/lib/python3.7/site-packages/ibflex/parser.py in parse_data_element(elem)
119 except KeyError as exc:
120 msg = f"{Class.__name__} has no attribute " + str(exc)
--> 121 raise FlexParserError(msg)
122
123 # FlexQueryResponse & FlexStatement are the only data elements
FlexParserError: CashReportCurrency has no attribute 'endingCashIBUKL'
I receive this error ibflex.parser.FlexParserError: Can't convert '20200406;214347' to <class 'datetime.date'>
today.
The same code works before.
Just got this when parsing the downloaded XML report. Will look into it.
Traceback (most recent call last):
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 159, in parse_element_attr
converted = ATTRIB_CONVERTERS[Type](value=value)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 355, in convert_enum
return Type(value) if value != "" else None
File "/usr/lib/python3.8/enum.py", line 315, in __call__
return cls.__new__(cls, value)
File "/usr/lib/python3.8/enum.py", line 617, in __new__
raise ve_exc
ValueError: 'Commission Adjustments' is not a valid CashAction
and
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "parse-tx.py", line 38, in <module>
parse()
File "parse-tx.py", line 20, in parse
response = parser.parse(filename)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 53, in parse
parsed = parse_element(root)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 85, in parse_element
return parse_data_element(elem)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 125, in <dictcomp>
contained_elements = {child.tag: parse_element(child) for child in elem}
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 80, in parse_element
return parse_element_container(elem)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 125, in parse_data_element
contained_elements = {child.tag: parse_element(child) for child in elem}
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 125, in <dictcomp>
contained_elements = {child.tag: parse_element(child) for child in elem}
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 83, in parse_element
return parse_element_container(elem)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 101, in parse_element_container
instances = tuple(parse_data_element(child) for child in elem)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 101, in <genexpr>
instances = tuple(parse_data_element(child) for child in elem)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 115, in parse_data_element
attrs = dict(
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 116, in <genexpr>
parse_element_attr(Class, k, v)
File "/home/alen/.local/lib/python3.8/site-packages/ibflex/parser.py", line 166, in parse_element_attr
raise FlexParserError(msg)
ibflex.parser.FlexParserError: CashTransaction.type - 'Commission Adjustments' is not a valid CashAction
This line
r1 = parser.parse(rdb)
throws this exception and OpenPosition has indeed 'serialNumber' attribute .
<OpenPosition accountId="xxxxxx"
acctAlias=""
model=""
currency="USD"
fxRateToBase="1"
assetCategory="STK"
symbol="MSFT"
description="MICROSOFT CORP"
conid="272093"
securityID="US5949181045"
securityIDType="ISIN"
cusip="594918104"
isin="US5949181045"
listingExchange="NASDAQ"
underlyingConid=""
underlyingSymbol=""
underlyingSecurityID=""
underlyingListingExchange=""
issuer=""
multiplier="1"
strike=""
expiry=""
putCall=""
principalAdjustFactor=""
serialNumber=""
deliveryType=""
commodityType=""
fineness="0.0"
weight="0.0 ()"
reportDate="20210514"
position="200"
markPrice="248.15"
positionValue="49630"
openPrice="219.73791395"
costBasisPrice="219.73791395"
costBasisMoney="43947.58279"
percentOfNAV="14.99"
fifoPnlUnrealized="5682.41721"
side="Long"
levelOfDetail="SUMMARY"
openDateTime=""
holdingPeriodDateTime=""
vestingDate=""
code=""
originatingOrderID=""
originatingTransactionID=""
accruedInt="" />
Is it somehow possible to directly parse the response from the api?
As far as I'm able to figure it out right now the client does only download and the parser requires a file.
I worked around it by manually doing the etree but it would be nice if that would somehow be possible. What I currently do is:
response = client.download(token, queryId)
root = ET.fromstring(response)
if root.tag != "FlexQueryResponse":
raise FlexParserError("Not a FlexQueryResponse")
statement = parser.parse_element(root)
assert isinstance(statement, Types.FlexQueryResponse)
Hello,
First of all, thank you for your package; it saves me a lot of time by not having to write my own parser. I'm trying to use it as instructed, have created a flex query, but when I try to read in the file, I get the following error:
ibflex.parser.FlexParserError: FIFOPerformanceSummaryUnderlying has no attribute 'subCategory'
Having added a subCategory optional attribute to the FIFOPerformanceSummaryUnderlying dataclass, I was then prompted with having no attribute 'figi'. All in all, I needed to add various attribute fields to a number of dataclasses in Types until it finally produced another error:
Python 3.11.1 (main, Feb 1 2023, 16:53:50) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
from ibflex import parser ; response = parser.parse("Summary.xml")
Traceback (most recent call last):
File "/Users/Jacob/.pyenv/versions/3.11.1/lib/python3.11/site-packages/ibflex/parser.py", line 131, in parse_data_element
return Class(**attrs)
^^^^^^^^^^^^^^
TypeError: FlexQueryResponse.init() got an unexpected keyword argument 'script'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/Users/Jacob/.pyenv/versions/3.11.1/lib/python3.11/site-packages/ibflex/parser.py", line 53, in parse
parsed = parse_element(root)
^^^^^^^^^^^^^^^^^^^
File "/Users/Jacob/.pyenv/versions/3.11.1/lib/python3.11/site-packages/ibflex/parser.py", line 85, in parse_element
return parse_data_element(elem)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/Jacob/.pyenv/versions/3.11.1/lib/python3.11/site-packages/ibflex/parser.py", line 133, in parse_data_element
raise FlexParserError(f"{Class.name} - " + str(exc))
ibflex.parser.FlexParserError: FlexQueryResponse - FlexQueryResponse.init() got an unexpected keyword argument 'script'
At this point, I'm not sure how to proceed/not sure if it's user error or a bug in the code because I think the 'script' keyword may have come from my flex file, in which case I'm not sure what settings I need to change in my flex file configuration to keep this from happening. Any help would be greatly appreciated.
Thank you in advance.
Hi,
I'm getting a bad response error if my flex query's output is a csv file.
example of stack trace:
>>> response = client.download(token, query_id)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/root/sample_env/env/lib/python3.8/site-packages/ibflex/client.py", line 117, in download
status = check_statement_response(response)
File "/root/sample_env/env/lib/python3.8/site-packages/ibflex/client.py", line 224, in check_statement_response
raise BadResponseError(response)
ibflex.client.BadResponseError: b'"ClientAccountID","AccountAlias","Model","CurrencyPrimary","FXRateToBase","AssetClass","Symbol","Description","Conid","SecurityID","SecurityIDType","CUSIP","ISIN","ListingExchange","UnderlyingConid","UnderlyingSymbol","UnderlyingSecurityID","UnderlyingListingExchange","Issuer","Multiplier","Strike","Expiry","TradeID","Put/Call","ReportDate","PrincipalAdjustFactor","DateTime","TradeDate","SettleDateTarget","TransactionType","Exchange","Quantity","TradePrice","TradeMoney","Proceeds","Taxes","IBCommission","IBCommissionCurrency","NetCash","ClosePrice","Open/CloseIndicator","Notes/Codes","CostBasis","FifoPnlRealized","FxPnl","MtmPnl","OrigTradePrice","OrigTradeDate","OrigTradeID","OrigOrderID","ClearingFirmID","TransactionID","Buy/Sell","IBOrderID","IBExecID","BrokerageOrderID","OrderReference","VolatilityOrderLink","ExchOrderID","ExtExecID","OrderTime","OpenDateTime","HoldingPeriodDateTime","WhenRealized","WhenReopened","LevelOfDetail","ChangeInPrice","ChangeInQuantity","OrderType","TraderID","IsAPIOrder","AccruedInterest"\n"<ACC-ID>","","","USD","1","STK","ALK","ALASKA AIR GROUP INC","4352","US0116591092","ISIN","011659109","US0116591092","NYSE","","","","","","1","","","3608520076","","20210312","","20210312,133059","20210312","20210316","ExchTrade","IBKRATS","0.3088","68.41","21.125008","-21.125008","0","-0.21125008","USD","-21.33625808","68.02","O","FP;P","21.33625808","0","0","-0.1204","0","","","0","","15738557515","BUY","1807825233","000102df.604bd43a.01.01","0030c27b.0000ebaf.604b5ec3.0001","","","N/A","0030c27b.00018c81.609ee3f4.0001","20210312,133059","","","","","EXECUTION","0","0","LMT","","N","0"\n"<ACC-ID>","","","USD","1","STK","ALK","ALASKA AIR GROUP INC","4352","US0116591092","ISIN","011659109","US0116591092","NYSE","","","","","","1","","","3608520082","","20210312","","20210312,133059","20210312","20210316","ExchTrade","IBKRATS","7","68.41","478.87","-478.87","0","-0.78874992","USD","-479.65874992","68.02","O","P","479.65874992","0","0","-2.73","0","","","0","","15738557532","BUY","1807825233","0000d322.604bcd09.01.01","0030c27b.0000ebaf.604b5ec3.0001","","","N/A","378047500B"
...
...
The check_statement_response
checks if 'FlexQueryResponse' is in the result - why is that required? and is it tested for all types of outputs (txt, XML, csv)?
Edit: maybe it's better to allow a parameter to skip checking in the client.download(...)
function?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.