YahooFinancials python error: TypeError: string indices must be integers about yahoofinancials

[JECSand]

@zachpaul38
Could you post a code sample (with tickers you are trying) and full traceback?
Also what version of python are you using? Please note that support for older versions of Python, notably 2.7, was dropped in v1.7.

From my end using Python 3.10 I'm unable to recreate the error:

from yahoofinancials import YahooFinancials
tech_stocks = ['AAPL', 'MSFT', 'INTC']
yahoo_financials_tech = YahooFinancials(tech_stocks)
mc = yahoo_financials_tech.get_market_cap()
print(mc)

returns:

{'AAPL': 1981512941568, 'MSFT': 1785047416832, 'INTC': 110706778112}

[zachpaul38]

Hi Connor, Thanks for the quick response! I am using anaconda with python 3.9 runtime. I have run this get_market_cap() function over the last 1-2 years but am just now having trouble. Here's the tracebook after pasting the code you sent me into a jupyter notebook block. I tried using version 1.6 and 1.7 but both are throwing the same error. I will try from another machine right now and let you know if that fixes the issue. Thanks! Zach

…

-- Python CODE --- from yahoofinancials import YahooFinancials tech_stocks = ['AAPL', 'MSFT', 'INTC'] yahoo_financials_tech = YahooFinancials(tech_stocks) mc = yahoo_financials_tech.get_market_cap() print(mc)

---------------------------------------------------------------------------TypeError Traceback (most recent call last) Input In [88], in <cell line: 4>() 2 tech_stocks = ['AAPL', 'MSFT', 'INTC'] 3 yahoo_financials_tech = YahooFinancials(tech_stocks)----> 4 mc = yahoo_financials_tech.get_market_cap() 5 print(mc) File ~/opt/anaconda3/lib/python3.9/site-packages/yahoofinancials/__init__.py:740, in YahooFinancials.get_market_cap(self) 739 def get_market_cap(self):--> 740 return self._stock_price_data('marketCap') File ~/opt/anaconda3/lib/python3.9/site-packages/yahoofinancials/__init__.py:658, in YahooFinancials._stock_price_data(self, data_field) 656 ret_obj = {} 657 for tick in self.ticker:--> 658 if self.get_stock_price_data()[tick] is None: 659 ret_obj.update({tick: None}) 660 else: File ~/opt/anaconda3/lib/python3.9/site-packages/yahoofinancials/__init__.py:606, in YahooFinancials.get_stock_price_data(self, reformat) 604 def get_stock_price_data(self, reformat=True): 605 if reformat:--> 606 return self.get_clean_data(self.get_stock_tech_data('price'), 'price') 607 else: 608 return self.get_stock_tech_data('price') File ~/opt/anaconda3/lib/python3.9/site-packages/yahoofinancials/__init__.py:494, in YahooFinanceETL.get_stock_tech_data(self, tech_type) 492 return self.get_stock_data(statement_type='keystats', tech_type=tech_type) 493 else:--> 494 return self.get_stock_data(tech_type=tech_type) File ~/opt/anaconda3/lib/python3.9/site-packages/yahoofinancials/__init__.py:481, in YahooFinanceETL.get_stock_data(self, statement_type, tech_type, report_name, hist_obj) 479 for tick in self.ticker: 480 try:--> 481 dict_ent = self._create_dict_ent(tick, statement_type, tech_type, report_name, hist_obj) 482 data.update(dict_ent) 483 except ManagedException: File ~/opt/anaconda3/lib/python3.9/site-packages/yahoofinancials/__init__.py:409, in YahooFinanceETL._create_dict_ent(self, up_ticker, statement_type, tech_type, report_name, hist_obj) 407 elif tech_type != '' and statement_type != 'history': 408 try:--> 409 re_data = self._scrape_data(YAHOO_URL, tech_type, statement_type) 410 except KeyError: 411 re_data = None File ~/opt/anaconda3/lib/python3.9/site-packages/yahoofinancials/__init__.py:169, in YahooFinanceETL._scrape_data(self, url, tech_type, statement_type) 167 stores = data["context"]["dispatcher"]["stores"]["QuoteSummaryStore"] 168 elif tech_type != '' and statement_type != 'history':--> 169 stores = data["context"]["dispatcher"]["stores"]["QuoteSummaryStore"][tech_type] 170 else: 171 stores = data["context"]["dispatcher"]["stores"]["HistoricalPriceStore"] TypeError: string indices must be integers [image: image.png]

On Tue, Jan 3, 2023 at 11:05 PM Connor Sanders ***@***.***> wrote: @zachpaul38 <https://github.com/zachpaul38> Could you post a code sample (with tickers you are trying) and full traceback? Also what version of python are you using? Please note that support for older versions of Python, notably 2.7, was dropped in v1.7. From my end using Python 3.10 I'm unable to recreate the error: from yahoofinancials import YahooFinancialstech_stocks = ['AAPL', 'MSFT', 'INTC']yahoo_financials_tech = YahooFinancials(tech_stocks)mc = yahoo_financials_tech.get_market_cap()print(mc) returns: {'AAPL': 1981512941568, 'MSFT': 1785047416832, 'INTC': 110706778112} — Reply to this email directly, view it on GitHub <#110 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZDMBT7OP7SHLPRLI7SIUDLWQSBAJANCNFSM6AAAAAATPYIK7U> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[JECSand]

@zachpaul38 I can try running this from anaconda as well. Are you getting this error when using 1.7 outside of anaconda?

[ivojfabris]

I am getting the same errors when I use this function. It has working for me for over a year and it stopped working.

part of the traceback using the posted above.
File "C:\Users\User\AppData\Roaming\Python\Python37\site-packages\yahoofinancials_init_.py", line 409, in _create_dict_ent
re_data = self.scrape_data(YAHOO_URL, tech_type, statement_type)
File "C:\Users\User\AppData\Roaming\Python\Python37\site-packages\yahoofinancials_init.py", line 169, in _scrape_data
stores = data["context"]["dispatcher"]["stores"]["QuoteSummaryStore"][tech_type]
TypeError: string indices must be integers

I would like to note that other functions have stopped working, making me think that there is something wrong with parsing.

[zachpaul38]

Interestingly enough I was able to get this function to work earlier today using a new python anaconda 3.7 env. It may be that there’s a conflict with another python library? Also note that this problem happened to me in the second or third quarter of last year but I was able to resolve it then while tinkering with the environments at that time as well. Zach

…

On Fri, Jan 6, 2023 at 4:19 PM ivojfabris ***@***.***> wrote: I am getting the same errors when I use this function. It has working for me for over a year and it stopped working. part of the traceback using the posted above. File "C:\Users\User\AppData\Roaming\Python\Python37\site-packages\yahoofinancials_ *init*_.py", line 409, in _create_dict_ent re_data = self. *scrape_data(YAHOO_URL, tech_type, statement_type) File "C:\Users\User\AppData\Roaming\Python\Python37\site-packages\yahoofinancials_init*.py", line 169, in _scrape_data stores = data["context"]["dispatcher"]["stores"]["QuoteSummaryStore"][tech_type] TypeError: string indices must be integers I would like to note that other functions have stopped working, making me think that there is something wrong with parsing. — Reply to this email directly, view it on GitHub <#110 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZDMBTYJREF53WGSJSTQWADWRCD45ANCNFSM6AAAAAATPYIK7U> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ivojfabris]

I am not using anaconda myself. I am using native python. The error happened in a python virtual environment but I did not update another library. This stopped working for me in my other desktop but it seems consistent. I am surprised that other people are not getting this error. Do you know what you changed? I am convinced that there was something wrong with the parsing.

[zachpaul38]

I am just a consumer of this library like yourself so I don’t know for sure. Since the functions work in some python environments, I doubt that the parser is the culprit, although I was also convinced that the parser had a bug in it until it worked under a new env.

…

On Fri, Jan 6, 2023 at 4:38 PM ivojfabris ***@***.***> wrote: I am not using anaconda myself. I am using native python. The error happened in a python virtual environment but I did not update another library. This stopped working for me in my other desktop but it seems consistent. I am surprised that other people are not getting this error. Do you know what you changed? I am convinced that there was something wrong with the parsing. — Reply to this email directly, view it on GitHub <#110 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZDMBTYYE57W2Y4YYPQJBADWRCGEDANCNFSM6AAAAAATPYIK7U> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ivojfabris]

So I been looking how the code works and it seems the error in my end is on the parser. The parsers builds the json with respect to "context" all the way down to "stores". However, the stores contains a string that seem to be base64 code. Once the python object looks for dictionary key it gets an error.

@JECSand do you happen to know how regex works in this case? Trying to decode base64 seems to fail, perhaps I am regex is not capturing the entire string? I did install jinja2 recently.

[JECSand]

@ivojfabris The only change that has occurred is Yahoo Finance seems to have encrypted their API response data. To solve this issue we had to add "_decryptData" on line 180. I just ran a test just now and it still works for me.

Some possibilities to look into:

1. What version of "pycryptodome" is being installed on your machine for this? I am using pycryptodome==3.16.0. I may need to specify a hard version of that package to get this working universally.

2. What OS are you running this on? Seems like it maybe from a windows machine. I've tested this on Linux and Mac. I can test from a Windows VM after I get off work this evening.

I can also research better ways to decrypt the data.

[JECSand]

@ivojfabris and @zachpaul38
I made some quick updates to the development branch, mainly swapped out pycryptodome with cryptography. Would you mind git cloning that branch and testing these changes out?

[ivojfabris]

@JECSand

I get the error in two platform. My server is running Ubuntu 20. My desktop is running Windows10. I used one of my laptops using Windows 11 and installed the package and it worked. However, if I run the code independently I do get the same error.

I do not have pycryptodome nor cryptography installed , I am not sure if it was one of the dependencies. However, let me install it and I'll test out. Thanks for the quick response and the help!

[ivojfabris]

@JECSand

I uploaded the development version to both my desktop and server and I no longer get any issues.

Yahoo Finance encrypted the data with EAS? Are you guys hosting the keys or are they publicly available?

Thanks again for the help.

[JECSand]

@ivojfabris Its publicly available. Going to push v1.8 tonight with the changes, hopefully this resolves your issue.

[JECSand]

@ivojfabris @zachpaul38 I just released v1.8, can you both confirm that this resolved the issue? I'll keep this issue open just in case.

[ivojfabris]

@JECSand I been testing the code from development branch. All my issues have been resolved. Looking at the changes you made it made sense why the old code would break, So I believe you can close the issue, but @zachpaul38 might feel differently.

Thanks again for resolving this issue so quickly.

[JECSand]

@ivojfabris Perfect and no problem at all, going to close this issue then.