sdabhi23 / bsedata

Python library for extracting real-time data from Bombay Stock Exchange (India)

bsedata

Python library for extracting real-time data from Bombay Stock Exchange (India).

Tests on master branch	Tests on dev branch	Code coverage	Documentation

Introduction

bsedata is a library for collecting real-time data from Bombay Stock Exchange (India). It can be used in various types of projects which require getting live quotes for a given stock or index or build large data sets for data analysis.

The data is as accurate as provided on the BSE website.

Please do not use this application for production usage. It is best used for learning and building application for your own use. For commercial application you better buy a data service.

This library uses MIT license hence liability lies with the user and not the author of the application.

As per India's IT act it is no where mentioned that we cannot scrape a website. It's up to me whether I consume the publicly available data "visually" or through my programme or using some kind of "web reader". As long as data you are scrapping is available in public domain and you are not breaching security and accessing data of private nature with malicious intent, it cannot be termed as illegal.

But it can certainly be termed "unethical" (which is equally bad) since you are damaging the service itself. Hence be prudent about how much stress you cause to the backend. BSE's website is very capable and even they will not mind if someone is scrapping their website for educational purposes.

Features

• Getting live quotes using stock codes in Python dicttionaries
• Getting list of top losers
• Getting list of top gainers
• Getting quotes for all the indices traded in BSE
• Helper APIs to check whether a given stock code or index code is correct
• Getting list of all indices and stocks

Roadmap

• Get details of an individual index
• Complete unit test coverage
• Daily OHLCV data

Dependencies

• BeautifulSoup 4
• Requests
• lxml
• A working internet connection 😉

Usage

Refer the documentation at https://bsedata.readthedocs.io/en/latest/usage.html

Change Log

Refer the documentation at https://bsedata.readthedocs.io/en/latest/chglog.html

License

MIT License

Copyright (c) 2018 - 2024 Shrey Dabhi

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

bsedata's Issues

Do update scrip code methods need to have access to local file system?

It seems the update script code methods needs access to the local file system, ' BSE(update_codes=True)andbse_object.updateScripCodes()`.

I get an OS error when deploying an app on AWS, Read-only file system: 'stk.json'

Do these methods need access to local file systems? Is there a workaround for this?

getScriptCode function is not working

Some of the Stocks are not listed in bsedata

I tried getting the value of Rail Vikas Nigam Limited but this is not mentioned in getQuotes().

but when I try to get the values by sending scripcode, it fetches the correct value.

I got the scripcode from BSE website.

Additions of Financial Data (Results & Annual Reports)

Financial Tab -> Annual Reports :
https://www.bseindia.com/stock-share-price/tata-motors-ltd/tatamotors/500570/financials-annual-reports/

Financial Tab -> Results :
https://www.bseindia.com/stock-share-price/tata-motors-ltd/tatamotors/500570/financials-results/
Prior Period:
https://www.bseindia.com/corporates/Comp_Results.aspx?Code=500570

Getting Inactive Stock error

Hi,
I tried VerifyScriptCode() then tried to get Quote of a company stilll getting error
Can you please help me ?

Historical EOD data for stocks

I want to download historical EOD OHLCV data for stocks. Is it gonna be added in this library or can you recommend a library where we can download bse historical EOD data from bse.

airtel data is not avaialble, getting error

from bsedata.bse import BSE
from pprint import pprint

b = BSE(update_codes = True)

q = b.getQuote('890157')
pprint(q)

Issue with getPeriodTrend

I've been using bsedata for personal analysis but recently the getPeriodTrend function has been giving me an error.

AttributeError: 'NoneType' object has no attribute 'split'

After fetching quotes for few companies it shows AttributeError.

Traceback (most recent call last):
  File "data.py", line 121, in <module>
    get_data()
  File "data.py", line 14, in get_data
    quote = b.getQuote(company_code)
  File "/home/masternachiket/anaconda3/envs/Trader/lib/python3.6/site-packages/bsedata/bse.py", line 60, in getQuote
    return quote.quote(scripCode)
  File "/home/masternachiket/anaconda3/envs/Trader/lib/python3.6/site-packages/bsedata/quote.py", line 47, in quote
    res['change'] = span.string.split('(')[0].strip()
AttributeError: 'NoneType' object has no attribute 'split'

Need additional parameters for Period Trend Data

the .getPeriodTrend() function only gives ( date : , value : , vol : ). Is there any way that i can get additional paramters or attributes from this like 52H, 52L etc.?

KeyError: 'priceBand'

For past one week or so, I was not able to get the updated script quotes. Here is the complete error that shows up:

---

KeyError Traceback (most recent call last)
in
----> 1 q = b.getQuote('533022')
2 print(q)

C:\ProgramData\Anaconda3\lib\site-packages\bsedata\bse.py in getQuote(self, scripCode)
58 :return: a dictionary which contain details about the stock
59 """
---> 60 return quote.quote(scripCode)
61
62 def getIndices(self, category):

C:\ProgramData\Anaconda3\lib\site-packages\bsedata\quote.py in quote(scripCode)
94 continue
95
---> 96 if res['priceBand'] != '':
97 for tbody in soup('tbody'):
98 try:

KeyError: 'priceBand'

Would be great, if the issue gets fixed at the earliest.

getting connection error

I have started getting an error suddenly while running the code.
Following is the error:

Traceback (most recent call last):
  File "C:\Python\lib\site-packages\urllib3\connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "C:\Python\lib\site-packages\urllib3\util\connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "C:\Python\lib\socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Python\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Python\lib\site-packages\urllib3\connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "C:\Python\lib\site-packages\urllib3\connection.py", line 308, in connect
    conn = self._new_conn()
  File "C:\Python\lib\site-packages\urllib3\connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x0000013E860174F0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python\lib\site-packages\requests\adapters.py", line 439, in send
    resp = conn.urlopen(
  File "C:\Python\lib\site-packages\urllib3\connectionpool.py", line 724, in urlopen
    retries = retries.increment(
  File "C:\Python\lib\site-packages\urllib3\util\retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='m.bseindia.com', port=443): Max retries exceeded with url: /StockReach.aspx?scripcd=534976 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000013E860174F0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    q = b.getQuote('534976')
  File "C:\Python\lib\site-packages\bsedata\bse.py", line 60, in getQuote
    return quote.quote(scripCode)
  File "C:\Python\lib\site-packages\bsedata\quote.py", line 36, in quote
    res = requests.get(baseurl + scripCode, headers=headers)
  File "C:\Python\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
  File "C:\Python\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Python\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "C:\Python\lib\site-packages\requests\adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='m.bseindia.com', port=443): Max retries exceeded with url: /StockReach.aspx?scripcd=534976 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x0000013E860174F0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))

Can anyone help????

Fine organics scrip not present

Fine Organic Industries Ltd6
(FINEORG | 541557 | INE686Y01026)

b.verifyScripCode('541557')) -- returns None even after running updateScripCodes()

Error on getQuote

When I am calling function getQuote with script code 590116 ex: objBSE.getQuote('590116').
I am getting an error:

---

InvalidStockException Traceback (most recent call last)
Cell In[14], line 1
----> 1 objBSE.getQuote('590116')

File ~/Library/Python/3.9/lib/python/site-packages/bsedata/bse.py:61, in BSE.getQuote(self, scripCode)
55 def getQuote(self, scripCode):
56 """
57 :param scripCode: A stock code
58 :returns: A dictionary which contain details about the stock
59 :raises InvalidStockException: Raised for stocks which have been suspended or no longer trading on BSE
60 """
---> 61 return quote.quote(scripCode)

File ~/Library/Python/3.9/lib/python/site-packages/bsedata/quote.py:53, in quote(scripCode)
51 if error_text_element is not None:
52 error_text=error_text_element.text
---> 53 raise InvalidStockException(status=error_text)
54 try:
55 if span['class'][0] == 'srcovalue':

InvalidStockException: Inactive stock

Currently, I am using this library. If you will allow, I can contribute to this library.

json Error while fetching historic data

from bsedata.bse import BSE
b = BSE(update_codes = True)
b.getPeriodTrend('524804', '12M')
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\Askanksha\AppData\Roaming\Python\Python310\site-packages\bsedata\bse.py", line 111, in getPeriodTrend
return periodTrend.getPeriodTrend(scripCode, timePeriod)
File "C:\Users\Askanksha\AppData\Roaming\Python\Python310\site-packages\bsedata\periodTrend.py", line 45, in getPeriodTrend
data = json.loads(res.content.decode('utf-8'))
File "C:\Program Files\Python310\lib\json_init_.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Program Files\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 3 column 1 (char 4)

Anyone else seeing this issue?
I am using 3.10.10 python version

Problem with getQuote()

when using the method getQuote with scripCodes like '538936', an error AttributeError: 'NoneType' object has no attribute 'split' is being thrown

Error while Fetching data from BSE

I am using the below code to fetch data from BSE but I am getting error. Not sure the reason for the error need help. Thanks

=======================CODE==============================
from pprint import pprint
from bsedata.bse import BSE
b = BSE()
print (b)

b.getQuote('541151')
pprint(b)
=========================ERROR===================================
Driver Class for Bombay Stock Exchange (BSE)
Traceback (most recent call last):
File "D:\Data\BSE Test.py", line 9, in
b.getQuote('541151')
File "C:\Users\srinath\AppData\Local\Programs\Python\Python38\lib\site-packages\bsedata\bse.py", line 60, in getQuote
return quote.quote(scripCode)
File "C:\Users\srinath\AppData\Local\Programs\Python\Python38\lib\site-packages\bsedata\quote.py", line 96, in quote
if res['priceBand'] != '':
KeyError: 'priceBand'

Historical Deliverable quantity data of each stock.

I am not able to extract the "deliverable volume" or "deliverable quantity" of any stock . Could you help me with this.

AttributeError: 'NoneType' object has no attribute 'split'

when I try to get detail of this code '538651' then give me Error like this

res['change'] = span.string.split('(')[0].strip()

this Exception generates due to not found change, industry or any other fields.
so please fix this issue

getScripCodes() returns error if updateScripCodes() is not executed before.

Executing getScripCodes returns in stk.json file not found error. To overcome this updateScripCodes() needs to be executed first as this makes the file stk.json.

Calling getScripCodes() throws FileNotFoundError

from bsedata.bse import BSE
b = BSE()
b.getScripCodes()

the above snippet throws FileNotFoundError as below.

C:\ProgramData\Anaconda3\lib\site-packages\bsedata\bse.py in getScripCodes(self)
89 :return: a dictionary with scrip codes as keys and company names as values
90 """
---> 91 f = open('stk.json', 'r')
92 return json.loads(f.read())
93

FileNotFoundError: [Errno 2] No such file or directory: 'stk.json'

AttributeError: 'NoneType' object has no attribute 'split'

---

Exception happened during processing of request from ('171.126.61.220', 60999)
Traceback (most recent call last):
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 138, in run
self.finish_response()
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 180, in finish_response
self.write(data)
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 274, in write
self.send_headers()
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 333, in send_headers
self._write(bytes(self.headers))
File "/usr/local/python3/lib/python3.7/wsgiref/headers.py", line 142, in bytes
return str(self).encode('iso-8859-1')
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 139-140: ordinal not in range(256)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 141, in run
self.handle_error()
File "/root/.virtualenvs/django/lib/python3.7/site-packages/django/core/servers/basehttp.py", line 88, in handle_error
super(ServerHandler, self).handle_error()
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 368, in handle_error
self.finish_response()
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 180, in finish_response
self.write(data)
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 274, in write
self.send_headers()
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 331, in send_headers
if not self.origin_server or self.client_is_modern():
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 344, in client_is_modern
return self.environ['SERVER_PROTOCOL'].upper() != 'HTTP/0.9'
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/python3/lib/python3.7/socketserver.py", line 647, in process_request_thread
self.finish_request(request, client_address)
File "/usr/local/python3/lib/python3.7/socketserver.py", line 357, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/local/python3/lib/python3.7/socketserver.py", line 717, in init
self.handle()
File "/root/.virtualenvs/django/lib/python3.7/site-packages/django/core/servers/basehttp.py", line 155, in handle
handler.run(self.server.get_app())
File "/usr/local/python3/lib/python3.7/wsgiref/handlers.py", line 144, in run
self.close()
File "/usr/local/python3/lib/python3.7/wsgiref/simple_server.py", line 35, in close
self.status.split(' ',1)[0], self.bytes_sent
AttributeError: 'NoneType' object has no attribute 'split'

Missing Price Band attribute

I have observed a very weird behavior in few responses where the price band attribute is not found on the page.

Reproduced randomly and accidentally in browser as well (on https://m.bseindia.com)

Cause: Unknown

Work around: Opened a stock page from https://bseinda.com which was also missing Price Band and force refresh it (shortcut key for Firefox: Ctrl + F5, for Chrome: Shift + F5).

Require better error messages (quote.py)

Hi,
I am using ur package and got error for BSE code 513715. The error is at Line number 43 in quote.py.

Kindly this this into consideration.

Your code is working fine for others BSE code.

I am attaching the image of error, what I have got.

KeyError: 'priceBand' in getQuote

>>> from bsedata.bse import BSE
>>> bse = BSE(update_codes=True)
>>> bse.getQuote("516072")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.7/site-packages/bsedata/bse.py", line 61, in getQuote
    return quote.quote(scripCode)
  File "/usr/local/lib/python3.7/site-packages/bsedata/quote.py", line 106, in quote
    if res['priceBand'] != '':
KeyError: 'priceBand'

the package seems to be raising this new error.

Problem Fetching some Indices

from bsedata.bse import BSE
b = BSE()
print(b)
indices = b.getIndices(category='market_cap/broad')
print(indices)

OUTPUT-:
Driver Class for Bombay Stock Exchange (BSE)
{'indices': []}

i am not able to get any data for this category as well as for some others too.

Also i would request to let me know if there is any way where i could fetch historical data of the indices as well as for the stocks which are not listed on nse.

Not able to run the project

bsedata.txt
Kindly see the attached file and let me know, how do I run the project.

Create custom exception

when using the method getQuote with scripCodes like '538936', an error AttributeError: 'NoneType' object has no attribute 'split' is being thrown