ranjanrak / optionchainstream

Live streaming option chain for equity derivatives using Kite connect Websocket based on redis.

Option Chain Stream

Live streaming option chain for equity derivatives using Kite connect Websocket.

This package uses Redis as storage backend. Here Redis is used to store real time streaming websocket data and instruments detail i.e all option strike details for contracts. Combination of these data structure are used to create real-time option chain stream for any given instrument.

Installation

pip install optionchain_stream

Request parameters

Field	Type	Detail
api_key	string	Kite connect API key
secret_key	string	Kite connect API secret
request_token	string	Kite connect one-time token obtained after the login flow
access_token	string	The authentication token obtained post the login flow using request_token and secret_key
option_symbol	string	Symbol of the instrument(eg: NIFTY, SBIN, ONGC, etc)
option_expiry_date	string	Option expiry date in yyyy-mm-dd format(eg: '2021-02-25', '2021-04-29')
underlying	boolean	Add NSE-EQ underlying stock in option stream for True input

Usage

from optionchain_stream import OptionChain
OptionStream = OptionChain("option_symbol", "option_expiry_date in yyyy-mm-dd format", "api_key",
                    "api_secret=None", "request_token=None", "access_token=None", underlying=False)

# You can directly pass access_token from previous active session 
Eg: OptionStream = OptionChain("ONGC", "2021-02-25", "your_api_key", access_token="XXXXXX")

# Generate new session by passing api_secret and request_token
Eg: OptionStream = OptionChain("ONGC", "2021-02-25", "your_api_key", api_secret="XXXXX",
                    request_token="XXXXXX")

# You can fetch underlying stock tick as well in option chain, by sending optional param `underlying=True`
Eg: OptionStream = OptionChain("ONGC", "2021-02-25", "your_api_key", access_token="XXXXXX", underlying=True)
                  Or
    OptionStream = OptionChain("ONGC", "2021-02-25", "your_api_key", api_secret="XXXXX",
                    request_token="XXXXXX", underlying=True)


# Sync master instrument data to DB(redis)     
# This sync is required only once daily at initial run             
OptionStream.sync_instruments()

# Stream option chain data in real-time
StreamData = OptionStream.create_option_chain()
for data in StreamData:
    print(data)

Response

Responses are JSON messages.
1> For underlying=True, response contain underlying stock tick as well. Eg. for option_symbol=ONGC, underlying is ONGC EQ NSE contract:

....{'token': 633601, 'symbol': 'ONGC', 'last_price': 112.35, 'change': 0.4470272686633885},....,
{'token': 24268034, 'symbol': 'ONGC21FEB87PE', 'last_price': 1.5, 'volume': 61600, 'change': 0.0, 'oi': 400400},.....

2> By default, option contracts are sent.

...., 'change': 54.09090909090908, 'oi': 7700},{'token': 24268034, 'symbol': 'ONGC21FEB87PE', 'last_price': 1.5, 'volume': 61600, 'change': 0.0, 'oi': 400400}, 
{'token': 24268290, 'symbol': 'ONGC21FEB88CE', 'last_price': 10.6, 'volume': 0, 'change': -12.033195020746897, 'oi': 15400}, {'token': 24268546, 'symbol': 
'ONGC21FEB88PE', 'last_price': 1.75, 'volume': 53900, 'change': 25.000000000000007, 'oi': 261800}, {'token': 24268802, 'symbol': 'ONGC21FEB89CE', 
'last_price':10.6, 'volume': 15400, 'change': -5.77777777777778, 'oi': 107800}, 
{'token': 24269058, 'symbol': 'ONGC21FEB89PE', 'last_price': 1.85, 'volume': 184800, 'change': -11.904761904761905, 'oi': 338800}, {'token': 24269314, 'symbol': 
'ONGC21FEB90CE', 'last_price': 10.0, 'volume': 654500, 'change': 2.5641025641025643, 'oi': 1632400}, {'token': 24269570, 'symbol': 'ONGC21FEB90PE', 'last_price': 
2.2, 'volume': 1925000, 'change': -2.2222222222222143, 'oi': 3326400}, {'token': 24269826, 'symbol': 'ONGC21FEB91CE', 'last_price': 9.3, 'volume': 15400, 
'change': -6.999999999999993, 'oi': 308000}, {'token': 24270082, 'symbol': 'ONGC21FEB91PE', 'last_price': 2.55, 'volume': 61600, 'change': -5.555555555555569, 
'oi': 323400}, {'token': 24270338, 'symbol': 'ONGC21FEB92CE', 'last_price': 8.7, 'volume': 30800,....

Response attributes

Field	Type	Detail
token	string	instrument_token for requested tradingsymbol
symbol	string	tradingsymbol of the instrument
last_price	float	Current market price
volume	int	Volume traded for the day
change	float	Price change % w.r.t previous day close/LTP
oi	float	Open Interest for a options contract

optionchainstream's Issues

pip install not working

Hi Ranjan, was excited to see this amazing peice of work and thank you for this

unfortunately have not been able to get the pip install to work and i get the following error

can you please correct this or set me in the right direction.

$ pip install optionchain_stream

Collecting optionchain_stream
Using cached optionchain_stream-0.2-py3-none-any.whl (8.2 kB)
ERROR: Requested optionchain_stream from https://files.pythonhosted.org/packages/e1/a9/c7da09471a507817f35fc06f182279f4bddaaf849a36e90d182b97c290d8/optionchain_stream-0.2-py3-none-any.whl#sha256=77a959ba13ac972a2a88969597cf36cf61f7df7c3ed27b2d4d5c9241ea7fd187 has different version in metadata: '0.1'

TypeError: cannot pickle '_thread.lock' object

I installed OptionChainStream package and executed below code.

from optionchain_stream import OptionChain

OptionStream = OptionChain("api_key", "secret_key", "request_token",

"option_symbol", "option_expiry_date in yyyy-mm-dd format")

Eg: OptionChain('XXXXXX', 'XXXXXXX', 'XXXXXX',

'ONGC', '2021-02-25')

OptionStream = OptionChain("xxx", "xxx", "xxx",
'ONGC', '2021-5-27')

Sync master instrument data to DB(redis)

This sync is required only once daily at initial run

OptionStream.sync_instruments()

Stream option chain data in real-time

StreamData = OptionStream.create_option_chain()

print(StreamData)
for data in StreamData:
print(data)

I am getting below error. Can you please help.

TypeError: cannot pickle '_thread.lock' object

URL Issue: NSE Site Not Accessible (400 Error)

Hey, I noticed that the NSE site link you gave isn't working—it's showing a 400 error. Here's the link you shared: https://archives.nseindia.com/content/fo/fo_mktlots.csv

Error on AWS

I am getting following error while running on AWS EC2, I using access token method..

I have checked the token it's working fine for other scripts.

TypeError: cannot pickle '_thread.lock' object

Hi,

I tried to upgrade the python package and run v0.5, but again it's showing the following error:

TypeError Traceback (most recent call last)
in
10 # Stream option chain data in real-time
11 StreamData = OptionStream.create_option_chain()
---> 12 for data in StreamData:
13 print(data)

~\anaconda3\lib\site-packages\optionchain_stream\option_chain.py in create_option_chain(self)
30 self.socketClient = WebsocketClient(self.api_key, self.api_secret, self.request_token, self.symbol, self.expiry)
31 # create streaming websocket data
---> 32 self.socketClient.queue_callBacks()
33 # Keep fetching streaming Queue
34 while 1:

~\anaconda3\lib\site-packages\optionchain_stream\websocket.py in queue_callBacks(self)
92 """
93 # Process to keep updating real time tick to DB
---> 94 Process(target=self.assign_callBacks,).start()
95 # Delay to let intial instrument DB sync
96 # For option chain to fetch value

~\anaconda3\lib\multiprocessing\process.py in start(self)
119 'daemonic processes are not allowed to have children'
120 _cleanup()
--> 121 self._popen = self._Popen(self)
122 self._sentinel = self._popen.sentinel
123 # Avoid a refcycle if the target function holds an indirect

~\anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
222 @staticmethod
223 def _Popen(process_obj):
--> 224 return _default_context.get_context().Process._Popen(process_obj)
225
226 class DefaultContext(BaseContext):

~\anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)
325 def _Popen(process_obj):
326 from .popen_spawn_win32 import Popen
--> 327 return Popen(process_obj)
328
329 class SpawnContext(BaseContext):

~\anaconda3\lib\multiprocessing\popen_spawn_win32.py in init(self, process_obj)
91 try:
92 reduction.dump(prep_data, to_child)
---> 93 reduction.dump(process_obj, to_child)
94 finally:
95 set_spawning_popen(None)

~\anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol)
58 def dump(obj, file, protocol=None):
59 '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60 ForkingPickler(file, protocol).dump(obj)
61
62 #

TypeError: cannot pickle '_thread.lock' object

Kindly let us know what could be the issue?

Thanks in advance

No connection could be made because the target machine actively refused it

How to solve this issue with this ?

[WinError 10061] No connection could be made because the target machine actively refused it.

Difficulty in running code

Please guide me how to execute option chain in this repo

reuse kite instance

Is there a way as of now to reuse kite session(created outside this package) instead of passing api_key and generating and new session

library raises key error because API response is changed

2023-04-03 11:07:44+0530 [KiteTickerClientProtocol (TLSMemoryBIOProtocol),client] Unhandled Error
        Traceback (most recent call last):
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/python/log.py", line 96, in callWithLogger
            return callWithContext({"system": lp}, func, *args, **kw)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/python/log.py", line 80, in callWithContext
            return context.call({ILogContext: newCtx}, func, *args, **kw)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/python/context.py", line 117, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/python/context.py", line 82, in callWithContext
            return func(*args, **kw)
        --- <exception caught here> ---
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/internet/posixbase.py", line 487, in _doReadOrWrite
            why = selectable.doRead()
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/internet/tcp.py", line 248, in doRead
            return self._dataReceived(data)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/internet/tcp.py", line 253, in _dataReceived
            rval = self.protocol.dataReceived(data)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/protocols/tls.py", line 329, in dataReceived
            self._flushReceiveBIO()
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/protocols/tls.py", line 295, in _flushReceiveBIO
            ProtocolWrapper.dataReceived(self, bytes)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/twisted/protocols/policies.py", line 110, in dataReceived
            self.wrappedProtocol.dataReceived(data)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/twisted/websocket.py", line 291, in dataReceived
            self._dataReceived(data)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/websocket/protocol.py", line 1213, in _dataReceived
            self.consumeData()
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/websocket/protocol.py", line 1225, in consumeData
            while self.processData() and self.state != WebSocketProtocol.STATE_CLOSED:
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/websocket/protocol.py", line 1591, in processData
            fr = self.onFrameEnd()
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/websocket/protocol.py", line 1713, in onFrameEnd
            self._onMessageEnd()
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/twisted/websocket.py", line 319, in _onMessageEnd
            self.onMessageEnd()
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/websocket/protocol.py", line 634, in onMessageEnd
            self._onMessage(payload, self.message_is_binary)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/autobahn/twisted/websocket.py", line 322, in _onMessage
            self.onMessage(payload, isBinary)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/kiteconnect/ticker.py", line 71, in onMessage
            self.factory.on_message(self, payload, is_binary)
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/kiteconnect/ticker.py", line 679, in _on_message
            self.on_ticks(self, self._parse_binary(payload))
          File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/optionchain_stream/websocket.py", line 47, in on_ticks
            'last_price':tick['last_price'], 'volume':tick['volume'], 'change':tick['change'],
        builtins.KeyError: 'volume'
        
2023-04-03 11:07:44+0530 [-] Connection error: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
2023-04-03 11:07:44+0530 [-] ERROR:root:closed connection on error: 1006 connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
2023-04-03 11:07:44+0530 [-] ERROR:kiteconnect.ticker:Connection closed: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
2023-04-03 11:07:44+0530 [-] ERROR:root:closed connection on close: 1006 connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
2023-04-03 11:07:44+0530 [KiteTickerClientProtocol (TLSMemoryBIOProtocol),client] <twisted.internet.tcp.Connector instance at 0x7fa53da843a0 disconnected IPv4Address(type='TCP', host='ws.kite.trade', port=443)> will retry in 2 seconds
2023-04-03 11:07:44+0530 [-] Stopping factory <kiteconnect.ticker.KiteTickerClientFactory object at 0x7fa5545cd390>
Process Process-2:
Traceback (most recent call last):
  File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/optionchain_stream/redis_instrument.py", line 70, in fetch_option_data
    token_data = json.loads(self.conn.get(optionContractKey))
  File "/usr/lib/python3.10/json/__init__.py", line 339, in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/optionchain_stream/websocket.py", line 31, in form_option_chain
    complete_option_data = self.instrumentClass.generate_optionChain(self.token_list)
  File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/optionchain_stream/instrument_file.py", line 117, in generate_optionChain
    optionData = self.redis_db.fetch_option_data(optionInstrument['symbol'], instrumentToken)
  File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/optionchain_stream/redis_instrument.py", line 72, in fetch_option_data
    raise Exception('Error - {}'.format(e))
Exception: Error - the JSON object must be str, bytes or bytearray, not NoneType
2023-04-03 11:07:47+0530 [-] Starting factory <kiteconnect.ticker.KiteTickerClientFactory object at 0x7fa5545cd390>
^C2023-04-03 11:07:48+0530 [-] Received SIGINT, shutting down.
Traceback (most recent call last):
  File "/home/root366/Dump/websocket streaming/test.py", line 12, in <module>
    for data in StreamData:
  File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/optionchain_stream/option_chain.py", line 45, in create_option_chain
    yield self.socketClient.q.get()
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 103, in get
    res = self._recv_bytes()
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 216, in recv_bytes
2023-04-03 11:07:48+0530 [-] ERROR:kiteconnect.ticker:Connection error: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
    buf = self._recv_bytes(maxlength)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 414, in _recv_bytes
2023-04-03 11:07:48+0530 [-] ERROR:root:closed connection on error: 1006 connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
    buf = self._recv(4)
  File "/usr/lib/python3.10/multiprocessing/connection.py", line 379, in _recv
2023-04-03 11:07:48+0530 [-] ERROR:kiteconnect.ticker:Connection closed: 1006 - connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
    chunk = read(handle, remaining)
2023-04-03 11:07:48+0530 [-] ERROR:root:closed connection on close: 1006 connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
KeyboardInterrupt
2023-04-03 11:07:48+0530 [KiteTickerClientProtocol (TLSMemoryBIOProtocol),client] <twisted.internet.tcp.Connector instance at 0x7fa53da843a0 disconnected IPv4Address(type='TCP', host='ws.kite.trade', port=443)> will retry in 2 seconds
2023-04-03 11:07:48+0530 [-] Stopping factory <kiteconnect.ticker.KiteTickerClientFactory object at 0x7fa5545cd390>
2023-04-03 11:07:48+0530 [-] Main loop terminated.

Can you please update code as we can clearly see in websocket.py, volume key is causing this issue.

File "/home/root366/enviroments/websocket streaming/lib/python3.10/site-packages/optionchain_stream/websocket.py", line 47, in on_ticks 'last_price':tick['last_price'], 'volume':tick['volume'], 'change':tick['change'], builtins.KeyError: 'volume'

Unable to live stream the bank nifty data

Dear @ranjanrak

from option_chain import OptionChain
import json

OptionStream = OptionChain("BANKNIFTY", "2022-12-04", "xxxxxx", access_token="xxxxxx", underlying=True)
# This sync is required only once daily at initial run             
OptionStream.sync_instruments()

# Stream option chain data in real-time
StreamData = OptionStream.create_option_chain()
for data in StreamData:
    print(data)
    content=json.dumps(data[0])
    with open("./optiondata.json", "w") as outfile:
         outfile.write(content)

It prints only one token data in loop. Am I doing any mistake here?

[{'token': 260105, 'symbol': 'NIFTY BANK', 'last_price': 43332.95, 'change': 0.5317402778180486}]

if I pass underlying value as False, it returns empty value

2022-12-05 22:08:42+0100 [-] Log opened.
2022-12-05 22:08:43+0100 [-] Connection error: 0 - error parsing tokens.
2022-12-05 22:08:43+0100 [-] ERROR:root:closed connection on error: 0 error parsing tokens.
[]
Traceback (most recent call last):

I have come across other issue in GitHub about TypeError: cannot pickle '_thread.lock' object #5". I have fixed it by below code snippet in websocket.py. it happens on windows and macOS

import platform
if platform.system() != "Linux":
    from multiprocessing import set_start_method
    set_start_method("fork")