Comments (4)
Please describe your problem more clearly. What does "slow to consumer the msg" mean? Is there any exception being thrown?
The problem might be that you are using a timeout of 0 seconds which might be too little for your queue. Use None
, or some large enough value.
from stompest.
what is the stompest consumer's max QPS?
I use the stompest consumer for activemq to consume message.
But sometimes the queue has many message block.
I found the problem is stompest consumer consume the message rate is slow.
class ActiveMQConsumer(object):
def __init__(self, dest, uri, login, passcode, message_type=None):
self.dest = dest
self.config = StompConfig(uri=uri, login=login, passcode=passcode)
self.message_type = message_type
self.client = Stomp(self.config)
self.parser = StompParser('1.0')
self.client.connect()
headers = {
StompSpec.ACK_HEADER: StompSpec.ACK_CLIENT_INDIVIDUAL,
'activemq.prefetchSize': '1000',
}
if message_type == 'MapMessage':
headers['transformation'] = 'jms-map-json'
self.client.subscribe(self.dest, headers)
def __map_json_to_dict(self, body):
ret_dict = {}
body_dict = json.loads(body)
for entry in body_dict['map']['entry']:
key = entry['string'][0]
value = entry['string'][1]
ret_dict[key] = value
return ret_dict
def recv(self):
if self.client.canRead(0):
self.frame = self.client.receiveFrame()
self.client.ack(self.frame)
if self.message_type == 'MapMessage':
body = self.__map_json_to_dict(self.frame.body)
else:
body = json.loads(self.frame.body)
return body
else:
return None
def ack(self):
self.client.ack(self.frame)
def __del__(self):
self.client.disconnect()
I use mutl-thread function to consume the message
mq_consumer = ActiveMQConsumer(dest=config.get('activemq', 'document_audit_consumer_dest'),
uri=config.get('activemq', 'document_audit_uri'),
login=config.get('activemq', 'document_audit_login'),
passcode=config.get('activemq', 'document_audit_passcode'),
message_type='MapMessage')
while True:
try:
msg = mq_consumer.recv()
if msg:
# LOG.debug(msg)
msg_queue.put(msg)
except Exception, e:
LOG.error(e, exc_info=True)
from stompest.
Please boil your problem down. Which queue is blocking? The ActiveMQ queue or your local msg_queue
? The sync
client can consume a lot of messages per second, definitely more than one. Please run the sync
example to convince yourself.
from stompest.
Please close this problem.
I restart the activemq, and the consumer qps is ok now.
Maybe is the problem is activemq.
from stompest.
Related Issues (20)
- ipv6 support HOT 1
- How to config exclusive=true ? HOT 4
- Messages not recieving after queue is idle for some time HOT 1
- failover option 'startupMaxReconnectAttempts' default value should be -1
- Support for asyncio HOT 2
- Asynch examples on ActiveMQ - Artemis HOT 1
- support SSL client auth in stompest.async HOT 5
- Exception <class 'select.error'>: (4, 'Interrupted system call') after upgrading stompest HOT 4
- stompest.async will be unusable on python 3.7+ HOT 10
- receiveFrame does not accept a timeout parameter HOT 3
- Thread-unsafe issue if we set ack=True in SubscriptionListener HOT 3
- How to dynamically increase the number of consumers HOT 1
- Invalid SUBSCRIBE format HOT 2
- StompFrame and StompHeartBeat __str__ method returns bytes in Python 3
- failover doesn't seem to be working HOT 1
- how to get JMSXDeliveryCount in the stomp frame HOT 1
- When use version=1.2, the consumer stop to connect on topic
- collections.MutableMapping is not available in Python 3.10 HOT 7
- STOMP over websocket
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stompest.