refinitiv / real-time-sdk Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
After 45s the OmmConsumer reports an error and the Reactor shuts down.
2016-05-09 20:45:55,407 ERROR [main] OmmConsumerImpl(OmmConsumerImpl.java:1132): loggerMsg
ClientName: X_1
Severity: Error
Text: login failed (timed out after waiting 45000 milliseconds) for nylabads2:14002)
loggerMsgEnd
2016-05-09 20:46:05,410 ERROR [main] OmmConsumerImpl(ChannelCallbackClient.java:442): loggerMsg
ClientName: ChannelCallbackClient
Severity: Error
Text: Received ChannelDown event on channel Y
Consumer Name X_1
RsslReactor @7b227d8d
RsslChannel @3d680b5a
Error Id 0
Internal sysError 0
Error Location null
Error text Reactor shutting down...
loggerMsgEnd
Exception in thread "main" Exception Type='OmmInvalidUsageException', Text='login failed (timed out after waiting 45000 milliseconds) for nylabads2:14002)'
at com.thomsonreuters.ema.access.OmmConsumerImpl.ommIUExcept(OmmConsumerImpl.java:945)
at com.thomsonreuters.ema.access.OmmConsumerImpl.handleAdminReqTimeout(OmmConsumerImpl.java:1134)
at com.thomsonreuters.ema.access.OmmConsumerImpl.initialize(OmmConsumerImpl.java:196)
at com.thomsonreuters.ema.access.OmmConsumerImpl.<init>(OmmConsumerImpl.java:91)
at com.thomsonreuters.ema.access.EmaFactory.createOmmConsumer(EmaFactory.java:158)
at Consumer.<init>(Consumer.java:143)
at Consumer.main(Consumer.java:151)
Hi,
I'm having a problem using the EMA API when processing an update on field 1352 with an RmtesEnum type. I can reproduce the issue with the Consumer200 example by adding this to the decode
method:
case DataType::RmtesEnum:
cout << fe.getRmtes().toString();
break;
And registering an ongoing interest in BOL.ST and ALFA.ST.
When an update is received on field 1352, sometimes it will loop infinitely in RmtesBufferImpl.cpp in this while
loop if it is entered:
RsslRet retCode = rsslRMTESApplyToCache( &_rsslBuffer, &_rsslCacheBuffer );
while ( RSSL_RET_BUFFER_TOO_SMALL == retCode )
{
free( _rsslCacheBuffer.data );
_rsslCacheBuffer.length += _rsslCacheBuffer.length;
_rsslCacheBuffer.data = (char*)malloc( _rsslCacheBuffer.allocatedLength );
if ( !_rsslCacheBuffer.data )
{
throwMeeException( "Failed to allocate memory in RmtesBufferImpl::apply( const char* , UInt32 )" );
return;
}
}
Is this something you can help with?
Thanks
EMA cannot be compiled on OracleLinux 6 series. When running make under the folder 'Elektron-SDK/Cpp-C/Ema/Src/Access', it always fails with the message 'makefile:27: *** unsupported environment. Stop.'. After looking into the makefile, it seems that EMA will only compile on OracleServer 7. Is this a bug or 6 series will not be supported any more?
Javadoc for com.thomsonreuters.upa.codec.Date#isBlank()
says:
Returns true if all members in Time are set to the values used to signify blank.
"Time" should be replaced with "Date".
In the docs, there is no explanation about where to place EmaConfig.xm file on the file system and how an OmmConsumerConfig object can be constructed out of it. Could you please explain it in more details?
Hi,
If you provide
and such things into public Maven repository, users do not have to ant build
by themselves.
Actually, I'm using Scala and having hard time to write build.sbt
to clone this repo as submodule, ant build
and add the above JARs into classpath,
while it is really simple if JARs are available in some maven repo.
Thanks,
I have implemented a NIP using the EMA Java libs (latest build 3.1.0.L1-SNAPSHOT).
When attempting to make a request for the dictionary from ADH via the below
long fldHandle = provider.registerClient(EmaFactory.createReqMsg().domainType(EmaRdm.MMT_DICTIONARY)
.name("RWFFld").filter(EmaRdm.DICTIONARY_NORMAL), appClient);
where appClient implements OmmProviderClient
I receive a nullPointerException
Exception in thread "main" java.lang.NullPointerException
at com.thomsonreuters.ema.access.DictionaryItem.open(DictionaryCallbackClient.java:1126)
at com.thomsonreuters.ema.access.ItemCallbackClient.registerClient(ItemCallbackClient.java:1719)
at com.thomsonreuters.ema.access.OmmBaseImpl.registerClient(OmmBaseImpl.java:363)
at com.thomsonreuters.ema.access.OmmNiProviderImpl.registerClient(OmmNiProviderImpl.java:201)
at com.thomsonreuters.ema.access.OmmNiProviderImpl.registerClient(OmmNiProviderImpl.java:193)
at com.rbs.trep.NiProvider.main(NiProvider.java:72)
The exception appears to happen before the request has even been made to the ADH ie its on the application side.
Another user has confimed they have the same issue
https://community.developers.thomsonreuters.com/questions/16675/ema-nip-dictionary-request.html?childToView=16716
Can you advise if this is a bug or an error with the way I have implemented the request?
thanks
ETA includes the following:
EnumType enumType = dictionary.entryEnumType(dictionaryEntry, fidEnumValue);
System.out.println (enumType.display().toString());
EMA only exposes access to the underlying value.
I'm seeing a regression between Elektron 1.0.8 and 1.1.0. I am using the closure mechanism to track the subject used to make subscriptions. When the OmmConsumerClient is called back after making a series of requests the wrong closure is passed to the onRefreshMsg method.
The specific scenario where this fails is when I am snapshotting a RIC chain. I make a series of snapshot requests each with different closure objects:
Here is how the request is made:
void snapshot(SubscriptionContext context) {
RmdsSubject subject = context.getSubject();
LOGGER.info("closure:{}, item:{}", context, subject.getItem());
consumer.registerClient(EmaFactory.createReqMsg()
.serviceName(subject.getService())
.name(subject.getOMMItem())
.interestAfterRefresh(false), dispatchingClient, context);
}
Here is the OmmConsumerClient callback:
@Override
public void onRefreshMsg(RefreshMsg msg, OmmConsumerEvent consumerEvent) {
if (consumerEvent.closure() instanceof SubscriptionContext) {
SubscriptionContext context = (SubscriptionContext) consumerEvent.closure();
if (!msg.name().equals(context.getSubject().getItem())) {
LOGGER.warn("Bad context: Msg.name: {}, closure:{}, item:{}", msg.name(), consumerEvent.closure(), ((SubscriptionContext) consumerEvent.closure()).getSubject());
}
handleMarketPrice(context,
MessageType.IMAGE,
msg,
msg.hasSeqNum() ? msg.seqNum() : 0);
} else {
LOGGER.info("Got refresh: {}", msg);
}
}
And here are the logs that show that the closure object is being used for the wrong messages:
2017-05-10 17:30:57.362 [INFO ] [EventWorker_3] elektron.ElektronSubscriber - closure:com.mi.ahl.pubsub.elektron.ChainSnapshotContext$LinkContext@41d8484, item:JGB1555R7
...
2017-05-10 17:30:57.402 [WARN ] [pool-11-thread-1] client.DispatchingClient - Bad context: Msg.name: JGB1555S7, closure:com.mi.ahl.pubsub.elektron.ChainSnapshotContext$LinkContext@41d8484, item:RSF|JGB1555R7
Out of ~150 items there are almost 50% with the wrong context, it seems to change run by run. If I move back to Elektron 1.0.8 there are no problems.
This is a major regression, it makes 1.1.0 unusable as I cannot trust that the correct closures are being sent back in my callback methods.
Hi,
We're seeing an issue where we are getting a thread deadlock in the ReactorChannel
when trying to register a ric using the registerClient
method on the OmmConsumer
.
We've put synchronized
blocks whenever we use the OmmConsumer
(we're we are subscribing/unsubscribing from rics) , but we're still seeing this issue.
Are there any other parts of the SDK which are non-thread safe and would cause this issue?
at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
at com.thomsonreuters.upa.valueadd.reactor.ReactorChannel.submit(ReactorChannel.java:494)
at com.thomsonreuters.ema.access.SingleItem.rsslSubmit(ItemCallbackClient.java:1147)
at com.thomsonreuters.ema.access.SingleItem.open(ItemCallbackClient.java:1006)
at com.thomsonreuters.ema.access.ItemCallbackClient.registerClient(ItemCallbackClient.java:540)
at com.thomsonreuters.ema.access.OmmBaseImpl.registerClient(OmmBaseImpl.java:340)
at com.thomsonreuters.ema.access.OmmConsumerImpl.registerClient(OmmConsumerImpl.java:70)
Hi all,
I am trying to use Java version of EMA to post data to TRCC.
chp01-eap-emea1.thomsonreuters.com:443
Since no Java documentation is available, I am trying to follow the C++ tutorial.
https://developers.thomsonreuters.com/elektron/elektron-sdk-cc/learning?content=26452&type=learning_material_item
When I try to perform a client login, it throws a null pointer exception.
Exception in thread "pool-2-thread-1" java.lang.NullPointerException
at com.thomsonreuters.ema.access.TunnelItem.addSubItem(ItemCallbackClient.java:279)
at com.thomsonreuters.ema.access.SubItem.open(ItemCallbackClient.java:886)
at com.thomsonreuters.ema.access.ItemCallbackClient.registerClient(ItemCallbackClient.java:2180)
at com.thomsonreuters.ema.access.OmmBaseImpl.registerClient(OmmBaseImpl.java:338)
at com.thomsonreuters.ema.access.OmmConsumerImpl.registerClient(OmmConsumerImpl.java:114)
at com.thomsonreuters.ema.examples.training.consumer.series300.example341__MarketPrice__OffStreamPost.AppClient.onStatusMsg(Consumer.java:130)
at com.thomsonreuters.ema.access.ItemCallbackClientConsumer.notifyOnStatusMsg(ItemCallbackClient.java:2514)
at com.thomsonreuters.ema.access.ItemCallbackClient.statusEventCallback(ItemCallbackClient.java:1133)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.sendTunnelStreamStatusEventCallback(Reactor.java:932)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.sendAndHandleTunnelStreamStatusEventCallback(Reactor.java:944)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.handleTunnelStreamMsg(Reactor.java:3660)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.sendAndHandleDefaultMsgCallback(Reactor.java:821)
at com.thomsonreuters.upa.valueadd.reactor.WlItemHandler.callbackUser(WlItemHandler.java:2708)
at com.thomsonreuters.upa.valueadd.reactor.WlItemHandler.readRefreshMsg(WlItemHandler.java:2004)
at com.thomsonreuters.upa.valueadd.reactor.WlItemHandler.readMsg(WlItemHandler.java:1735)
at com.thomsonreuters.upa.valueadd.reactor.Watchlist.readMsg(Watchlist.java:277)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.performChannelRead(Reactor.java:1758)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.dispatchChannel(Reactor.java:1389)
at com.thomsonreuters.upa.valueadd.reactor.ReactorChannel.dispatch(ReactorChannel.java:469)
at com.thomsonreuters.ema.access.OmmBaseImpl.rsslReactorDispatchLoop(OmmBaseImpl.java:1128)
at com.thomsonreuters.ema.access.OmmBaseImpl.run(OmmBaseImpl.java:1261)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
where AppClient.onStatusMessage
ElementList elementList = EmaFactory.createElementList();
ElementEntry elementEntry = EmaFactory.createElementEntry();
elementEntry.ascii("Password", "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
ReqMsg reqMsg = EmaFactory.createReqMsg();
reqMsg.domainType(EmaRdm.MMT_SYSTEM)
.name("GEM-XXX-XXXXXX")
.privateStream(true)
.attrib(elementList)
.streamId(statusMsg.streamId());
System.out.println("Submit GenericMsg from onStatus");
subStreamHandle = ommConsumer.registerClient(reqMsg, this, 1, tunnelStreamHandle);
Is there any problems? Or am i doing anything wrong?
It will be great if someone could give me a hint on it.
Thanks.
I have been running a few tests and I'm having trouble with the resiliency of the OmmNiProvider. It appears from the logs that after a connection failure, everything recovers nicely and the reactor channel underneath is being dispatched again. However, any subscription clients do not see the messages that are sent after a network failure. Here are the steps:
The adh seems to recognise that the application is active again after the outage:
<rmds.1.adh.sourceSSLDistribution.sslDispatcher.EXT2: Info: Wed Aug 16 10:49:54 2017>
Server "EXT2" stopped for RSSL application "EXT2:2" from instance "LONMRT02S.46". The application terminated the session.
<END>
<rmds.1.adh.sourceSSLDistribution.sslDispatcher.channel: Warning: Wed Aug 16 10:50:10 2017>
Session disconnected (route "ripcSession.24", host "", port ). Channel initialization failed. Text:
<Impl/ripcsrvr.c:3335> Error: 1007 Unknown connection type.
<END>
<rmds.1.adh.sourceSSLDistribution.sslDispatcher.channel: Info: Wed Aug 16 10:50:10 2017>
Accepted source RSSL channel connection. SocketId: 24 Hostname: LONMRT02S IP: 10.200.31.216 Port: 53788
<END>
<rmds.1.adh.sourceSSLDistribution.sslDispatcher.EXT2: Info: Wed Aug 16 10:50:10 2017>
Server "EXT2" started for RSSL application "EXT2:0" from instance "LONMRT02S.48". The application sent a service up state.
<END>
The logs from the Reactor channel appear to show that everything has been setup correctly for the recovery:
2017-08-16 10:50:09.642 WARN [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: ChannelDictionary| Severity: Warning| Text: RDMDictionary stream state was changed to suspect with status message| streamId 3| Reason State: Open/Suspect/None - text: "channel down."| loggerMsgEnd|
2017-08-16 10:50:09.642 WARN [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: ChannelDictionary| Severity: Warning| Text: RDMDictionary stream state was changed to suspect with status message| streamId 4| Reason State: Open/Suspect/None - text: "channel down."| loggerMsgEnd|
2017-08-16 10:50:10,618 INFO [publisher] ElektronClient - Published 22 messages so far.
2017-08-16 10:50:10.618 TRACE [pub_worker_0] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: AHL_NIPROVIDER_2| Severity: Trace| Text: Received UpdateMsg with market domain; Handle = 1, user assigned streamId = 0.| loggerMsgEnd|
2017-08-16 10:50:10.618 ERROR [pub_worker_0] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: AHL_NIPROVIDER_2| Severity: Error| Text: Internal error: rsslChannel.submit() failed in OmmNiProviderImpl.submit(UpdateMsg)RsslChannel 0| Error Id -1| Internal sysError 0| Error Location Reactor.submitChannel| Error Text ReactorChannel is closed, aborting.| loggerMsgEnd|
2017-08-16 10:50:10,619 DEBUG [pub_worker_0] InjectableEmaFactory - Failed to submit UpdateMsg. Reason: ReactorReturnCodes.FAILURE. Error text: ReactorChannel is closed, aborting.
2017-08-16 10:50:10.679 INFO [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: ChannelCallbackClient| Severity: Info| Text: Received ChannelUp event on channel Channel| Instance Name AHL_CONSUMER_CUSTOM_1| Component Version ads2.6.9.L1.linux.tis.rrg 64-bit| loggerMsgEnd|
2017-08-16 10:50:10.680 INFO [ProviderDispatcher] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: ChannelCallbackClient| Severity: Info| Text: Received ChannelUp event on channel Channel| Instance Name AHL_NIPROVIDER_2| Component Version adh2.6.9.L1.linux.tis.rrg 64-bit| loggerMsgEnd|
2017-08-16 10:50:10.686 TRACE [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: LoginCallbackClient| Severity: Trace| Text: RDMLogin stream was open with refresh message| LoginRefresh: | streamId: 1| name: ****| nameType: 1| State: Open/Ok/None - text: "Login accepted by host rmds."| isSolicited: true| applicationId: | applicationName: ADS| position: 10.200.31.216| providePermissionProfile: 0| providePermissionExpressions: 1| singleOpen: 1| allowSuspectData: 1| supportBatchRequests: 1| supportBatchReissues: 1| supportBatchCloses: 1| supportOMMPost: 1| supportOptimizedPauseResume: 0| supportStandby: 0| supportViewRequests: 0| supportEnhancedSymbolList: 1| State: State: Open/Ok/None - text: "Login accepted by host rmds."| loggerMsgEnd|
2017-08-16 10:50:10.688 TRACE [ProviderDispatcher] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: LoginCallbackClient| Severity: Trace| Text: RDMLogin stream was open with refresh message| LoginRefresh: | streamId: 1| name: ****| nameType: 1| State: Open/Ok/None - text: "Login accepted by host rmds."| isSolicited: true| applicationId: | supportProviderDictionaryDownload: 1| State: State: Open/Ok/None - text: "Login accepted by host rmds."| loggerMsgEnd|
2017-08-16 10:50:10,689 INFO [ProviderDispatcher] PublishingWorkerFactory - LoginClient. Msg: RefreshMsg
streamId="1"
domain="Login Domain"
solicited
RefreshComplete
state="Open / Ok / None / 'Login accepted by host rmds.'"
itemGroup="00 00"
name="****"
nameType="1"
Attrib dataType="ElementList"
ElementList
ElementEntry name="ApplicationId" dataType="Ascii" value="(blank data)"
ElementEntry name="SupportProviderDictionaryDownload" dataType="UInt" value="1"
ElementListEnd
AttribEnd
RefreshMsgEnd
, Event: com.thomsonreuters.ema.access.OmmEventImpl@1583c06b
2017-08-16 10:50:10.689 TRACE [ProviderDispatcher] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: ChannelCallbackClient| Severity: Trace| Text: Received ChannelReady event on channel Channel| Instance Name AHL_NIPROVIDER_2| loggerMsgEnd|
2017-08-16 10:50:10.689 TRACE [ProviderDispatcher] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: AHL_NIPROVIDER_2| Severity: Trace| Text: Reload of user submitted source directories.| loggerMsgEnd|
2017-08-16 10:50:10.689 TRACE [ProviderDispatcher] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: AHL_NIPROVIDER_2| Severity: Trace| Text: User submitted source directoies were sent out on the wire after reconnect.| loggerMsgEnd|
2017-08-16 10:50:10,690 INFO [ProviderDispatcher] PublishingWorkerFactory - LoginClient. Msg: StatusMsg
streamId="1"
domain="Login Domain"
state="Open / Ok / None / 'channel up'"
name="****"
nameType="1"
StatusMsgEnd
, Event: com.thomsonreuters.ema.access.OmmEventImpl@1583c06b
2017-08-16 10:50:24,372 INFO [publisher] ElektronClient - Published 23 messages so far.
2017-08-16 10:50:24.374 WARN [ProviderDispatcher] [EMA] access.OmmNiProviderImpl - loggerMsg| ClientName: ChannelCallbackClient| Severity: Warning| Text: Received ChannelDownReconnecting event on channel Channel| RsslReactor @41867bf1| RsslChannel @69297e3| Error Id 0| Internal sysError 0| Error Location null| Error text SocketChannel.read returned -1 (end-of-stream)| loggerMsgEnd|
2017-08-16 10:50:24.375 TRACE [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: DirectoryCallbackClient| Severity: Trace| Text: Received RDMDirectory update message| loggerMsgEnd|
2017-08-16 10:50:24.375 TRACE [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: ChannelCallbackClient| Severity: Trace| Text: Received ChannelReady event on channel Channel| Instance Name AHL_CONSUMER_CUSTOM_1| loggerMsgEnd|
2017-08-16 10:50:24.375 TRACE [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: DirectoryCallbackClient| Severity: Trace| Text: Received RDMDirectory update message| loggerMsgEnd|
2017-08-16 10:50:24.375 TRACE [ElektronConsumerDispatcher] [EMA] access.OmmConsumerImpl - loggerMsg| ClientName: DirectoryCallbackClient| Severity: Trace| Text: Received Update action for RDMService| Service name EXT2| Service id 8027| loggerMsgEnd|
2017-08-16 10:50:38,732 INFO [ProviderDispatcher] PublishingWorkerFactory - Connection recovery!!
2017-08-16 10:50:38,732 INFO [publisher] ElektronClient - Published 24 messages so far.
Given the OmmNiProvider has logged in again and send the Directory messages, should I expect the provider to be fully functional after an outage like this? At the moment I will have to uninitialise the original Provider and instantiate a new one to work around this issue.
Have I missed some configuration somewhere?
What is the contribution policy and specifically copyright policy? Is copyright centralized, are contributors required to sign over their copyrights? Should source files be modified to add contributor copyright?
The fooAsString() methods seem to be implemented in an improvable way.
They (temporarily) change the internal state of the underlying upa object:
_originalFormat = ommDtImpl._rsslDate.format();
ommDtImpl._rsslDate.format(_format);
_dtString = ommDtImpl._rsslDate.toString();
ommDtImpl._rsslDate.format(_originalFormat);
Potentially, this could lead to problems regarding multi-threading as these four lines are not atomic.
A dedicated formatToString(int format) method in upa.codec.Date/Time could be more safe.
When a Consumer application receives a STATUS message. The onAllMsg and onUpdateMsg callback methods won't be called anymore when EMA Java application receives further updates.
Source code files for problem reproduction:
EMAJ_callback_not_called.zip
Additional log to investigate this issue is below:
emaj_callbacks_are_not_called_log.txt
This behavior hasn't occurred in other APIs such as RFA Java, or EMA C++.
Hi Dev Team,
As per TREP team suggestion from https://community.developers.thomsonreuters.com/questions/17574/view.html. Posting my query directly to Dev Team. Please help.
We recently migrated our code using TREP EMA API using jdk 1.8. After that we faced 2 times below ConcurrentModificationException error in PROD env and hence our process terminated. Then we had to start our service again. Kindly help to fix this issue ASAP.
We made the connection to TREP using DACSID provided by MDS team and reference code EMA Consumer - Requesting and displaying MarketPrice data shared by Umer Nalla/Team.
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.remove(HashMap.java:1451)
at com.thomsonreuters.ema.access.OmmBaseImpl.rsslReactorDispatchLoop(OmmBaseImpl.java:985)
at com.thomsonreuters.ema.access.OmmBaseImpl.uninitialize(OmmBaseImpl.java:258)
at com.thomsonreuters.ema.access.OmmConsumerImpl.uninitialize(OmmConsumerImpl.java:46)
at com.d3k.reuters.DealLocator.startDealService(DealLocator.java:193)
at com.d3k.reuters.DealLocator.(DealLocator.java:75)
at com.d3k.reuters.ReutersStartup.startService(ReutersStartup.java:66)
at com.d3k.reuters.ReutersStartup.main(ReutersStartup.java:81)
Please let me know for any details required.
Thanks. Raj
Hello,
Let me explain my situation: I have a Consumer application that connects to several ads servers depending on the source we want to reach (for example ads-idn for IDN_SELECTFEED). The problem in EMA is we can declare several Consumer in the EmaConfig.xml but in our Consumer application we can use only one Consumer at a time: createOmmConsumerConfig ().consumerName (name).
Am I right or is there any solution I don't know?
So, as I need to create in my application one Consumer for each ads/source, I need for access to the OmmConsumerConfig. Nevertheless, this interface doesn't expose any methods to retrieve all the consumers declared in the EmaConfig.xml (ConsumerGroup) and the implementation class OmmConsumerConfigImpl is not accessible (as the XmlConfigReader which would be so useful...).
Is there any way to change it? By adding methods in the interface OmmConsumerConfig or by declaring more classes as public? Or is there another way to access the information inside my OmmConsumerConfig?
Thanks.
Hi,
I've found a bug in the RealImpl where a Double.NaN value is checked using the == operator. A NaN is never equal to another NaN and so Double.isNaN() method should be used.
The two places below should be fixed, it prevents encoding via the double/float value methods and is a bug as it's an impossible to reach if block:
There may be other places within ETA and EMA where this is an issue, I guess it's worth have a quick grep over the codebase.
Thanks
Tom
The OmmNiProviderConfigImpl returns null for the two addAdminMsg methods. This conflicts with the documentation and all the other methods where you expect the method to return a reference to this object.
Incorrect implementation:
https://github.com/thomsonreuters/Elektron-SDK/blob/master/Java/Ema/Src/main/java/impl/com/thomsonreuters/ema/access/OmmNiProviderConfigImpl.java#L125
I just tried a clean build as follows:
git clone https://github.com/thomsonreuters/Elektron-SDK Elektron-SDK
cmake -HElektron-SDK -BElektron-SDK_build -G "Visual Studio 15 2017 Win64"
msbuild Elektron-SDK_build\esdk.sln /p:Configuration=Debug_MDd;Platform=x64
However, I get error RC1015 due to missing afxres.h files when the build eventually reaches the following files:
A work-around it to replace afxres.h with windows.h.
Hi, guys. According to this doc
The Elektron Object API: Under development now, this API provides developers with easy-to-use, open-source business objects using
the Thomson Reuters Domain Models (e.g. order books, yield curves, volatility surfaces, and other financial markets models.) This API is
expected to be in beta at the end of 2015 and will be enhanced by customers and Thomson Reuters based on customer feedback.
So is there any info when this api will be available here?
This issue is from this question.
I can replicate the issue by modifying the example (Consumer100) and using breakpoint to freeze and run threads.
The modified code is:
`
void AppClient::onRefreshMsg( const RefreshMsg& refreshMsg, const OmmConsumerEvent& e)
{
cout << refreshMsg << endl; // defaults to refreshMsg.toString()
consumer.reissue(ReqMsg().serviceName("API_ELEKTRON_EPD_RSSL").name("IBM.N"), e.getHandle());
}
int main( int argc, char* argv[] )
{
try {
consumer.registerClient( ReqMsg().serviceName( "API_ELEKTRON_EPD_RSSL" ).name( "IBM.N" ), client );
consumer.registerClient(ReqMsg().serviceName("API_ELEKTRON_EPD_RSSL").name("CNY="), client);
sleep( 60000000 ); // API calls onRefreshMsg(), onUpdateMsg(), or onStatusMsg()
} catch ( const OmmException& excp ) {
cout << excp << endl;
}
return 0;
}
`
When the application receives refreshMsg, it will call reissue. The deadlock happens when the main thread is calling OmmConsumer.registerClient() while EMA thread (API Dispatch Mode) is in onRefreshMsg callback.
Hi,
We're using Elektron-SDK 1.1.1 and had a channel down event and the ChannelGroup contained a box that happened to be down. We actually had a few down. At startup the dead box is skipped and a good box is found.
However, after that box had an issue, it moves through the ChannelGroup and found another dead box at which point the Reactor gets a failure and the library throws exceptions.
Here are the logs - some details (e.g. name) have been removed.
// by this point we were having a resync - was on channel 3 or 4
StatusMsg
streamId="76433"
domain="MarketPrice Domain"
state="Open / Suspect / None / 'Service not available'"
name="----"
serviceName="ELEKTRON_AD"
StatusMsgEnd
// channel goes down or is dead
loggerMsg
ClientName: ChannelCallbackClient
Severity: Warning
Text: Received ChannelDownReconnecting event on channel Channel_5
RsslReactor Channel is null
Error Id 0
Internal sysError 0
Error Location Reactor.processWorkerEvent
Error text Reconnection failed: java.nio.channels.UnresolvedAddressException
loggerMsgEnd
// [assumption] caused by exception above
loggerMsg
ClientName: ChannelCallbackClient
Severity: Error
Text: Received ChannelDown event on channel Channel_5
Instance Name Consumer_1
RsslReactor Channel is null
Error Id -1
Internal sysError 0
Error Location WlItemHandler.dispatch
Error text ReactorCallbackReturnCodes.FAILURE was returned from defaultMsgCallback(). This caused the Reactor to shutdown.
loggerMsgEnd
// Channel goes down
StatusMsg
streamId="1"
domain="Login Domain"
state="Closed / Suspect / None / 'channel closed'"
name="-----" // contained the username
nameType="1"
StatusMsgEnd
loggerMsg
ClientName: SingleItem
Severity: Error
Text: Internal error: ReactorChannel.submit() failed in SingleItem.submit(CloseMsg)RsslChannel 0
Error Id -1
Internal sysError 0
Error Location ReactorChannel.submit
Error Text Reactor is shutdown, submit aborted.
loggerMsgEnd
This is followed by unregister/register messages throwing the following exception:
com.thomsonreuters.ema.access.OmmInvalidUsageExceptionImpl: Failed to close item request. Reason: ReactorReturnCodes.FAILURE. Error text: Reactor is shutdown, submit aborted.
at com.thomsonreuters.ema.access.OmmBaseImpl.ommIUExcept(OmmBaseImpl.java:1172)
at com.thomsonreuters.ema.access.OmmConsumerImpl.handleInvalidUsage(OmmConsumerImpl.java:419)
at com.thomsonreuters.ema.access.SingleItem.rsslSubmit(ItemCallbackClient.java:3018)
at com.thomsonreuters.ema.access.SingleItem.close(ItemCallbackClient.java:2845)
at com.thomsonreuters.ema.access.ItemCallbackClient.unregister(ItemCallbackClient.java:2247)
at com.thomsonreuters.ema.access.OmmBaseImpl.unregister(OmmBaseImpl.java:428)
at com.thomsonreuters.ema.access.OmmConsumerImpl.unregister(OmmConsumerImpl.java:150)
....
My suspicion is that the 'java.nio.channels.UnresolvedAddressException' other than at startup has caused a problem.
Sam
I'm using EMA Java 1.1.1 and testing the ChannelSet failover behaviour on a NIP.
If the application does failover to the second ADH, the application encounters the below exception when the API sends Dictionary Request message to the second ADH.
(I raised this on the forums as well https://community.developers.thomsonreuters.com/questions/23010/ema-channelset-failover.html)
Please exception messages below:
Jan 11, 2018 6:13:06 PM com.thomsonreuters.ema.access.ChannelCallbackClient reactorChannelEventCallback
WARNING: loggerMsg
ClientName: ChannelCallbackClient
Severity: Warning
Text: Received ChannelDownReconnecting event on channel Channel_1
RsslReactor Channel is null
Error Id 0
Internal sysError 0
Error Location Reactor.processWorkerEvent
Error text Error initializing channel: errorId=-1 text=Connection refused: no further information
loggerMsgEnd
Jan 11, 2018 6:13:07 PM com.thomsonreuters.ema.access.ChannelCallbackClient reactorChannelEventCallback
INFO: loggerMsg
ClientName: ChannelCallbackClient
Severity: Info
Text: Received ChannelUp event on channel Channel_3
Instance Name Provider_1_1
Component Version adh3.2.0.L1.linux.tis.rrg 64-bit
loggerMsgEnd
Exception in thread "main" java.lang.NullPointerException
at com.thomsonreuters.ema.access.NiProviderDictionaryItem.rsslSubmit(DictionaryCallbackClient.java:1920)
at com.thomsonreuters.ema.access.NiProviderDictionaryItem.open(DictionaryCallbackClient.java:1777)
at com.thomsonreuters.ema.access.ItemCallbackClient.registerClient(ItemCallbackClient.java:1990)
at com.thomsonreuters.ema.access.OmmBaseImpl.registerClient(OmmBaseImpl.java:380)
at com.thomsonreuters.ema.access.OmmNiProviderImpl.registerClient(OmmNiProviderImpl.java:280)
at com.thomsonreuters.ema.access.OmmNiProviderImpl.registerClient(OmmNiProviderImpl.java:263)
at com.thomsonreuters.ema.examples.training.niprovider.series300.example350__Dictionary__Streaming.NiProvider.main(NiProvider.java:135)
I have been running some tests with the Elektron SDK v1.1.1 and v1.2.0 and
identified a difference in the behaviour between the two - very likely a
regression.
The problem occurs while using just the ETA library (aka. librssl.lib) and the
following functions:
rsslRealToString()
rsslNumericStringToReal()
The basis of my test is as follows:
My input data for the test was:
RsslReal rssl_real;
rssl_real.isBlank = false;
rssl_real.hint = RSSL_RH_EXPONENT7;
rssl_real.value = std::numeric_limits::max(); // 9223372036854775807
Running this test against Elektron SDK v1.1.1 shows the value to be converted
successfully.
Running this test against Elektron SDK v1.2.0 shows rsslNumericStringToReal()
fails with the error code RSSL_RET_INVALID_DATA.
Could you confirm/explain the difference in behaviour.
When would RSSL_ENCRYPTED become available in EMA Java. Thanks.
Hi,
Can we please get a C# version of these APIs released? Ideally they would target one of the .Net Standard API versions.
The readme in master
points to a JS preview
but there's nothing in it. Plans or update for this version?
When I am requesting a Market Price I am facing the problem that the day's first request doesn't result in a valid refresh message. Instead I am receiving a StatusMsg of this kind:
StatusMsg
streamId="6"
domain="MarketPrice Domain"
state="Open / No Change / None / '*SERVICE_NAME: Requested'"
name="..."
serviceId="..."
serviceName="..."
StatusMsgEnd
If a send a second request message I will receive the expected RefreshMsg.
This is the ReqMsg I am sending:
ReqMsg reqMsg = EmaFactory.createReqMsg();
reqMsg.serviceName("...")
.name("...")
.domainType(DomainTypes.MARKET_PRICE);
To be a little more precise: The problem seems to occur for every request after the system has been idle for a while (at least the client did not send any requests).
Am I missing something in my request configuration or is it an issue on the provider or framework side?
Hi,
Did you think about using a cross-platform build system like cmake or gyp ?
If you think it is valuable, I am willing to help and begin a poc on a branch.
As-is the Elektron SDK open source project is really difficult to setup in an IDE and doesn't allow for easy debugging with full source. The current project layout is a far cry from most other open source projects in this respect.
I've taken the repository (Java parts only) and Mavenized the whole lot. It now opens instantly in any IDE and it builds in any IDE. Have a look at https://github.com/Addicticks/ElektronSDK-Mavenized.
The project now looks like this when opened in an IDE:
Sub-projects with the name "Provider" in them is for closed-source projects where only the binary is provided. External artifacts are taken from Maven Central where ever possible (e.g. SLF4J, Apache Commons, etc).
The project doesn't require the developer to have access to upload to a Maven repo. Therefore any junior Java developer (or senior) should be able to open the project in his favorite IDE and be instantly productive.
Dependencies are defined between sub-projects ... as you would expect.
Mavenizing the project build is also a pre-cursor (although not strictly required) for making the Elektron SDK released artifacts available in some centralized Maven repository, e.g. Maven Central or Thomson Reuters' own Maven repo.
Please, can you consider this. I believe the setup cost is currently just too high for anyone to really bother with the open source Elektron SDK ... and that's just too bad, because it deserves more.
(Apologies for not creating a pull request. This is fundamental change in build structure, directory structure, etc, and therefore doesn't lend itself towards a pull request).
Regards
Lars
When will you support programmatic configuration? The current configuration utilities are very basic and I can't configure what I want (different nodes per environment with automatic failover).
There are two main problems that could do with some attention:
Hi,
The repository is very big: it took me 30 minutes to clone it.
Most of its contents are documentations. Would it be move them in a separate repository?
What do you think ?
This occurs if you do a debug build of Eta (which enables RSSL_ASSERT) and construct such a large Map that a memory reallocation is required.
Actual output (requires debug build of Eta):
$ ./BigMap
Fatal Error: _levelInfo->_encodingState == RSSL_EIS_ENTRIES, Unexpected encoding attempted (file .../Eta/Impl/Codec/mapEncoder.c, line 425)
Aborted (core dumped)
Expected output (which you get with an optimized Eta build): should execute silently and without error
$ ./BigMap
$
Testcase:
#include "Ema.h"
using namespace thomsonreuters::ema::access;
int main(void) {
Map map;
for (int i = 0; i < 10000; ++i) {
FieldList fieldList;
fieldList.complete();
map.addKeyInt(i, MapEntry::DeleteEnum, fieldList);
}
map.complete();
return 0;
}
I am trying to get Dynamic Views in EMA Java, and not able to limit the fields:
OmmArray fields = EmaFactory.createOmmArray();
fields.add(EmaFactory.createOmmArrayEntry().intValue(22));
OmmArray symbols = EmaFactory.createOmmArray();
symbols.add(EmaFactory.createOmmArrayEntry().ascii("TRI.N"));
ElementList batch = EmaFactory.createElementList();
batch.add(EmaFactory.createElementEntry().array(EmaRdm.ENAME_BATCH_ITEM_LIST, symbols));
batch.add(EmaFactory.createElementEntry().uintValue(EmaRdm.ENAME_VIEW_TYPE, 1));
batch.add(EmaFactory.createElementEntry().array(EmaRdm.ENAME_VIEW_DATA, fields));
consumer.registerClient(EmaFactory.createReqMsg().serviceName("ELEKTRON_AD").payload(batch), appClient);
The Ema library decoder implementation uses placement new to write objects of one class on top of objects of another class (see Decoder::create and StaticDecoder::create). I tracked down crashes in MapDecoder::~MapDecoder() on program exit when compiling Ema using GCC with link-time optimization to the use of placement new in StaticDecoder::create. What happens is that in
MapDecoder::~MapDecoder()
{
if ( _atExit )
{
if ( _elementListSetDef )
delete _elementListSetDef;
if ( _fieldListSetDef )
delete _fieldListSetDef;
}
else
{
if ( _elementListSetDef )
g_pool._elementListSetDefPool.returnItem( _elementListSetDef );
if ( _fieldListSetDef )
g_pool._fieldListSetDefPool.returnItem( _fieldListSetDef );
}
StaticDecoder::morph( &_key, DataType::NoDataEnum );
StaticDecoder::morph( &_load, DataType::NoDataEnum );
StaticDecoder::morph( &_summary, DataType::NoDataEnum );
}
the optimizers remove the three calls to StaticDecoder::morph. If you know how this stuff works then you will understand how removing these can cause a crash immediately afterwards (in my case, the NoDataImpl destructor was run on _summary even though it was created earlier, using placement new, as a FieldList; normally the morph call would have turned it back into a NoDataImpl, avoiding this destructor mismatch).
Were the optimizers wrong to do this? What Ema is doing is essentially the same as the second example in https://miyuki.github.io/2016/10/21/std-launder.html, see also the test cases in https://gcc.gnu.org/ml/libstdc++/2016-10/msg00194.html. It seems likely that the optimizers are indeed allowed to remove the calls, i.e. the Ema code is wrong. To make it correct, std::launder is needed. The following patch fixed the crash for me:
--- a/Ema/Src/Access/Impl/StaticDecoder.cpp
+++ b/Ema/Src/Access/Impl/StaticDecoder.cpp
@@ -143,7 +143,7 @@ void StaticDecoder::morph( Data* data, DataType::DataTypeEnum dType )
}
}
-void StaticDecoder::create( Data* data, DataType::DataTypeEnum dType )
+void StaticDecoder::create( Data*& data, DataType::DataTypeEnum dType )
{
switch ( dType )
{
@@ -250,4 +260,6 @@ void StaticDecoder::create( Data* data, DataType::DataTypeEnum dType )
new (data) OmmError();
break;
}
+
+ data = std::launder(data);
}
diff --git a/Ema/Src/Access/Impl/StaticDecoder.h b/Ema/Src/Access/Impl/StaticDecoder.h
index fa7e5d710c..16f6825314 100755
--- a/elektron-latest/Ema/Src/Access/Impl/StaticDecoder.h
+++ b/elektron-latest/Ema/Src/Access/Impl/StaticDecoder.h
@@ -66,7 +66,7 @@ public :
// helper utilities
static void morph( Data* , DataType::DataTypeEnum );
- static void create( Data* , DataType::DataTypeEnum );
+ static void create( Data*& , DataType::DataTypeEnum );
};
}
I didn't audit the Ema code for other places that might need std::launder, however Decoder::create seems likely to be one such place.
Unfortunately, I don't see how you can apply this patch, as std::launder is not widely available (it's from C++17). With GCC you can emulate this C++17 feature using this code:
namespace std
{
template <typename T>
T *
launder (T *p)
{
return __builtin_launder (p);
}
}
but that's not going to work for other compilers, or even for older versions of GCC. I'm reporting this anyway as documentation for others hitting this issue, and as it seems like something you should be aware of.
Microsoft's VCPKG on GitHub is an increasingly popular mechanism for making third-party C++ libraries available to application developers, similarly to nuget for C#. Could the C++ components of Elektron-SDK be added to the public repository of libraries available through vcpkg? Are there technical or commercial impediments?
rfa.omm.OMMDateTime provided toCalendar and toDate methods.
ema.access.OmmDateTime lacks those.
The implementing rfa.internal.rwf.RwfDateTime class constructs an internal GregorianCalendar object with timezone = GMT and locale = UK.
If an application would like to convert the ema OmmDateTime to a Calendar, are these timezone/locale settings still applicable?
Are dates/times always relative to GMT in ema?
I have updated from SDK 1.0.5 to 1.0.7 and now I am facing some problems when I try to close the consumer.
I have written a small snippet that is pretty similar to the examples given in Dev Guide:
public class TrepClient {
private static final OmmConsumerClient CONSUMER = new OmmConsumerClient() {
@Override
public void onUpdateMsg(UpdateMsg paramUpdateMsg, OmmConsumerEvent paramOmmConsumerEvent) {
// TODO Auto-generated method stub
}
@Override
public void onStatusMsg(StatusMsg paramStatusMsg, OmmConsumerEvent paramOmmConsumerEvent) {
// TODO Auto-generated method stub
}
@Override
public void onRefreshMsg(RefreshMsg paramRefreshMsg, OmmConsumerEvent paramOmmConsumerEvent) {
// TODO Auto-generated method stub
}
@Override
public void onGenericMsg(GenericMsg paramGenericMsg, OmmConsumerEvent paramOmmConsumerEvent) {
// TODO Auto-generated method stub
}
@Override
public void onAllMsg(Msg paramMsg, OmmConsumerEvent paramOmmConsumerEvent) {
System.out.println(paramMsg);
}
@Override
public void onAckMsg(AckMsg paramAckMsg, OmmConsumerEvent paramOmmConsumerEvent) {
// TODO Auto-generated method stub
}
};
public static void main(String[] args) throws InterruptedException {
OmmConsumer consumer = null;
try {
OmmConsumerConfig config = EmaFactory.createOmmConsumerConfig() //
.host("...") //
.applicationId("...") //
.username("...");
consumer = EmaFactory.createOmmConsumer(config);
ReqMsg reqMsg = EmaFactory.createReqMsg();
reqMsg.serviceName("...") //
.name("...") //
.domainType(DomainTypes.MARKET_PRICE);
consumer.registerClient(reqMsg, CONSUMER);
Thread.sleep(1000);
} finally {
if (consumer != null) {
consumer.uninitialize();
}
}
}
}
And this is the output:
Okt 18, 2016 2:51:38 PM com.thomsonreuters.ema.access.ConfigErrorTracker log
INFORMATION: loggerMsg
ClientName: EmaConfig
Severity: Info
Text: reading configuration file [EmaConfig.xml] from [...]
loggerMsgEnd
Okt 18, 2016 2:51:38 PM com.thomsonreuters.ema.access.ConfigErrorTracker log
SCHWERWIEGEND: loggerMsg
ClientName: EmaConfig
Severity: Error
Text: Cannot locate configuration source EmaConfig.xml
loggerMsgEnd
Okt 18, 2016 2:51:38 PM com.thomsonreuters.ema.access.ChannelCallbackClient reactorChannelEventCallback
INFORMATION: loggerMsg
ClientName: ChannelCallbackClient
Severity: Info
Text: Received ChannelUp event on channel Channel
Instance Name EmaConsumer_1
Component Version ads2.6.10.L1.linux.tis.rrg 64-bit
loggerMsgEnd
RefreshMsg
streamId="5"
domain="MarketPrice Domain"
solicited
RefreshComplete
state="Open / Ok / None / '*None'"
itemGroup="00 03"
name="..."
nameType="1"
serviceId="..."
serviceName="..."
Payload dataType="FieldList"
FieldList FieldListNum="0" DictionaryId="1"
FieldEntry fid="2" name="RDNDISPLAY" dataType="UInt" value="(blank data)"
FieldListEnd
PayloadEnd
RefreshMsgEnd
Okt 18, 2016 2:51:39 PM com.thomsonreuters.ema.access.ChannelDictionary processCallback
WARNUNG: loggerMsg
ClientName: ChannelDictionary
Severity: Warning
Text: RDMDictionary stream was closed with status message
streamId 3
Reason State: Closed/Suspect/None - text: "channel down."
loggerMsgEnd
StatusMsg
streamId="5"
domain="MarketPrice Domain"
state="Closed / Suspect / None / 'channel down.'"
name="..."
serviceId="..."
serviceName="..."
StatusMsgEnd
Okt 18, 2016 2:51:39 PM com.thomsonreuters.ema.access.ChannelDictionary processCallback
WARNUNG: loggerMsg
ClientName: ChannelDictionary
Severity: Warning
Text: RDMDictionary stream was closed with status message
streamId 4
Reason State: Closed/Suspect/None - text: "channel down."
loggerMsgEnd
Exception in thread "pool-2-thread-1" java.lang.NullPointerException
at com.thomsonreuters.upa.valueadd.reactor.Reactor.sendChannelEventCallback(Unknown Source)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.sendAndHandleChannelEventCallback(Unknown Source)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.dispatchChannel(Unknown Source)
at com.thomsonreuters.upa.valueadd.reactor.ReactorChannel.dispatch(Unknown Source)
at com.thomsonreuters.ema.access.OmmBaseImpl.rsslReactorDispatchLoop(Unknown Source)
at com.thomsonreuters.ema.access.OmmBaseImpl.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
By the way, sometimes the application exits with another stacktrace:
Exception in thread "pool-2-thread-1" java.nio.channels.ClosedSelectorException
at sun.nio.ch.SelectorImpl.selectedKeys(Unknown Source)
at com.thomsonreuters.ema.access.OmmBaseImpl.rsslReactorDispatchLoop(Unknown Source)
at com.thomsonreuters.ema.access.OmmBaseImpl.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Hello,
I am facing an issue while making snapshot with EMA through an ads server. The issue is linked to the number of products I send to the ads (around 75,000). When reaching around 30,000 the ReactorChannel becomes DOWN and so my application doesn't send anything anymore to the ads. The problem is all the remaining products are stored in the unsent queue and the number of WorkerEvent incredibly increases (more than 10 millions instances whereas I have only 75,000 products I am requesting with two fields for each).
Do you have any idea why this is happening? Is there any limit somewhere regarding the number of requests to send through a ReactorChannel?
Thanks in advance.
Hi,
We're seeing some strange behaviour. We have an NiProvider application and a load test that runs our application at 50k msg/sec. The load test publishes the messages and then records them coming back in through it's own subscriptions.
We're tuning some of the EmaConfig.xml values. Our baseline value for LoginRequestTimeOut is 1000, if I set this higher to 30000 then the test can only produce 25k msg/sec, if I set it to 0 (disabling the timeout), the same thing happens. That is the only parameter I change between runs and it's the same every time. I don't understand how this parameter should affect ongoing application performance. I even added a 10 second wait after sending initial Image data for all the subjects before starting the higher message rate load test. It looks like the parameter is used here:
How does the LoginRequestTimeOut parameter go on and affect overall application throughput? Is there something else that needs to be included in the documentation to help us set a good value. From the documentation it looks like it should be fine to allow a bit longer for the initial login request to complete. From our test we're seeing that this is detrimental to application performance. Can you provide any guidance or explanation?
Hello,
Currently in the Preview branch / Js folder there is nothing. Are going to release something soon?
The javadocs of some OmmDateTime interface methods hides those from the DateTime:
ema.access.OmmDateTimeImpl forwards these methods directly to the upa.codec.DateTime(Impl).
Is my understanding correct, that the magic numbers of the upa "indicates blank" have to be checked when using the Omm methods?
Hi there,
When I sign into Customer Zone and navigate to the software download page I cannot see the Elektron API package listed. Do I need to be permissioned to download this?
Regards,
Richard.
I worked with some of these APIs before and am thrilled to see this available on GitHub :-)
Having tried to build and ETA and run some examples on my Ubuntu, I think the instructions (README.md) didn't emphasize enough about the supported platforms. Looking at the makefile, this only builds on RedHat, SUSE, Oracle Linux, it seems? (also, shouldn't makefile all
command be instead make all
?)
Hi, All
We found an issue, when there are massive downstream disconnect/reconnect event. UPA provider will stop working. We are using EMA v1.0.8
Below is the call stack which consumes the most CPU.
"pool-4-thread-1" #30 prio=5 os_prio=0 tid=0x00007f88e8fa8800 nid=0x489f runnable [0x00007f8879376000]
java.lang.Thread.State: RUNNABLE
at com.thomsonreuters.upa.transport.RsslSocketChannel$BigBuffersPool.poll(RsslSocketChannel.java:63)
at com.thomsonreuters.upa.transport.RsslSocketChannel.acquirePair(RsslSocketChannel.java:3943)
at com.thomsonreuters.upa.transport.ReadBufferStateMachine.acquireFragmentBuffer(ReadBufferStateMachine.java:1010)
at com.thomsonreuters.upa.transport.ReadBufferStateMachine.processExtendedMessage(ReadBufferStateMachine.java:855)
at com.thomsonreuters.upa.transport.ReadBufferStateMachine.updateStateCurrentLenKnown(ReadBufferStateMachine.java:496)
at com.thomsonreuters.upa.transport.ReadBufferStateMachine.advanceOnSocketChannelRead(ReadBufferStateMachine.java:251)
at com.thomsonreuters.upa.transport.RsslSocketChannel.performReadIO(RsslSocketChannel.java:1496)
at com.thomsonreuters.upa.transport.RsslSocketChannel.read(RsslSocketChannel.java:1230)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.performChannelRead(Reactor.java:1712)
at com.thomsonreuters.upa.valueadd.reactor.Reactor.dispatchAll(Reactor.java:3217)
at com.thomsonreuters.ema.access.OmmServerBaseImpl.rsslReactorDispatchLoop(Unknown Source)
at com.thomsonreuters.ema.access.OmmServerBaseImpl.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
The Gradle build script fails when executed from my IDE. The reason is the cloneBinaryPack
which makes assumptions about working dir. I'm no Gradle expert but I've understood as much as that such an assumption must never exist in a Gradle build script .. even if it seems to work well for some types of executions.
Here's the offending code:
doLast {
// check if we are in GSG package
File gsgDir = new File('../Elektron-SDK-BinaryPack') // bad, makes assumptions on working dir
if (!gsgDir.exists()) {
// if not in GSG package try to clone BinaryPack
File dir = new File('../Elektron-SDK-BinaryPack') // bad, makes assumptions on working dir
if (!dir.exists()) {
and here's the fix:
doLast {
// check if we are in GSG package
File gsgDir = file('../Elektron-SDK-BinaryPack')
if (!gsgDir.exists()) {
// if not in GSG package try to clone BinaryPack
File dir = file('../Elektron-SDK-BinaryPack')
if (!dir.exists()) {
(i.e. use the file()
method instead of new File()
)
Hi, All
I'm using EMAJ 1.0.8. Tried to submit a Map with summary only but failed to do so, got various exceptions when payload(the map without entry.). Please advise. Thank you. Below is sample code.
FieldList mapSummaryData = EmaFactory.createFieldList();
mapSummaryData.add(...); //Meaningful summary data.
Map map = EmaFactory.createMap();
map.summaryData(mapSummaryData);
//Meaningless map entry, but I had to add one here, otherwise .payload will throw exception.
FieldList entryData = EmaFactory.createFieldList();
map.add(EmaFactory.createMapEntry().keyAscii("", MapEntry.MapAction.ADD, EmaFactory.createFieldList()));
provider.submit(
EmaFactory.createRefreshMsg().domainType(EmaRdm.MMT_SYMBOL_LIST)
.serviceName("DIRECT_FEED").solicited(true).clearCache(true).state(OmmState.StreamState.OPEN,
OmmState.DataState.OK, OmmState.StatusCode.NONE, "Refresh Completed")
.payload(mapSummaryData)
.complete(true),
itemHandle);
With EMAJ 1.0.8, I'm trying to get channel event (up/down) with below sample code:
consumer.registerClient(EmaFactory.createReqMsg().domainType(DomainTypes.SOURCE), channalStatusClient);
I use ChannelSet in "EmaConfig"
It's ok if "Channel_1" is good. But in case "Channel_1" failure & "Channel_2" is good, I got NPE when registerClient.
Call stack:
Exception in thread "main" java.lang.NullPointerException
at com.thomsonreuters.ema.access.DirectoryItem.submit(DirectoryCallbackClient.java:952)
at com.thomsonreuters.ema.access.DirectoryItem.open(DirectoryCallbackClient.java:810)
at com.thomsonreuters.ema.access.ItemCallbackClient.registerClient(ItemCallbackClient.java:1745)
at com.thomsonreuters.ema.access.OmmBaseImpl.registerClient(OmmBaseImpl.java:350)
at com.thomsonreuters.ema.access.OmmConsumerImpl.registerClient(OmmConsumerImpl.java:73)
Also, please kindly advise is this the right way to hook the channel event. Thank you.
I've been able to successfully use the ChannelSet configuration with the OmmConsumerImpl. The consumer has a ChannelSet with two different ADS servers. When one server goes down the reactor picks up another channel from the ChannelSet and starts using that. The subscription messages keep flowing after the consumer picks up the new channel.
I can see that the OmmNIProviderImpl can be configured to use a ChannelSet. However, it seems to just stick to the first channel in the ChannelSet. When the underlying channels disconnect during failures the OmmNIProviderImpl doesn't pick up the other channels. It keeps using the first channel in the ChannelSet. Also if it times out connecting to the first channel it doesn't try the next channel after the login timeout.
Here is a log output of an NI Provider configured with two ADH servers in a channel set. It initially connects to the first channel and successfully sends data. When I stop that ADH server you can see that second channel gets activated and connects out to the seconds ADH. However the OmmNIProvider doesn't seem to use the new channel. It keeps accepting messages but never sends them to the ADH as it maintains a link to the disconnected channel.
2017-06-16 15:55:25,711 WARN [pool-12-thread-1] OmmNiProviderImpl - loggerMsg
ClientName: ChannelCallbackClient
Severity: Warning
Text: Received ChannelDownReconnecting event on channel ADH_CH2_DEV
RsslReactor @46d03ade
RsslChannel @4817cc74
Error Id 0
Internal sysError 0
Error Location null
Error text SocketChannel.read returned -1 (end-of-stream)
loggerMsgEnd
2017-06-16 15:55:26,203 DEBUG [OmmPublishWorker_0] OmmPublisherWorker - Publishing UPDATE for STAT.TEST
2017-06-16 15:55:26,203 ERROR [OmmPublishWorker_0] OmmNiProviderImpl - loggerMsg
ClientName: NIPROVIDER_DEV_2
Severity: Error
Text: Internal error: rsslChannel.submit() failed in OmmNiProviderImpl.submit(UpdateMsg)RsslChannel 0
Error Id -1
Internal sysError 0
Error Location Reactor.submitChannel
Error Text ReactorChannel is closed, aborting.
loggerMsgEnd
2017-06-16 15:55:26,204 WARN [OmmPublishWorker_0] ProviderErrorClient - onInvalidUsage: Failed to submit UpdateMsg. Reason: ReactorReturnCodes.FAILURE. Error text: ReactorChannel is closed, aborting.
2017-06-16 15:55:26,748 INFO [pool-12-thread-1] OmmNiProviderImpl - loggerMsg
ClientName: ChannelCallbackClient
Severity: Info
Text: Received ChannelUp event on channel ADH_CH1_DEV
Instance Name NIPROVIDER_DEV_2
Component Version adh2.6.9.L1.linux.tis.rrg 64-bit
loggerMsgEnd
My provider is configured with these properties:
...
<Channel>
<Name value="ADH_CH1_DEV"/>
<GuaranteedOutputBuffers value="5000"/>
<Host value="<adh1>"/>
<Port value="14003"/>
</Channel>
<Channel>
<Name value="ADH_CH2_DEV"/>
<GuaranteedOutputBuffers value="5000"/>
<Host value="<adh2>"/>
<Port value="14003"/>
</Channel>
...
<NiProvider>
<Name value="NIPROVIDER_DEV"/>
<ChannelSet value="ADH_CH2_DEV, ADH_CH1_DEV"/>
<ReconnectAttemptLimit value="2"/>
</NiProvider>
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.