open-coap / java-coap Goto Github PK

Implementing coap(s)+tcp in Leshan, I faced some issues that could be solve with API additions :
(I found workaround for each of them, so there is no urgency)

1. A way to iterate over keys of TransportContext

Maybe a Collection<Key<?>> getKeys() methods ? or make TransportContext implements Iterable ?

Here an example of usage.

2. I need to add all keys from a TransportContext to an other existing TransportContext.

Proposed API :

• Adding to TransportContext a method with(TransportContext contextToMerge)
• Adding a CoapRequest.Builder a method addContext(TransportContext contextToMerge)

coapRequest.options().getUriQueryMap() seems to only support Option in the form of a "key=value" pair.
It should also support more simple form like "key" without value.

(See more detailed at : #27 (comment))

The RFC7641 - Observing Resources in the Constrained Application Protocol (CoAP) § 3.2. Notifications says :

In the event that the resource changes in a way that would cause a
normal GET request at that time to return a non-2.xx response (for
example, when the resource is deleted), the server sends a
notification with an appropriate response code (such as 4.04 Not
Found) and removes the client's entry from the list of observers of
the resource. Non-2.xx responses do not include an Observe Option.

I think that currently java-coap doesn't allow this.
Because when I try to send a 4.04 Not Found as notification, it is never sent.

I guess this is because of NotificationValidator, see :

Not directly linked but that exception was silently ignore in my code (I mean I didn't even see a log), I don't know if there is something to improve in java-coap or in my code for this last point.

I face some NPE in Deduplicator like this :

java.util.concurrent.CompletionException: java.lang.NullPointerException
	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:273)
	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:280)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:604)
	at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:614)
	at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1983)
	at com.mbed.coap.server.messaging.DuplicateDetector.apply(DuplicateDetector.java:58)
	at com.mbed.coap.server.messaging.DuplicateDetector.apply(DuplicateDetector.java:31)
	at com.mbed.coap.utils.Filter.lambda$andThen$1(Filter.java:34)
	at com.mbed.coap.utils.Filter.lambda$then$3(Filter.java:53)
	at com.mbed.coap.server.messaging.CoapDispatcher.handleObservation(CoapDispatcher.java:103)
	at com.mbed.coap.server.messaging.CoapDispatcher.handle(CoapDispatcher.java:66)
	at com.mbed.coap.server.CoapServer.handle(CoapServer.java:73)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1595)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
	at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011)
	at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
	at com.mbed.coap.server.DefaultDuplicateDetectorCache.put(DefaultDuplicateDetectorCache.java:76)
	at com.mbed.coap.server.DefaultDuplicateDetectorCache.put(DefaultDuplicateDetectorCache.java:30)
	at com.mbed.coap.server.messaging.DuplicateDetector.putResponse(DuplicateDetector.java:72)
	at com.mbed.coap.server.messaging.DuplicateDetector.lambda$apply$0(DuplicateDetector.java:59)
	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
	... 16 common frames omitted

This happens at CoAP client side in my tests where a NON Confirmable Observe Notification is Received.
Same tests using a CON doesn't raise an NPE.

If it helps I can provide tcpdump/wireshark capture. Let's me know.

This issue will be used to discuss / provide feedback of new API for CoAP Observe.

This was triggered by #27 (comment) :

It would be great if you could have a look at least at usage diff: https://github.com/open-coap/java-coap/tree/observation-handlers#coap-client-complete-usage, and give feedback, especially about naming (those are always the hardest ;))

I have regularly same naming problem with Leshan 😅

@szysas if I want to test that version ☝️. What is the recommended way ?

• is there a nightly build available in a maven repo ?
• OR should I build locally ?

This issue aims to discuss about how to manage API changes in the future (not a short term question):

• should Semantic Versioning) be used ? Or anything else ?
• which tooling can be used ?

This was triggered by #48 (comment):

Not the first time when I update java-coap there is API break between 2 minor version, for now this is not a problem as we are in development phase, but if we release a stable version of Leshan including java-coap dependency, that could be an issue. Did you already consider to maybe move to Semantic Versioning ?

Yes, I am aware of that. When needed, we can switch to be more strict with API changes.

Following #27, I created this issue.

The need :

is there a way to check if a message (request or response) is CON or not ? something like : if (!CON.equals(coapRequest.getType()))

A possible way :

At the moment there is no way to check it if incoming request or response was NON. If needed this info can be easily added to TransportContext

I begin to play with coap-tcp API of java-coap.

I understand this is more or less like CoAP over UDP except I need to :

• use TcpCoapServer.builder(),
• provide TCP capabilities using : .blockSize(...).maxIncomingBlockTransferSize(...).maxMessageSize(...) ,
• provide a CoapTcpTransport.

For the last point (providing a CoapTcpTransport), I only found implementation for client : SocketClientTransport. but nothing for server. Except an out of API SingleConnectionSocketServerTransport class in tests which seems to support only 1 client at a time.

Then I tell myself that maybe there is only server implementation based on netty ? but it seems that NettyCoapTransport implements CoapTransport only.

Did I missed something ? 🤔

As part of the tests 1338 I would like to replace the californium coap stack by java-coap in the LwM2M Leshan client. In the first stage, I was able to register the modified leshan-demo-client in the LwM2M server (using RegistrationManager) but unfortunately I have a problem with response. I must forward some information from CoapResponse to the LwM2M layer. Is there any callback mechanism implemented in RegistrationManager that allows me to get a access to coapresponse Object?

I updated Leshan code to use the new java-coap API.

I think the API is largely better now 👍

Only issue I faced was when I wanted to modify a CoapRequest to add element in TransportContext.

I didn't find better way than :

TransportContext extendedContext = observeRequest.getTransContext() //
                            .with(LwM2mKeys.LESHAN_OBSERVED_PATHS, paths);

CoapRequest modifiedObservedRequest = new CoapRequest(observeRequest.getMethod(),
        observeRequest.getToken(), observeRequest.options(), observeRequest.getPayload(),
        observeRequest.getPeerAddress(), extendedContext);

super.add(modifiedObservedRequest);

A solution to avoid the verbose coapRequest creation could be #68 (comment) and so here this could look like :

TransportContext extendedContext = observeRequest.getTransContext() //
                            .with(LwM2mKeys.LESHAN_OBSERVED_PATHS, paths);

super.add(coapRequest.modify().context(extendedContext).build());

Maybe we could also add a method to modify transportContext in the builder ? something like :

super.add(coapRequest.modify()
                     .context(ctx -> ctx.with(LwM2mKeys.LESHAN_OBSERVED_PATHS, paths))
                     .build());

Note that:

• those new methods can be added without deprecating with*(...) methods a first time. (if you think this is too soon)
• modify() method probably also makes sense for CoapResponse

(If you are curious the whole migration change are available at : eclipse-leshan/leshan@c2f8232)

I have a first working version of coap+tcp in Leshan based on java-coap. (#69 (comment))
This is probably not production ready, especially about how connection is managed but this is good enough to be able to play with it.

So I tried to begin to look at adding coaps+tcp support. In LWM2M generally the first step is using Pre-shared Key. (Far easier to set-up than RPK or x509)

Naively, I was thinking that I will be able to set-up this quickly but I understand there is no Pre-Shared Key support in java-coap ? more exactly java-coap API is based on JSSE API (SSLSocketFactory more precisely) and I understand that PSK is not supported by the default SunJSSE Provider (default implementation of JSSE API) ... 😞

Am I right or did I missed something ?

Currently we can only set ACK_TIMEOUT and MAX_RETRANSMIT but not ACK_RANDOM_FACTOR.

This issue was triggered by : #27 (comment)

I recently discover that LWM2M v1.1.1 recommends to use new CoAP Request-Tag and Echo option defined in RFC9175

The idea is mainly to solve issues described by : draft-ietf-core-attacks-on-coap.

See more details at :

• eclipse-leshan/leshan#1506
• eclipse-leshan/leshan#1507

AFAIK, this is not implemented by java-coap, right ?
Is it something that is planned ?

Observe relation cancelling is defined at rfc7641§3.6. Cancellation.
I will use :

• Passive Cancel wording when a client send an RST message to a notification.
• Active Cancel wording when a client send a request with observe option set to 1.

Reading rfc8323§7.4. Cancellation, I understand that Passive Cancel is not possible with coap+tcp.

When you are using java-coap for coap, you can do Passive Cancel just by removing "observation" from ObservationsStore which is fine.
When you do that with coap+tcp, notifications received are just ignore by the client. Concretely you get a log at client side which looks like :

2023-10-27 17:06:09,455 CoapTcpDispatcher [WARN] Can not process CoAP message [127.0.0.1:56090 205 MID:0 Token:0x349ddba051b8041d ContTp:0 obs:6 pl:'IN2dR8X4']
If client doesn't do an Active Cancel, server will continue to send notifications.

The behavior totally makes sense regarding RFCs but from a user point of view I ask my self if this could make sense to automatically send an Active Cancel when we get unwanted notification with "coap+tcp" ? 🤔

(I will be unavailable next few days. I'll be back on Thursday November 2nd.)

Following #27, I created this issue.

Short Summary :

Being able to store observe relation at CoAP client side in a shared store.
The idea is to not lost observation relation on reboot but also being able to share observe relation in a cluster.

Long story :

A LWM2M server in production often need to persist observe relation or to shared when deployed in a cluster.
Note that from an CoAP observe perspective, a LWM2M server act as a CoAP client.

About persistency :
When you create a LWM2M server based on Leshan and you manage lot of clients in production with lot of Observe Relation. When you reboot your server, this will be an issue to resend all observe requests.

About cluster :
Most of people who use Leshan in a production server want to be able to launch it in a cluster and so observation relation must be shared between all instance of the cluster. Currently in Californium this is achieve with a ObservationStore interface.

This issue aims to discuss about :

• if this makes sense to add this feature to java-coap ?
• how it should be implemented ?

Some challenge we need to have in mind are probably described in leshan wiki.

Some design questions :

• What should be persisted exactly ?
• How should be identified the observe relation ?

Summary :

Leshan is a java implementation of LWM2M and until now it was strictly based on californium.
Recently we did massive changes to abstract transport Layer. (See : eclipse-leshan/leshan#1025)

Now to test this abstraction we are playing with java-coap (See : eclipse-leshan/leshan#1373)

I will use this general issue to ask questions and provides feedback about it.

Some questions :

1. In californium, there is a endpoint concept. If I simplify the concept a lot this is the socket where message are received. In californium a coap server can have several endpoints. If I understand java-coap correctly, 1 server is only associated to 1 socket right ?

2. is there a way to check if a message (request or response) is CON or not ? something like : if (!CON.equals(coapRequest.getType()))

3. is there a way to set a message (request or response) as CON ? something like : coapRequest.setConfirmable(true);

4. is there a way to execute code after the message( mainly needed for response) is sent ? Maybe a callback once the message is sent ?

5. Patch, IPatch and Fetch seems not implemented ? correct ? is it planned to implement it ? (I think we mainly need Ipatch and fetch for lwm2m composite operation)

6. If you are curious you can have a look at how we are using java-coap, See : https://github.com/eclipse/leshan/compare/5b0b66fa675da2ad9224d15e828036d0c06d0f40
   E.g : I created a kind of ResourceService to be able to add some Resource (see RegistrationResource), I don't know if this is a good idea or if this could give you some hint about changing java-coap API ? 🤷‍♂️

(more question is coming soon 😁 )

Currently this is not possible to do :

RouterService.RouteBuilder routerBuilder = new RouterService.RouteBuilder();
routerBuilder //
        .any("/", new RootResource() //
        .any("/myResource", new MyResource() //
        .any("/*", new Default());
router = routerBuilder.build();

If you do that RootResource will never be called.
I think this is because / and /* will create 2 RequestMatcher which will be considered as equal and so Default will override RootResource in handlers hashmap.
Maybe the solution is to add isPrefixed attribute to equals() and hashcode() methods ?

(triggered by #27 (comment))

There is maybe some copy/paste typo in SingleConnectionSocketServerTransport ?

Should it be ?

-    private static final Logger LOGGER = LoggerFactory.getLogger(SingleConnectionSSLSocketServerTransport.class);
+    private static final Logger LOGGER = LoggerFactory.getLogger(SingleConnectionSocketServerTransport.class);

-           LOGGER.debug("SSLSocketServer is listening on " + serverSocket.getLocalSocketAddress());
+           LOGGER.debug("ServerSocket is listening on " + serverSocket.getLocalSocketAddress());

When you receive a request you can look to TransportContext to get "identifiication" of foreign peer.
But I can see nothing (let me know if I'm wrong) for ensuring identity of foreign peer when sending a request.

I give an example to better understand the issue.

Imagine 2 peer (A and B) which act both as coap server and client in a dynamic IP environment (e.g. peer A is behind a NAT)
Now imagine peer A initiates a (D)TLS handshake, then send a coap request to peer B.
Peer B check its identity and store its IP to send request later.
Now a Peer C which get same address IP than peer A (maybe because of NAT expiration), initiates an (D)TLS handshake with different authorized credentials.
Now Peer B want to send a request to peer A with current API the message will go to peer C.

If we could set a TransportContext constraint on outgoing request, it could be checked by transport layer.

I'm not sure I'm clear, let me know if this need more explanation. 🙏

Note that this kind of check should also probably be done to :

• match request and response
• match request and notification
• match blockwise request together

I don't know if this is a good idea but I face some situation where I need to handle a coap request OR a coap (separated) response in a same way. This is not too surprising as from a CoAP RFC point of view all request and response are conceptually a CoAP Message.

In my case, I need the remote peer address and the transport context of the message.
See :

• https://github.com/eclipse-leshan/leshan/blob/170d4095618e1371470ef27c61df5de4885b8610/leshan-tl-javacoap-core/src/main/java/org/eclipse/leshan/transport/javacoap/identity/DefaultCoapIdentityHandler.java#L27-L51
• https://github.com/eclipse-leshan/leshan/blob/170d4095618e1371470ef27c61df5de4885b8610/leshan-tl-javacoap-core/src/main/java/org/eclipse/leshan/transport/javacoap/identity/DefaultTlsIdentityHandler.java#L33-L47

Since recently (maybe release v6.12.0), there is a new responseTimeout concept. This is a great addition.

Currently this is mandatory but If someone want to manage this kind of timeout on its own currently there is no way to deactivate this.
A workaround could be to set a very long timeout but maybe, it could make sense to be able to deactivate this ?