kontalk / tigase-extension Goto Github PK

This is needed for country selection during registration. When a user tries to register, depending on the country code, trigger either a delegated registration or a local registration. The provider should act as a registration provider exposed to the kontalk:jabber:iq:register plugin that will have an internally configured list of supported providers for each country code.

Convert all plugins to XMPPProcessorAbstract, leveraging the utility methods for different use-cases when possible.

A more-or-less working probe engine.

• security: check JID of sender for incoming probe replies
• caching: cache some results for some time
• changes to roster match: backwards compatibility (localhost-only match + ignore packets in the plugin directed to the probe component)

Implement server list command.

Currently, the expired users query uses only the last_logout field. In reality, a user who has never logged in should be expired too after some time.
This involves cleaning up dummy user accounts created because of #50.
Exclude admin and special users (such as vhost-manager and db-properties).

Implement a registration provider for other servers that can't afford a SMS provider. This provider will contact a pre-authorized server that will send a verification code for it.
Possible implementation: a web service that the server will contact to request such a service - similar to the Nexmo verify API (actually we could use pretty much the same code)

As of subject.

This one seems better then Cognalys: https://checkmobi.com/

Write a new optimized module off of OfflineMessages for handling expiration and other things.

Because Kontalk is based on phone number hashes, it wouldn't make sense to register the same number twice in two different servers. Therefore, when a registration attempt occurs, ask confirmation to the user that the old account on the other server will be invalidated.

This must be done both by check locally and remotely (network probe).

Use error response with type=modify and motivation "User account already exists on another server". Require the user to resubmit the registration form, appending an additional field to enforce the process.

And take the opportunity to write a common component (actually it might just be a simple class with a singleton instance) to broadcast requests to the network and receive replies from them.

During subscription workflow, key is not always available for writing encrypted messages. Ensure that it can be requested as soon as the contact invites you or accepts your invitation.

Send push notifications to users with pending messages on startup.

This is part of the initial planning of Kontalk.

A server should look at its JID cache tables and see how many clients are connected locally and to other servers. It should redirect a client to another server if the load is too much (or even to just balance it out).

Might be worthy to look into XEP-0051.

Clients shall include some data in the stream to indicate the server that it doesn't want to be redirected (in order to avoid redirection loops).

Discard messages containing chat state messages only (e.g. typing notifications). Also delay delivery receipts.

Implement a registration form instruction list for jabber:iq:register iq type=get requests.

Remove nexmo-sdk.jar from the repository and integrate the new 1.6 version.

The Nexmo team doesn't seem to be interested...
https://github.com/AssuredLabor/nexmo-java-sdk

Actually they finally moved to Gradle/Maven:

https://github.com/Nexmo/nexmo-java

Revert commit d49f3d8. Will have to wait until bug 85 in Tigase is fixed.

References:

• https://projects.tigase.org/boards/4/topics/5210
• https://projects.tigase.org/issues/2561#note-7

Sometimes it would be nicer to have messages sent to an unauthorized user kept in storage for some time, because probably soon after an invitation workflow will occur. However, those messages must be sent in cleartext because the public key is not available yet.

http://xmpp.org/rfcs/rfc6121.html#sub-preapproval

Tigase ticket: https://projects.tigase.org/issues/1590

Unfortunately not implemented by Tigase, this is needed for an optimal subscription workflow.

• implement state transitions
• add <sub/> stream feature
• prevent subscribed presence to reach destination
• do not send roster pushes for pre-approvals
• do not emit roster items for state none_pre_approval_in
• hide approved attribute for pre_approval_in states
• reply with subscribed on behalf of the user when the contact sends subscribe and do not forward the subscribe

Adding a server requires the server to restart. This is not very dynamic. Implement an admin command to reload the server list.

Using GPGME for authentication is too slow. Consider caching the public keys. Authentication currently has the following steps:

• import key (to check for changes - use simple byte comparison or hashing against the cached value)
• retrieve key with new changes (only if key changed and needed GnuPG processing)
• verify key validity (expired, revoked - only if key changed)
• check for valid server signature (only if key changed)

All these steps can be skipped if the key is not changed.

Consider checking for null values first.

tigase.db.TigaseDBException: Problem loading user info from repository
        at org.kontalk.xmppserver.presence.JDBCPresenceRepository.getLastLogout(JDBCPresenceRepository.java:120)
        at org.kontalk.xmppserver.presence.LastActivity.getTime(LastActivity.java:76)
        at org.kontalk.xmppserver.presence.LastActivity.processNullSessionPacket(LastActivity.java:137)
        at tigase.xmpp.XMPPProcessorAbstract.process(XMPPProcessorAbstract.java:73)
        at tigase.server.xmppsession.SessionManager$ProcessorWorkerThread.process(SessionManager.java:2455)
        at tigase.util.WorkerThread.run(WorkerThread.java:128)
Caused by: java.sql.SQLException: Value '0000-00-00 00:00:00' can not be represented as java.sql.Timestamp
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:996)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:935)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:924)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:870)
        at com.mysql.jdbc.ResultSetRow.getTimestampFast(ResultSetRow.java:928)
        at com.mysql.jdbc.ByteArrayRow.getTimestampFast(ByteArrayRow.java:130)
        at com.mysql.jdbc.ResultSetImpl.getTimestampInternal(ResultSetImpl.java:5943)
        at com.mysql.jdbc.ResultSetImpl.getTimestamp(ResultSetImpl.java:5609)
        at org.kontalk.xmppserver.presence.JDBCPresenceRepository.getLastLogout(JDBCPresenceRepository.java:113)
        at org.kontalk.xmppserver.presence.LastActivity.getTime(LastActivity.java:76)
        at org.kontalk.xmppserver.presence.LastActivity.processNullSessionPacket(LastActivity.java:137)
        at tigase.xmpp.XMPPProcessorAbstract.process(XMPPProcessorAbstract.java:73)
        at tigase.server.xmppsession.SessionManager$ProcessorWorkerThread.process(SessionManager.java:2455)
        at tigase.util.WorkerThread.run(WorkerThread.java:128)

It uses missed calls with a random number that must be guessed by the user to be verified: https://www.cognalys.com/
Basic plan provides 50 verifications/day which would be a great saving.

A protocol to migrate an account automatically from one server to another. Useful when a server is dismissed or is down for long periods.

Clients should then send an automatic subscription request to all buddies in the roster.

Provide an in-band extension to retrieve provider information to be displayed in the app splash screen.

Following #77, this needs to be done on two levels (since sponsors may differ):

• server brand (server list + another ad-hoc command for on demand information)
• registration provider brand (registration response)

The first idea was to make it only in the server list command, but on second thought it could be useful to also add this information to the registration protocol (i.e. some more data form fields on the reponse) for the registration provider brand, and via another ad-hoc command for the server brand.

Ref. kontalk/androidclient#518.

https://docs.nexmo.com/index.php/verify

• verification provider should partially handle the registration workflow
• base verification provider used for "manual" workflow (that is, manually storing codes and sending SMS)
• base verification provider used for "delegated" workflow (e.g. Nexmo Verify, delegated registration maybe)
• refactor KontalkIqRegister to delegate code generation and storing to the actual verification provider

Remove unused components and plugins (e.g. amp, iq:register, carbons) and adjust operating system parameters.

References:

• http://docs.tigase.org/tigase-server/snapshot/Administration_Guide/html/
• http://web.archive.org/web/20140214003006/http://www.tigase.org/content/tigase-load-tests-again-500k-user-connections

• fs.file-max / ulimit nofile
• disable unused components

Current tests are using an unknown key. Generate one and version it.

Send broadcasts to all users in the local server for scheduled down time or other stuff. Use one of the scripts provided by Tigase as a template.

• send messages to all registered users in the domain of the sender JID
• use server key to sign message (add new method to GnuPGInterface)

Prepare server for Groovy script execution:

• Groovy dependency in pom
• save scripts in tigase-extension
• tigase-kontalk/pom.xml: copy scripts over from tigase-extension

This broadcast message thing also requires modifications to the Android client because as soon as the conversation window opens, it will try to subscribe to the user (as a temporary workaround, do not subscribe to non-complete JIDs - because broadcast messages will come from the domain JID).
Also, processing of signed-only messages must be implemented.

One more note: since the user is never added either to the roster or to the users table, UsersProvider will always try to request a sync. We should probably avoid that by using isDomainJID maybe.

I already tested with mixed Kontalk 2.x and 3.x servers on the same host that concurrent accesses to the same keyring causes crashes, so using multiple contexts on the same keyring won't work.
Also, when keyrings get big (as in very big), GnuPG performance degrades like hell.

Crazy thing to make concurrent calls to GnuPG

Waiting to implement an efficient GnuPG library implementation...

Since GnuPG is currently used as a key/value storage (where keys are fingerprints), the key space can be splitted (e.g. 16 GnuPG home directories for each first hex digit of the fingerprint; for what is worth, every even number from 2 to 16 will do).
By using multiple keyrings, fingerprint key space can be splitted in many parts and GnuPG operations can happen on multiple contexts simultaneously.

• configuration parameter for number of partitions
• splitted keyring implementation, transparent to the caller (make an interface to the standard gnupg-for-java methods)
• shell script for converting unified keyring to splitted keyring

Implement XEP-0191: Blocking Command.

Behaviours might be related:

• during mobile inactive state, presence are delivered to mobile nevertheless
• during mobile inactive state, presence are not delivered to desktop

It might be related to having multiple sessions.

Look also into a problem with isDeliveryReceipt().

And remove all those extra db-uri settings (actually not all of them):
https://projects.tigase.org/projects/tigase-server/repository/revisions/a254e0699986677de918058ace38908796410e0f

And maybe we could also get rid of kontalk/tigase-server@d5c2d83.

It seems like setting the table encoding to utf8mb4 wasn't enough...

2015-03-04 20:46:25.027 [in_9-sess-man]    JDBCRepository.addDataList()       WARNING:  Error adding data list, user_id: [email protected], subnode: offline/msgoffline, key: messages, uid: 183, nid: 420, list: [<message type="chat" xmlns="jabber:client" from="[email protected]/xxxxxx" id="yve5jL0QynUf3eZ38Eq9w2CRk60dRD" to="[email protected]"><body>😉</body><active xmlns="http://jabbr.org/protocol/chatstates"/><request xmlns="urn:xmpp:receipts"/><delay from="beta.kontalk.net" stamp="2015-03-04T20:46:25.023Z" xmlns="urn:xmpp:delay">Offline Storage - beta.kontalk.net</delay></message>]
java.sql.SQLException: Incorrect string value: '\xF0\x9F\x98\x89</...' for column 'pval' at row 1
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3491)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3423)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1936)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2060)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2542)
        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1734)
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2019)
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1937)
        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1922)
        at tigase.db.jdbc.JDBCRepository.addDataList(JDBCRepository.java:180)
        at tigase.db.jdbc.JDBCRepository.addDataList(JDBCRepository.java:200)
        at tigase.db.UserRepositoryMDImpl.addDataList(UserRepositoryMDImpl.java:63)
        at tigase.db.NonAuthUserRepositoryImpl.addOfflineDataList(NonAuthUserRepositoryImpl.java:101)
        at tigase.xmpp.impl.OfflineMessages$MsgRepositoryImpl.storeMessage(OfflineMessages.java:506)
        at tigase.xmpp.impl.OfflineMessages.savePacketForOffLineUser(OfflineMessages.java:301)
        at tigase.xmpp.impl.OfflineMessages.postProcess(OfflineMessages.java:151)

This ia serious problem. User accounts are re-created with empty data while accessing stuff like public keys, last activity data, etc.
Remove &autoCreateUser=true from the db-uris and check for UserNotFoundExceptions.

Integrate 3rd party services for hosting attachment data, or even better ask the user to upload them to their own space (e.g. GDrive, Dropbox, etc.).

Although experimental, it's better than nothing.
http://xmpp.org/extensions/xep-0352.html

Currently, SSL client certificate passing works only on STARTTLS. Make it work also on old direct SSL connections.

Apparently, using last-seen attribute from the roster is not enough: that is the last seen value from the user's point of view, it's not absolute. Use the last_logout field from the tig_users table and inject the delay element through a custom roster implementation extending RosterFlat and overriding just getCustomChild.

Roster implementation class must then set by mean of system properties, namely roster-implementation.

Delete inactive users after some time (e.g. a month). Also ensure unsubscription to and from all users.

Continuing from kontalk/xmppserver#24

Discussion going on in kontalk/desktopclient-java#1.

In the current implementation, push notifications are sent immediately for each message until the client connects. Remember it for some time and don't send it repeatedly, otherwise we risk consuming too much requests (although GCM has no limits, Google might check).

Support for a IQ to unregister from push notifications.

Instead of using roster element last-seen or a custom roster implementation (#42), we could implement XEP-0012: Last Activity backed by the last_logout value.

If we end up using this, we'll need to unconfigure sess-man/plugins-conf/presence/offline-roster-last-seen.

A nice and clean server administration UI, possibly implemented as a XMPP component.

Hi,

i made all of setup as u described on wiki -setup for local- and change to my server ips.But when i can get "request verification code" im getting that error :

2629-2728/? W/ActivityManager﹕ Activity stop timeout for ActivityRecord{125b077 u0 com.foursquare.robin/.activities.MainActivity t461}
07-26 00:45:49.537 19509-19574/? W/NumberValidator﹕ connection error - trying next server in list
org.jivesoftware.smack.SmackException: javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.parsePackets(XMPPTCPConnection.java:1008)
at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.access$200(XMPPTCPConnection.java:935)
at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader$1.run(XMPPTCPConnection.java:950)
at java.lang.Thread.run(Thread.java:818)
Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:322)
at com.android.org.conscrypt.OpenSSLSocketImpl.waitForHandshake(OpenSSLSocketImpl.java:623)
at com.android.org.conscrypt.OpenSSLSocketImpl.getInputStream(OpenSSLSocketImpl.java:585)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.initReaderAndWriter(XMPPTCPConnection.java:642)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.proceedTLSReceived(XMPPTCPConnection.java:742)
at org.jivesoftware.smack.tcp.XMPPTCPConnection.access$900(XMPPTCPConnection.java:139)
at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.parsePackets(XMPPTCPConnection.java:1001)
            at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader.access$200(XMPPTCPConnection.java:935)
            at org.jivesoftware.smack.tcp.XMPPTCPConnection$PacketReader$1.run(XMPPTCPConnection.java:950)
            at java.lang.Thread.run(Thread.java:818)
Caused by: java.security.cert.CertificateException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.
at com.android.org.conscrypt.TrustManagerImpl.checkTrusted(TrustManagerImpl.java:318)
at com.android.org.conscrypt.TrustManagerImpl.checkServerTrusted(TrustManagerImpl.java:219)
at com.android.org.conscrypt.Platform.checkServerTrusted(Platform.java:114)
at com.android.org.conscrypt.OpenSSLSocketImpl.verifyCertificateChain(OpenSSLSocketImpl.java:550)
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)

Trusting the provided public key on registration time just matching the user id isn't enough. Let the user use a bridge certificate signed with the key is going to handshake SSL with, so the server will know what key to sign - after that, the base64-encoded public key won't be needed any more.

An implementation client-side is also required to create a bridge certificate used just for this purpose.

Look into Tigase configuration for the admin accounts and set the account purger to never expire them.

See MobileV3.

Since unavailable presence is not broadcasted when requesting the roster, fingerprint updates are not published at login time. Instead of broadcasting unavailable presences, we could add a fingerprint attribute to the <item/> node at node building time (that is, do not store it).

https://projects.tigase.org/issues/2530#note-16