otrv4 / libotr-ng

A new implementation of OTR with support for version 4. This is a mirror of https://bugs.otr.im/otrv4/libotr-ng

OTR Next Generation library

Build

Before you try to build libotr-ng, verify you have installed:

• autoconf - https://www.gnu.org/software/autoconf/autoconf.html
• splint - http://splint.org/documentation
• valgrind - http://valgrind.org/. At least, version 3.12.0

Then, install the following dependencies:

• libglib2.0-dev
• libgcrypt 1.8.0 or newer
• libgoldilocks
• libsodium-dev
• libotr 4.x

How to install

To generate project configuration:

$ autoreconf --install

To configure the project:

$ ./configure

To build and install:

$ make && make install

To run the tests:

$ make test

Configure and install the library with other options

To configure the project with OTRNG debug output:

./configure CFLAGS="-ggdb3 -O0 -D DEBUG_API=yes"

To configure the project with debug option using gdb:

$ ./configure CFLAGS="-g -ggdb3 -O0" CXXFLAGS="-g -ggdb3 -O0" LDFLAGS="-g -ggdb3" --disable-shared
$ gdb [path/test/executable]

To configure project with debug option using lldb:

$ ./configure CFLAGS="-g -ggdb3 -O0" CXXFLAGS="-g -ggdb3 -O0" LDFLAGS="-g -ggdb3" --disable-shared
$ lldb [path/test/executable]

To configure project with debug option:

$ ./configure --enable-debug

To run make code-style:

Install:

• clang-format-3.5

Usage

This library is primarily meant to be used by instance messaging clients to provide OTRv4 encryption on top of the library. The main entry points for the functionality can be found in client.h - the functions otrng_client_receive and otrng_client_send can be used as starting points for investigation of the functionality.

Thread safety

The libotr-ng library is not inherently thread safe. Since the library is meant to be used in widely different environments with different threading libraries and process requirements, it would be difficult for the library itself to use a specific thread safety paradigm. Thus, the client application will have to ensure this.

Almost all functions in this library are safe in the sense that they only modify structures and memory that has been explicitly sent in as arguments to the functions. There is no global state in the system, with the exception of the out-of-memory handler. This means that it is possible to concurrently call functions as long as they don't modify the same memory area. However, for large chunks of the OTR functionality, the functions will touch on central areas relating to lists of clients.

The limit of interaction between OTR functions is guided by the otrng_global_state_s structure - for client instances that are part of d ifferent global states, it is completely safe to call functions concurrently.

In terms of how to ensure thread safety, the easiest way would be to make sure there's always a lock around calls to OTR functionality, so that only one call from a client to the library can happen at the same time. Another way would be to always serialize calls using a message queue. Another way would be - like in Pidgin - to not have multipled threads at all. However how you choose to do it, it's important to ensure this thread safety, since the results of not doing so can be potentially very dangerous.

Funding

The work made hare was partially supported by the NlNet Foundation. Find information here.

License

libotr-ng's Issues

Hash the user profile

As defined on the spec, do a KDF of the user profile when calculating the Auth.

1. For interactive:
   For Auth-I and Auth-R message:
   t = 0x0 || KDF_2(Bobs_User_Profile) || KDF_2(Alices_User_Profile) || Y || X || B || A || KDF_2(Φ)

2. For non interactive
   For Non-Interactive-Auth message:
   t = KDF_2(Bobs_User_Profile) || KDF_2(Alices_User_Profile) || Y || X || B || A || KDF_2(Φ) || their_shared_prekey

Multi-instance conversation

From the plugin perspective, something like this should be possible:

1. Alice (on device 1101) has an OTR conversation with Bob (on device 1201).
2. Bob (on device 1202) starts a new OTR conversation with Alice (still on device 1101).
3. Alice should have both conversations running simultaneously, and should be able to receive and send messages to both conversations.

libotr keeps a separate context for each of these conversations and automatically manages when to add new contexts when a new instance tag is seen.

When a message is sent, you may choose to send it to a particular instance tag or tell the library which strategy to use when deciding which instance should receive the message (OTRL_INSTAG_BEST, OTRL_INSTAG_RECENT, OTRL_INSTAG_RECENT_RECEIVED, OTRL_INSTAG_RECENT_SENT).

At the moment, we only support a single conversation with a peer (identified by its ID, like [email protected]).

Memory leaks while executing some tests

Currently, there seems to be three memory leaks (that show up randomly on ci):

==10767== Use of uninitialised value of size 8
==10767== at 0x5149819: encodeblock (b64.c:89)
==10767== by 0x51499BA: otrl_base64_encode (b64.c:108)
==10767== by 0x5149B5B: otrl_base64_otr_encode (b64.c:218)
==10767== by 0x4289ED: serialize_and_encode_data_msg (otrv4.c:1405)
==10767== by 0x428C1C: send_data_message (otrv4.c:1450)
==10767== by 0x428F69: otrv4_prepare_to_send_data_message (otrv4.c:1531)
==10767== by 0x42900C: otrv4_prepare_to_send_message (otrv4.c:1550)
==10767== by 0x42906F: otrv4_close_v4 (otrv4.c:1566)
==10767== by 0x42911E: otrv4_close (otrv4.c:1589)
==10767== by 0x41B599: otr4_client_disconnect (client.c:332)
==10767== by 0x40C4AB: test_invalid_auth_r_msg_in_not_waiting_auth_r (test_client.c:614)
==10767== by 0x4EA3A80: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4002.0)
==10767== Uninitialised value was created by a stack allocation
==10767== at 0x4203AB: calculate_brace_key (key_management.c:352)

==10611== 16 bytes in 1 blocks are possibly lost in loss record 176 of 427
==10611== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10611== by 0x514ABC9: otrl_mem_malloc (mem.c:60)
==10611== by 0x57E4E89: do_malloc (global.c:912)
==10611== by 0x57E64D2: _gcry_malloc_secure_core (global.c:946)
==10611== by 0x57E64D2: _gcry_xmalloc_secure (global.c:1145)
==10611== by 0x57E6596: _gcry_xcalloc_secure (global.c:1190)
==10611== by 0x58A8259: _gcry_mpi_resize (mpiutil.c:187)
==10611== by 0x58A5FCB: mpi_fromstr (mpicoder.c:134)
==10611== by 0x58A5FCB: _gcry_mpi_scan (mpicoder.c:602)
==10611== by 0x57E23C8: gcry_mpi_scan (visibility.c:357)
==10611== by 0x41B34F: dh_init (dh.c:39)
==10611== by 0x40B30D: test_valid_identity_msg_in_waiting_auth_r_lower (test_client.c:647)
==10611== by 0x4EA3A80: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4002.0)
==10611== by 0x4EA3C45: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4002.0)

==10611== 32 bytes in 1 blocks are possibly lost in loss record 251 of 427
==10611== at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10611== by 0x514ABC9: otrl_mem_malloc (mem.c:60)
==10611== by 0x57E4E89: do_malloc (global.c:912)
==10611== by 0x57E63CC: _gcry_malloc (global.c:936)
==10611== by 0x57E63CC: _gcry_xmalloc (global.c:1110)
==10611== by 0x58A80F1: _gcry_mpi_alloc_secure (mpiutil.c:105)
==10611== by 0x58A5CBE: _gcry_mpi_scan (mpicoder.c:519)
==10611== by 0x57E23C8: gcry_mpi_scan (visibility.c:357)
==10611== by 0x41B422: dh_keypair_generate (dh.c:66)
==10611== by 0x41C811: key_manager_generate_ephemeral_keys (key_management.c:115)
==10611== by 0x41EA1E: start_dake (otrv4.c:427)
==10611== by 0x421F76: receive_query_message (otrv4.c:467)
==10611== by 0x421F76: receive_message_v4_only (otrv4.c:1262)
==10611== by 0x421F76: otrv4_receive_message (otrv4.c:1292)
==10611== by 0x419907: otr4_client_receive (client.c:279)

Implement the interaction with the prekey server

As it will be defined in issue 77. This will replace the mock of #36 , but keep in mind what has been implemented there.

Should we include a copyright statement in the files?

Fix Pidgin Plugin to work with shared prekeys

Pidgin plugin is broken with the introduction of shared prekeys in libotrv4

Receive Identity Message while on state WAITING_AUTH_I

Mem leak of tests when not placed in a specific order

The test test_api_multiple_clients will fail when not listed at the bottom of the test.c file.
Seems like tests are interfering with each other. Seems like any test with do_ake_fixture will fail if not placed correctly on the file.

Check d335b38

Check full backwards compatibility with libotr

Why

The library should be backwards compatible with libotr.

References

See otrv4/otrv4#48

Tasks

• Check what is missing from libotr to be "fully" compatible.
• Check that all callbacks are working.
• Check that everything from the bottom list is covered.

Open questions

None.

Be consistent with scalar serialization

This includes at least:

• change the SNIZKPK to use the SCALAR and not MPI (spec and implementation),
• change all SMP messages to use the SCALAR type and not MPI (spec and implementation),

From: 9302e15#commitcomment-22181306

Use autotools

We need to use autotools (autoconf, automake, libtool) to make the library more portable and manage dependencies and other requirements.

OTR full flow

• Send Query message
• Reply with pre-key
• Receive pre-key and reply with DRE-Auth
• Receive and very DRE-Auth

Error handling for the Dakez

Manage instance tags

Like OTR3, we should create an manage instance tags

Add non-interactive AKE to the spec

Allow generating v3 and v4 long term keys independently

The current UI only shows the fingerprint for the OTRv4 long-term key, and the generate button is only enabled if the OTRv4 key is missing.

Update libdecaf to master branch

https://sourceforge.net/p/ed448goldilocks/code/ci/master/tree/

Generate Public Shared PreKey

Generate and incorporate Public Shared Prekey to the user profile.

Invalid read of size 16 on memchr and fgets

On OsX, Xcode 8.3, there are an 'Invalid read of size 16' on:

1. test api/smp
2. test api/smp_abort
3. test api/dh_rotation
4. test instance_tag/generates_when_file_is_full
5. test client/sends_fragments

They are related to memchr and fgets on extract_tlvs function. Seems like uint8_t tlvs is not initialized or not null terminated, or tlvs are being dereferenced.

This seemed to have been hidden by the used of stdbool, for some reason.

It only fails on this osx, so it might be an optimization that went wrong on it or that valgrind is not checking correctly. Oddly, when trying to print the address of pointer, valgrind complains of conditional jump.

Finish SMP

Error managing

• How we are managing errors?
• Is it protected, to not leak information?
• Is it standard around our code?
• How good it is defined in the spec?
• Check bools
• Check normal error: success and failures
• Check all paths, specially the discard on instance tags.
• Check bools and otrv4 errors: use 1 or 0?

Free when is actually needed

On several places on the library, some things that don't need to be freeded are being done so. Check for example what was done here: 0a75ad0. A data_message_free(msg) was called when only a free(msg) was needed. This should be checked consistently on the library as well as on tests.

Add fragmentation

Check the double ratchet implementation

Some things might be missing or incorrect.

Add OTRv3 support to OTRv4

Implement Heartbeat Messages

Why

As we are currently implementing the revision number 2 of the OTRv4 specification, we need to correctly implement heartbeat messages.

Reference

The current implementation seems not to scale to work with a client.

See "Data Exchange" section, second paragraph, and "Receiving a Data Message" section for reference around heartbeat messages

For further information, check issue #24

Tasks

• Check the current behavior of heartbeat messages.
• Correctly make time configurable.
• Make/change the current implementation to work with a client.
• Correctly check the use of the 'IGNORE_UNREADABLE' as it is stated on the OTRv4 spec.
• Check the behaviors of the 'IGNORE_UNREADABLE' with the TLVs
• Correctly test the heartbeat messages and see which functions really need to be exposed to the tests (check the exposition of the otrl_base64_otr_decode function).
• Check how all TLVs are working with this.

Open questions

• Might be worth checking the behavior of the 'IGNORE_UNREADABLE' flag with Dave Goulet or DrWhax.

User Profile signatures

We are not signing nor verifying user profile signatures.

Correctly generate the ssid and the shared secret (for non interactive)

Client Profile API

To create an API to generate and publish Client Profile.

Should we really use enums? Maybe try to use the "string" enum pattern instead?

Why

The library should use good C practices.

Reference

21st Century C, chapter 7, "Enums and string". The way "open" syscalls work.

Tasks

• Check the current behavior of enums.
• Consider changing them to the "string" enum pattern if found needed.
• Change the needed ones.
• Add to our 'style.md' file this idiom for the library, and state why it is good to do this.

Open questions

None.

Add support to type 0 TLVs (padding)

We can receive padding TLVS but cant send.

Send heartbeat messages

This might also need the implementation of the 'IGNORE_UNREADABLE' flag.

Safely delete old ratchet keys

Correctly handle memory and functional behavior of TLVs

Why

A library should be sufficiently secure and functional in the ways it is written. TLV handling should follow that.

Reference

As the TODO here included: ee751f0 and as discussed here: f1bfaff and here f93d7b4

Tasks

• Check the correct behavior when padding TLVs
• Correctly and safely write 'otrv4_tlv_new', taken into account when a NULL value is passed.
• Correctly think about memory and rewrite append_padding_tlv.
• Check over the tests how TLVs are handled. Some TLVs that don't have the 'value' type are been putted like so, for example.

Open questions

• Is all memory correctly managed over TLVs?

What about policies to be used by plugins?

Should we implement the policies in the plugin?

How should we communicate v3 + v4 long-term keys (and fingerprints) to the user?

The OTRv4 long-term (public) key is published together with a user profile that's signed with the OTRv3 long-term (private) key.

Can we make use of this and simplify the UI? Can we omit OTRv4 fingerprint when there's a profile signed by a trusted OTRv3 key?

Incorporate the non-interactive Deniable Authenticated Key Exchange

Incorporate the Non-interactive Deniable Authenticated Key Exchange:

• Generate the public shared PreKey #35
• Generate pre key messages
• Generate non interactive auth messages
• Incorporate both messages to DAKE
• Incorporate into data messages flow. Take into account the two cases:
• • normal "flow" of data messages
• • a data message can be attached in the Non-Interactive-Auth message. Generate the nonce on that step
• Update the protocol state machine for non-interactive DAKE

Refactor the pidgin plugin

There are a few things we could probably remove, like:

• ConnContext. Remove the functions:
  • otrg_plugin_context_to_trust
  • otrg_plugin_conv_to_selected_context
  • conn_context_to_plugin_conversation
• Handle warnings

Support stable releases of libgcrypt for Debian and Fedora

Currently, our libotrv4 library requires a version of libgcrypt (1.7.6) that is higher than the stable versions that we can get from package managers on Debian and Fedora. We depend on 1.7 because we use constants like GCRY_MD_SHA3_512 that are defined in versions of libgcrypt higher than 1.6.9.

Debian jessie (stable) supports a patched version of libgcrypt called: 1.6.3-2+deb8u2.
Fedora 25 (stable) supports the version: 1.6.6-1.

If we intend to support these stable releases for major linux distributions, we will need to use an additional library that provides SHA3 functions and constants.

Fingerprint manager

Managing fingerprints and trust level is not specified in the protocol, but it's still implemented in the original libotr. It's seems to have a significant amount of code.

We are considering to implement a minimal manager that's gonna be used only on the reference pidgin plugin.

Tasks:

• Add a "fingerprint store" to the plugin (it's currently inside libotr as part of user state).
• Update the "Known fingerprints" UI to use this storage.

Derive root and chain keys from previous root key when needed

The first root and chain keys do not use the current root key as it doesn't exist yet. But when deriving later root keys, the current root key must be passed into the KDF. As in the spec:

derive_ratchet_keys(R_i-1, K):
R_i = KDF_2(0x01 || R_i-1 || K)
Ca = KDF_2(0x02 || R_i-1 || K)
Cb = KDF_2(0x03 || R_i-1 || K)
return R, decide_between_chain_keys(Ca, Cb)

Choose a padding method

Currently, on libotr3 the padding method used seems loosely based on PKCS#7 as described on RFC 5622. If this is the case, the value of each added byte is the number of bytes that are added, not zero as currently seems to be. If the padding method chose is Zero padding, as usually specified in hashes and macs, it must be taken in account that this has not been specified for encryption.

Padding does not seemed to be defined on libotr3 C library as well as on the protocol.

We need, therefore, to chose a padding method for this lib.

Reveal old MAC keys

Public API

At the moment we have two sets of API:

Protocol API (#include <libotr4/otrv4.h>)

This is the API for "Off-the-Record Messaging Protocol version 4". A protocol manages a connection that allows different versions of the wire protocols defined in the spec (version 3 and version 4). Once two running protocols decide on a wire protocol version, they use it to start a connection by running a key-exchange algorithm.

A connection is able to process incoming zero-terminated *char messages ("strings") from the IM network and decide how to respond to it. Incoming messages may cause a zero-terminated *char message (OTR-encoded "strings") to be send as a response, or contain a *uint8_t plaintext message (array of bytes).

A connection is able to process *uint8_t plaintext message (array of bytes) into a zero-terminated *char message (either OTR-encoded or plaintext message "strings") before sending to the IM network.

Client API (#include <libotr4/client.h>)

This is a "messaging client". A client manages multiple conversations with other recipients/senders, identified by a identity ("[email protected]", for example).

A client knows which conversation to use in order to send/receive zero-terminated *char messages ("strings") to/from a recipient/sender.

Incorporate the shared session state

Incorporate the shared session state (phi) to the Auth.

Random failure of some tests

Locally, on OS, the 'test_fixtures' sometimes fails, while comparing root keys.

'do_ake_fixture: assertion failed: (alice->keys->current->root_key ==  bob->keys->current->root_key)'. 

Might be related to manually setting 'our_dh->pub' in 'identity_message_in_waiting_auth_r' test or to removing ratchet id.

Expire the otr session

As defined on the spec, section Session expiration. This will probably need to have all the offline/non interactive scenarios set, as well as the heartbeat messages. This also potentially impacts the non-interactive-auth message when an encrypted message is attached to it and a ratchet begins. Will these messages have an expiration time, then?

Auto generate key pair

Generate the extra symmetric key

Take the shared prekey and phi from the client

Why

A library should be able to take values from a client and not use dummy values.

Reference

Replace dummy values introduced on 208388b

Tasks

• Check that phi_val variable needs to be actually a char.
• On otrng.c, check this code and remove the dummy value:

char *phi_val = NULL;
    if (!phi) {
      phi = "";
    }
    phi_val = otrng_strdup(phi);
    if (!phi_val)
      return ERROR;

• On otrng.c, remove the dummy value (notice that this variable name is wrong as this is not the symm key):

uint8_t sym_key[ED448_PRIVATE_BYTES] = {0x01};

• Create a functionality to take these values from the client.

Depends on/related to

• Issue #58

Open questions

• Are all the types used on this functionality correct?