peppelinux / pymultildap

OpenLDAP proxy or simple python3 LDAP client to handle multiple LDAP connections, data aggregation and manipulation strategies

pyMultiLDAP

pyMultiLDAP can gather data from multiple LDAP servers, can do data aggregation and manipulation with rewrite rules. pyMultiLDAP can act also as a proxy server, behind openldap's slapd-sock backend or any custom implementation.

Features

• LDAP client to many servers as a single one;
• Custom functions to manipulate returning data (rewrite rules);
• Export data in python dictionary, json or ldiff format;
• Proxy Server, exposing a server daemon usable with slapd-sock backend.

pyMultiLDAP doesn't write to LDAP servers, it just handle readonly data. It's also used to automate smart data processing on-the-fly.

See example/settings.py.example and multildap/attr_rewrite.py to understand how to configure and extend it.

Tested on

• Debian9;
• Debian10.

Setup

Configure multiple connections and search paramenters in settings.py.

Install

git clone https://github.com/peppelinux/pyMultiLDAP.git
cd pyMultiLDAP
pip install -r requirements
python3 setup.py install

or use pipy [WIP]

pip install pyMultiLDAP

LdapClient Class usage

from multildap.client import LdapClient
from settings import LDAP_CONNECTIONS

lc = LdapClient(LDAP_CONNECTIONS['SAMVICE'])

# get all the results
lc.get()

# apply a filter
lc.get(search="(&(sn=de marco)(schacPersonalUniqueId=*DMRGPP83*))")

Search and get

See examples/run_test.py.

Difference between .search and .get:

• search relyies on connection configuration and returns result as it come (raw);
• get handles custom search filter and retrieve result as dictionary, json, ldif or python object format. It also apply rewrite rules.

import copy

from multildap.client import LdapClient
from settings import LDAP_CONNECTIONS

lc = LdapClient(LDAP_CONNECTIONS['DEFAULT'])

kwargs = copy.copy(lc.conf)
kwargs['search']['search_filter'] = "(&(sn=de medici)(givenName=aurora))"
r = lc.search(**kwargs['search'])

Results in json format

from multildap.client import LdapClient
from . settings import LDAP_CONNECTIONS


for i in LDAP_CONNECTIONS:
    lc = LdapClient(LDAP_CONNECTIONS[i])
    print('# Results from: {} ...'.format(lc))

    # get all as defined search_filter configured in settings connection
    # but in json format
    r = lc.get(format='json')
    print(r)

    # set a custom search as method argument
    r = lc.get(search="(&(sn=de marco)(schacPersonalUniqueId=*DMRGPP345tg86H))", format='json')
    print(r)

    print('# End {}'.format(i))

Run the server

Network address

multildapd.py -conf settings.py -port 1234

Unix domain socket (for slapd-sock backend)

multildapd.py -conf ./settings.py -loglevel "DEBUG" -socket /var/run/multildap.sock -pid /var/run/multildap.pid -uid openldap

Dummy test without any ldap client connection configured, just to test slapd-sock:

multildapd.py -conf ./settings.py -dummy -loglevel "DEBUG" -socket /var/run/multildap.sock -pid /var/run/multildap.pid

Test Unix domain socket from cli

nc -U /tmp/multildap.sock

Interfacing it with OpenLDAP slapd-sock

The Slapd-sock backend to slapd uses an external program to handle queries. This makes it possible to have a pool of processes, which persist between requests. This allows multithreaded operation and a higher level of efficiency. Multildapd listens on a Unix domain socket and it must have been started independently;

This module may also be used as an overlay on top of some other database. Use as an overlay allows external actions to be triggered in response to operations on the main database.

Configure slapd-sock as database

Add the module.

ldapadd -Y EXTERNAL -H ldapi:/// <<EOF
dn: cn=module,cn=config
objectClass: olcModuleList
cn: module
olcModuleLoad: back_sock.la
EOF

Create the database.

ldapadd -Y EXTERNAL -H ldapi:/// <<EOF
dn: olcDatabase={4}sock,cn=config
objectClass: olcDbSocketConfig
olcDatabase: {4}sock
olcDbSocketPath: /var/run/multildap.sock
olcSuffix: dc=proxy,dc=testunical,dc=it
olcDbSocketExtensions: binddn peername ssf
EOF

Add an Overlay if you want to wrap an existing backend

ldapmodify -H ldapi:// -Y EXTERNAL <<EOF
dn: olcOverlay=sock,olcDatabase={1}mdb,cn=config
changetype: add
objectClass: olcConfig
objectClass: olcOverlayConfig
objectClass: olcOvSocketConfig
olcOverlay: sock
olcDbSocketPath: /var/run/multildap/multildap.sock
olcOvSocketOps: bind unbind search
olcOvSocketResps: search
EOF

Remember to configure an ACL otherwise only ldapsearch -H ldapi:// -Y EXTERNAL as root would fetch ldif. Remember to add a space char ' ' after every olaAccess line, otherwise you'll get Implementation specific error(80).

export BASEDC="dc=testunical,dc=it"

ldapadd -Y EXTERNAL -H ldapi:/// <<EOF
dn: olcDatabase={4}sock,cn=config
changeType: modify
replace: olcAccess
olcAccess: to *
 by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage
 by * break
# the following permits self BIND by users
olcAccess: to dn.subtree="dc=proxy,$BASEDC"
 by self read
 by * break
# the following two permits SEARCH by idp and foreign auth system
olcAccess: to dn.subtree="ou=people,$BASEDC"
 by dn.children="ou=auth,$BASEDC" read
 by self read
 by * break
olcAccess: to dn.subtree="ou=people,$BASEDC"
 by dn.children="ou=idp,$BASEDC" read
 by self read
 by * break
olcAccess: to *
 by anonymous auth
 by * break
EOF

Authentication (BIND) on top of the multildapd must be configured with attribute rewrite_dn_to regarding every connections in the settings.py. If abstent the specified connection will be excluded from authentication. TODO: adopt openldap proxy authz statements.

ldapsearch -H ldap://localhost:389 -D "uid=peppe,dc=proxy,dc=testunical,dc=it" -w thatsecret -b 'uid=peppe,dc=proxy,dc=unical,dc=it'

Hints

See databases currently installed:

• ldapsearch -Y EXTERNAL -H ldapi:/// -b 'cn=config' -LLL "olcDatabase=*";
• Use client_strategy = RESTARTABLE instead of REUSABLE in your settings.py for better performances;
• A Backend can not be deleted via ldapdelete/modify until OpenLDAP 2.5 will be released;
• Changing the socket path

ldapmodify -Y EXTERNAL -H ldapi:/// <<EOF
dn: olcDatabase={4}sock,cn=config
changetype: modify
replace: olcDbSocketPath
olcDbSocketPath: /var/run/multildap.sock
EOF

• Deploy a dummy socket listener with socat, just to debug incoming connection from slapd-sock.

socat -s UNIX-LISTEN:/tmp/slapd-sock,umask=000,fork EXEC:"$your_command"

Other slapd-sock resources:

• slapsock
• slapd-trigger
• ldap.h search scopes
• slapd-sock in OpenLDAP ML

Todo

• Example configuration with slapd's Proxy Authorization Rules (authzTo: dn.regex:^uid=[^,]*,dc=example,dc=com$);
• Only SEARCH, BIND and UNBIND is usable, other LDAP methods should be implemented;

pymultildap's Issues

Possible bug in _decode_elements

pyMultiLDAP/client.py

Lines 45 to 47 in dd8dcbb

	def _decode_elements(self, attr_dict):
	return {k:[e.decode(self.conf['encoding'] if isinstance(e, bytes) else e) for e in v]
	for k,v in attr_dict.items() }

This seems wrong:

e.decode(self.conf['encoding'] if isinstance(e, bytes) else e)

I am guessing what you wanted was:

e.decode(self.conf['encoding']) if isinstance(e, bytes) else e

---

progression on this:

• use proper names
• accumulate results

result = {
    attr_name: decoded_vals
    for attr_name, attr_vals in attr_dict.items()
    for decoded_vals in [[
        decoded_val
        for attr_val in attr_vals
        for decoded_val in [
            attr_val.decode(self.conf['encoding'])
            if isinstance(attr_val, bytes)
            else attr_val
        ]
    ]]
}

Nesting cannot be avoided; it follows the structure of the data. What you can do is split the different things you do in separate functions.

def _decode_elements(self, attr_dict):
    def decode_attr_val(attr_val, encoding):
        value = (
            attr_val.decode(encoding)
            if isinstance(attr_val, bytes)
            else attr_val
        )
        return value

    def decode_attr_vals(attr_vals, encoding):
        values = [decode_attr_val(attr_val, encoding) for attr_val in attr_vals]
        return values

    encoding = self.conf['encoding']
    result = {
        attr_name: decode_attr_vals(attr_vals, encoding)
        for attr_name, attr_vals in attr_dict.items()
    }
    return result

AttributeError: module 'gevent' has no attribute 'signal'

Hi,

I tried this package but I've got an error:
multildapd.py -conf setting.py -port 1234

[2021-07-14 06:35:31,182] [INFO] multildapd: Process run on pid: 322
Traceback (most recent call last):
  File "/usr/local/bin/multildapd.py", line 259, in <module>
    gevent.signal(signal.SIGTERM, stop_app, **{'pidfile': pidfile,
AttributeError: module 'gevent' has no attribute 'signal'

Steps to reproduce:

1. I use docker (I tried local mashine too but same result - Ubuntu Python 3.8) docker run -it python:3.6 /bin/bash
2. pip3 install proxyldap
3. create setting.py from example
4. multildapd.py -conf setting.py -port 1234
   this return error:

[2021-07-14 06:35:31,182] [INFO] multildapd: Process run on pid: 322
Traceback (most recent call last):
  File "/usr/local/bin/multildapd.py", line 259, in <module>
    gevent.signal(signal.SIGTERM, stop_app, **{'pidfile': pidfile,
AttributeError: module 'gevent' has no attribute 'signal'

Did I something wrong? Which Python version should I use? Thanks.

Fanda