kevinhorvatin / google-apps-for-your-domain-ldap-sync Goto Github PK

This is a Question , Whether this tool will sync Active Directory Passwords.

Please confirm,

Regards,

Rahul
[email protected]

Please bare with me as I don't know much about Python.  I have downloaded and 
installed Python 
2.5 on my 2003 Active Directory server.  Then downloaded your tarball and just 
extracted it onto 
the desktop.  I then opened Command Prompt and navigated to the desktop then 
ran 
sync_ldap.py.  I received the following errors:

Traceback (most recent call last):
File "C:\Documents and 
Settings\Administrator\Desktop\gafyd-ldap-sync\sync_ldap.py", line 
38, in <module>
import ldap_ctxt
File "C:\Documents and 
Settings\Administrator\Desktop\gafyd-ldap-sync\sync_ldap.py", line 
28, in <module>
import ldap
ImportError: No module named ldap

Any help appreciated.

What steps will reproduce the problem?
1. Sync
2. change the user so that is "disappears" from filer (e.g. make it
disabled in AD)
3. sync (user is disabled in google apps)
4. reenable the user in AD
5. the user is not enabled on google apps

What is the expected output? What do you see instead?

I would expect that the user would be enabled in google apps.

We have a non regular class its a derived class from organizationalPerson, 
i tried to set a limit to see a max 100 users but the sync_ldap crashes, i
have the debug output.

Command: set ldap_page_size 100
Command: testFilter
Traceback (most recent call last):
  File "./sync_ldap.py", line 183, in ?
    (config, ldap_context, user_database, google_context, log_config) = \
  File "./sync_ldap.py", line 112, in DoMain
    cmd.cmdloop()
  File "/usr/lib/python2.4/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.4/cmd.py", line 219, in onecmd
    return func(arg)
  File "/usr/local/adm/arc/google-apps-ldap-sync/commands.py", line 186, in
do_testFilter
    self.new_users = self.ldap_context.Search(attrlist=['cn'])
  File "/usr/local/adm/arc/google-apps-ldap-sync/ldap_ctxt.py", line 299,
in Search
    users = self._PagedAsyncSearch(query, sizelimit, attrlist=attrlist)
  File "/usr/local/adm/arc/google-apps-ldap-sync/ldap_ctxt.py", line 249,
in _PagedAsyncSearch
    query, attrlist=attrlist, serverctrls=serverctrls)
  File "/usr/lib/python2.4/site-packages/ldap/ldapobject.py", line 468, in
search_ext
    EncodeControlTuples(serverctrls),
  File "/usr/lib/python2.4/site-packages/ldap/controls.py", line 93, in
EncodeControlTuples
    result = [
  File "/usr/lib/python2.4/site-packages/ldap/controls.py", line 45, in
getEncodedTuple
    return
(self.controlType,self.criticality,self.encodeControlValue(self.controlValue))
  File "/usr/lib/python2.4/site-packages/ldap/controls.py", line 78, in
encodeControlValue
    return _ldap.encode_page_control(size,cookie)
TypeError: an integer is required

What steps will reproduce the problem?
1. set ldap_page_size 1000 
2.
3.

What is the expected output? What do you see instead?
Expected output is to see the app connect successfully but it doesn't it fails.

What version of the product are you using? On what operating system?
Redhat AS 5 
Copyright 2006, Google, Inc.
All Rights Reserved.
sync_ldap.py 0.9
Please provide any additional information below.

What steps will reproduce the problem?
1. reinstalled Google Apps Directory Sync for Email Security
2. Enter all configuration settings
3. At the Simulate Sync Screen everything is checked

What is the expected output? What do you see instead?
*  Click Simulate Sync and the screen never progresses to the the simulation 
applet.
*  If I click on Simulate Sync in the navigation pane the Simulate Sync button 
reloads and the Simulation Results button appears
*  If I click on the Simulation Results button I'm taken to the results screen 
and sync is in the Initializing... state

What version of the product are you using? On what operating system?
Windows Server 2003 Standard Edition, SP2

Please provide any additional information below.

What steps will reproduce the problem?
1. Mapping the GoogleQuota attribute to None
2. Use the command syncOneUser add that user
3.

What is the expected output? What do you see instead?

I see this output:

Command: syncOneUser (uid=ivan)
The user record for uid=ivan,ou=personas,dc=cyc,dc=ull,dc=es is now:
Now looking up 'ivan' in Google Apps for Your Domain...
User 'ivan' not found
The recommended action for this user is to
treat user as having been: added

Proceed with that action (y/n) y
03-16 19:30 root         ERROR    error: ProvisioningApiError: **Error 
on 'quota' argument: must be an integer
03-16 19:30 root         ERROR    failure to handle 'added' on 
uid=ivan,ou=personas,dc=cyc,dc=ull,dc=es: ProvisioningApiError: **Error 
on 'quota' argument: must be an integer

What version of the product are you using? On what operating system?

Google Apps LDAP Sync v1.2. Linux

Please provide any additional information below.

Hi.  We at the Open Planning Project are trying to use
google-apps-for-your-domain-ldap-sync to synchronize our ldap server to our
google apps account.  After the provided shell script failed, we noticed
that there was no way to specify TLS in the resulting GUI.  So we
downloaded
http://google-apps-for-your-domain-ldap-sync.googlecode.com/files/gafyd-ldap-syn
c-r100.tar.gz
as we read in the documentation that this was a supported option.  Firstly,
I was surprised to find that the tarball extracted to the current working
directory and not a subdirectory!  But not a big issue.

In our config, we set `tls_option = 'demand'`.  However, the running
`connect` gives 

{{{
CONFIDENTIALITY_REQUIRED: {'info': 'TLS confidentiality required', 'desc':
'Confidentiality required'}
01-07 14:55 root         ERROR    Connection failed
}}}

We were able to bypass this error by using the following patch:

{{{
$ diff -u ldap_ctxt.py.orig ldap_ctxt.py
--- ldap_ctxt.py.orig   2010-01-07 15:09:41.948808760 -0500
+++ ldap_ctxt.py    2010-01-07 15:09:49.252902412 -0500
@@ -143,6 +143,7 @@
       self._config.TestConfig(self, ['ldap_url'])
       self.conn = ldap.initialize(self.ldap_url)
       self.protocol_version = 3
+      self.conn.start_tls_s()
       self.conn.bind_s(self.ldap_admin_name, self.ldap_password,
         ldap.AUTH_SIMPLE)
       return None
}}}

Not sure why this works and the code above this in the python module does
not?  In my experience with python-ldap, I've always started TLS this way
and not with ldap.OPT_X_*

What steps will reproduce the problem?
1. Try using TLS on a machine.  It didn't work for us.
2. Not sure what else there is to try
3.

What is the expected output? What do you see instead?

Would expect a successful connect.  Instead I get the error:

{{{
CONFIDENTIALITY_REQUIRED: {'info': 'TLS confidentiality required', 'desc':
'Confidentiality required'}
01-07 14:55 root         ERROR    Connection failed
}}}

What version of the product are you using? On what operating system?

http://google-apps-for-your-domain-ldap-sync.googlecode.com/files/gafyd-ldap-syn
c-r100.tar.gz

{{{
$ uname -a
Linux worktop 2.6.31.9-174.fc12.x86_64 #1 SMP Mon Dec 21 05:33:33 UTC 2009
x86_64 x86_64 x86_64 GNU/Linux
$ cat /etc/fedora-release 
Fedora release 12 (Constantine)
}}}

Please provide any additional information below.

1. run sync_ldap.py
2. set primary_key sAMAccountName
3. check CFG file

Look for primary_key.  Its not there.  It should be.

Similar to issue #13.

Multivalued LDAP attributes are lists and most of the code which handles
LDAP attributes assumes single string values.

Most applications choose the first value as default if multivalued which is
reasonable behavior.


Example, a multivalued SN creates this issue on update:
Traceback (most recent call last):
  File "./sync_ldap.py", line 183, in ?
    (config, ldap_context, user_database, google_context, log_config)
= \
  File "./sync_ldap.py", line 112, in DoMain
    cmd.cmdloop()
  File "/usr/lib/python2.4/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.4/cmd.py", line 219, in onecmd
    return func(arg)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/commands.py",
line 293, in do_updateUsers
    attrlist=attrs)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/
ldap_ctxt.py", line 257, in Search
    return self.AsyncSearch(filter_arg, sizelimit, attrlist=attrlist)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/
ldap_ctxt.py", line 239, in AsyncSearch
    return userdb.UserDB(config=self._config, users=users)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/userdb.py",
line 304, in __init__
    self._AddUsers(users)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/userdb.py",
line 1196, in _AddUsers
    self.db[dn] = self._MapUser(attrs)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/userdb.py",
line 1260, in _MapUser
    if ldap_user_xform.MeetsPrereqs(attrs):
  File "/root/temp/google-apps-for-your-domain-ldap-sync/
user_transformation_rule.py", line 83, in MeetsPrereqs
    self.Mapping(ldap)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/
user_transformation_rule.py", line 187, in Mapping
    mappings['%sCallback' % attr] = self._TransformAttr(attr, attrs)
  File "/root/temp/google-apps-for-your-domain-ldap-sync/
user_transformation_rule.py", line 162, in _TransformAttr
    r = eval('self.%s(attrs)' % callback_name)
  File "<string>", line 0, in ?
  File "/root/temp/google-apps-for-your-domain-ldap-sync/
user_transformation_rule.py", line 116, in GoogleFirstName
    return NameSplit(ldap['givenName'], ldap['sn'],
ldap['displayName'])[0]
  File "/root/temp/google-apps-for-your-domain-ldap-sync/
user_transformation_rule.py", line 56, in NameSplit
    surname_pieces = surname.split()
AttributeError: 'list' object has no attribute 'split' 

What steps will reproduce the problem?
1.Having a user with non-ascii characters in name or surname
2.Use the command syncOneUser add that user
3.

What is the expected output? What do you see instead?

I see this output:

Command: syncOneUser (uid=ivan)
The user record for uid=ivan,ou=personas,dc=cyc,dc=ull,dc=es is now:
Now looking up 'ivan' in Google Apps for Your Domain...
User 'ivan' not found
The recommended action for this user is to
treat user as having been: added

Proceed with that action (y/n) y
03-15 21:30 root         ERROR    error: ProvisioningApiError: 
**Error: : 'ascii' codec can't decode byte 0xc3 in position 2: ordinal 
not in range(128)
03-15 21:30 root         ERROR    failure to handle 'added' on 
uid=ivan,ou=personas,dc=cyc,dc=ull,dc=es: ProvisioningApiError: 
**Error: : 'ascii' codec can't decode byte 0xc3 in position 2: ordinal 
not in range(128)

and only if I'm using a XML file then I can see the next output before 
exit:

 Command: stop
Stopping
Update your config file isaatc_ull_es.cfg (y/n)n
Traceback (most recent call last):
  File "/usr/local/share/gafyd-ldap-sync/sync_ldap.py", line 178, in ?
    (config, ldap_context, user_database, google_context, log_config) = \
  File "/usr/local/share/gafyd-ldap-sync/sync_ldap.py", line 147, in 
DoMain
    user_database.WriteDataFile(options.data_file)
  File "/usr/local/share/gafyd-ldap-sync/userdb.py", line 612, in 
WriteDataFile
    self._WriteXMLFile(fname, dns)
  File "/usr/local/share/gafyd-ldap-sync/userdb.py", line 1104, in 
_WriteXMLFile
    f.write(doc.toprettyxml(encoding="utf-8"))
  File "/usr/lib/python2.4/codecs.py", line 501, in write
    return self.writer.write(data)
  File "/usr/lib/python2.4/codecs.py", line 178, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 198: 
ordinal not in range(128)

What version of the product are you using? On what operating system?

Google Apps LDAP Sync v1.2. Linux

Please provide any additional information below.

The expception ProvisioningApiError is raised in _ToXML() method, inside 
provisioning_backend.py (Google Apps Provisioning API 1.0). I do some 
test to the API trying to insolate the problem.

I saw the same error after invoke the next sentence:

api.CreateAccountWithEmail("tést", "test", "test", "test")

but the next sentence works fine:

api.CreateAccountWithEmail(u"tést", "test", "test", "test")

It happens because the API must convert the arguments to unicode (using 
unicode()) to compose the XML that must be sended in the request to 
Google. For arguments that already are unicode strings (see the first 
argument y the second sentence) nothing happens. But for arguments that 
are byte strings with non-ascii characters an exception is raised.

A hack would be to set the default encoding in site.py to utf-8. But many 
people say that it is a bad trick. The good solution could be only use 
unicode strings inside the "application", invoke Google Apps Provisioning 
API using that unicode strings and only convert them to byte strings 
(with str()) before and after I/O. Many other packages (like 
LDAPUserFolder for Zope) do that to solve all the problems with the 
enconding.

1. Delete a user from Active Directory
2. Run sync.  It reports the user is exited successfully.
3. Run sync again.  It reports the user is exited successfully.

It should never retry a user who has been deleted (e.g. is no longer in the
filter criteria).

Querying a directory containing more than 1000 users truncates beginning at
the 1001st user.

Need 
1. parameters to specify both a CA crt file or CA cert directory
2. parameter to specify whether requiring, accepting or disallowing TLS

What steps will reproduce the problem?
1. Having a user with nonascii characters in lastName.

2. Command: syncOneUser uid=usuario

3.

What is the expected output? What do you see instead?

It was spected:

The user record for uid=ilegon,ou=people,dc=desic-sl,dc=com is now:
Now looking up 'ilegon' in Google Apps for Your Domain...
Google Apps for Your Domain returned the following data:
userName                      : usuario
firstName                     : Manuel
lastName                      : Legón
accountStatus                 : unlocked
emailLists                    : 
aliases                       :
Google Apps for Your Domain matches your database, and
your database matches LDAP. No action needed.

I see this, instead:

Command: syncOneUser uid=usuario
The user record for uid=usuario,ou=people,dc=xxxx,dc=xxx is now:
Now looking up 'ilegon' in Google Apps for Your Domain...
Google Apps for Your Domain returned the following data:
userName                      : usuario
firstName                     : Manuel
Traceback (most recent call last):
  File "sync_ldap.py", line 178, in ?
    (config, ldap_context, user_database, google_context, log_config) = \
  File "sync_ldap.py", line 107, in DoMain
    cmd.cmdloop()
  File "/usr/lib/python2.4/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.4/cmd.py", line 219, in onecmd
    return func(arg)
  File "/root/ldap-sync/google-apps-for-your-domain-ldap-sync/commands.py",
line 623, in do_syncOneUser
    user_rec = self._FetchOneUser(username)
  File "/root/ldap-sync/google-apps-for-your-domain-ldap-sync/commands.py",
line 916, in _FetchOneUser
    self._PrintGoogleUserRec(user_rec)
  File "/root/ldap-sync/google-apps-for-your-domain-ldap-sync/commands.py",
line 1026, in _PrintGoogleUserRec
    print '%-30s: %s' % (str(key), str(val))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in
position 3: ordinal not in range(128)

What version of the product are you using? On what operating system?

Revision: 59. Linux.

Please provide any additional information below.

Changing command.py, line 1026 from:

      print '%-30s: %s' % (str(key), str(val))

to:

      print '%-30s: %s' % (str(key), val)

it works.

Scripts fail to operate when there are more than one email included in
ldap, i.e instead of a string '[email protected]' we have a list ['[email protected]',
'[email protected]']. Scrip tries to perform mail.find('@') , but "mail" is not a
string but list and there is no "find" for list type.

What steps will reproduce the problem?
1. Use the command attrList without specify some action.
2.
3.

What is the expected output? What do you see instead?

I see this ouput:

Copyright 2006, Google, Inc.
All Rights Reserved.
Setting logging level to 20
Command: attrList
Traceback (most recent call last):
  File "/usr/local/share/gafyd-ldap-sync/sync_ldap.py", line 178, in ?
    (config, ldap_context, user_database, google_context, log_config) = \
  File "/usr/local/share/gafyd-ldap-sync/sync_ldap.py", line 107, in 
DoMain
    cmd.cmdloop()
  File "/usr/lib/python2.4/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.4/cmd.py", line 219, in onecmd
    return func(arg)
  File "/usr/local/share/gafyd-ldap-sync/commands.py", line 448, in 
do_attrList
    if len(toks) < 2:
UnboundLocalError: local variable 'toks' referenced before assignment

What version of the product are you using? On what operating system?

Google Apps LDAP Sync v1.2. Linux

Please provide any additional information below.

I attach a patch that solve this minor problem.

What is the expected output? What do you see instead?

It should create one or more nicknames based upon the contents of an ldap
attribute

What steps will reproduce the problem?
1. Configure sync_ldap.py to operate against an AD domain 
2. Witness the following error



This is due to AD sending a referral after the results, and python-ldap
following those referrals by default.  The following patch resolves the
issue, although i'm not 100% the integration with the config code is correct.

diff -u orig/ldap_ctxt.py new/ldap_ctxt.py
--- orig/ldap_ctxt.py   2008-06-24 20:53:52.000000000 -0400
+++ new/ldap_ctxt.py    2008-08-19 18:44:26.000000000 -0400
@@ -62,6 +62,7 @@
                   'ldap_base_dn': messages.MSG_LDAP_BASE_DN,
                   'ldap_timeout': messages.MSG_LDAP_TIMEOUT,
                   'ldap_page_size': messages.MSG_LDAP_PAGE_SIZE,
+                  'ldap_follow_referrals': messages.MSG_LDAP_FOLLOW_REFERRALS,
                   'tls_option': messages.MSG_TLS_OPTION,
                   'tls_cacertdir': messages.MSG_TLS_CACERTDIR,
                   'tls_cacertfile': messages.MSG_TLS_CACERTFILE}
@@ -80,6 +81,7 @@
     self.ldap_timeout = TIMEOUT_SECS
     self.ldap_url = None
     self.ldap_page_size = 0
+    self.ldap_follow_referrals = 'false'
     self.tls_option = 'never'
     self.tls_cacertdir = '/etc/ssl/certs'
     self.tls_cacertfile = ''
@@ -91,6 +93,10 @@
     self._required_config = ['ldap_url', 'ldap_user_filter', 'ldap_base_dn']
     self.config_changed = False
     self.conn = None
+    if self.ldap_follow_referrals == 'true':
+      ldap.set_option(ldap.OPT_REFERRALS, 1)
+    else:
+      ldap.set_option(ldap.OPT_REFERRALS, 0)
     if self.tls_option == 'demand':
       ldap.set_option(ldap.OPT_X_TLS, ldap.OPT_X_TLS_DEMAND)
     elif self.tls_option == 'allow':
@@ -292,6 +298,8 @@
       raise RuntimeError('Not connected')

     self.conn.network_timeout = self.ldap_timeout
+    self.conn.bind_s(self.ldap_admin_name, self.ldap_password,
+        ldap.AUTH_SIMPLE)

     users = None
     try:
diff -u orig/messages.py new/messages.py
--- orig/messages.py    2007-12-14 18:38:38.000000000 -0500
+++ new/messages.py     2008-08-19 18:44:38.000000000 -0400
@@ -157,6 +157,8 @@

 MSG_LDAP_URL = "URL for your LDAP server."

+MSG_LDAP_FOLLOW_REFERRALS = "Set to 'true' if LDAP referrals should be
followed.  Defaults to false"
+
 MSG_LDAP_PAGE_SIZE = """Some ldap servers (like Active Directory) will return 
 only a limited number of responses to a query and require the ldap client 
 to request more on subsequent requests.  If you have such a server, set
this to

What steps will reproduce the problem?
1. download trunk (was r96) according to instructions at
http://code.google.com/p/google-apps-for-your-domain-ldap-sync/
2. run `python sync_ldap.py`

What is the expected output? What do you see instead?
Expected to see something like:
Copyright 2006, Google, Inc.
All Rights Reserved.
Command: 

instead:
$ python sync_ldap.py 
Traceback (most recent call last):
  File "sync_ldap.py", line 44, in <module>
    import sync_google
  File "/Users/joe/google-apps-for-your-domain-ldap-sync/sync_google.py",
line 35, in <module>
    import google_result_queue
ImportError: No module named google_result_queue

What version of the product are you using? On what operating system?
$ svn info
Path: .
URL: http://google-apps-for-your-domain-ldap-sync.googlecode.com/svn/trunk
Repository Root:
http://google-apps-for-your-domain-ldap-sync.googlecode.com/svn
Repository UUID: 3138a281-5f33-0410-9cd3-d9e6c97ba8da
Revision: 96
Node Kind: directory
Schedule: normal
Last Changed Author: RobertEscorcio
Last Changed Rev: 96
Last Changed Date: 2008-07-24 11:43:49 -0400 (Thu, 24 Jul 2008)

Running on OS X 10.5.4

What steps will reproduce the problem?
1. ./sync_ldap.py on a Fedora system.

What is the expected output?
Script should use the default installed Python interpreter (typically
/usr/bin/python).

What do you see instead?
"bash: ./sync_ldap.py: /usr/bin/python2.4: bad interpreter: No such file or
directory"

What version of the product are you using? On what operating system?
SVN revision 49, on Fedora 8 x86_64

Please provide any additional information below.
The versioned python isn't a universal convention. 

I might recommend checking for the version using something like:
#!/usr/bin/env python
import sys
if sys.version_info[:2] < (2, 4):
  print "need python 2.4"
  sys.exit(1)

 Thanks!

What steps will reproduce the problem?


What is the expected output? What do you see instead?
C:\google\gldap-sync>python sync_ldap.py
Traceback (most recent call last):
  File "sync_ldap.py", line 38, in <module>
    import ldap_ctxt
  File "C:\google\gldap-sync\ldap_ctxt.py", line 27, in <module>
    import ldap
  File "C:\google\python-ldap\Lib\ldap\__init__.py", line 22, in <module>
    from _ldap import *
ImportError: No module named _ldap

What version of the product are you using? On what operating system?
Using Windows Server 2003. I am using Python 2.6. Its installed under my
folder c:\python. I have installed all the google related tools under -
c:\google\gldap-sync
c:\google\python-api
c:\google\python-ldap

this is my PYTHONPATH =
C:\google\python-api;C:\google\python-ldap\Lib;C:\python\Lib;C:\python\libs;C:\g
oogle\python-ldap

Please provide any additional information below.