inseefr / sugoi-api

Sugoi Rest Services

Sugoi: User management Api

Sugoi provides an API to manage users with multi tenancy in mind.

• Sugoi: User management Api
• Installation
• Configuration
• Contributing
• License

Sugoi is an API to manage users, organizations and applications. These objects are scoped to realms to isolate object with separated purposes (employees, clients, contacts...).

View more

A react frontend for this API can be found here : https://github.com/InseeFrLab/sugoi-ui.

A Keycloak storage extension is here : https://github.com/InseeFrLab/keycloak-http-storage-provider (Work in Progress)

Installation

Download and extract release zip, modify configuration file as needed and launch :

java -jar -Dspring.profiles.active=local sugoi-api.jar

Some other ways could be found in docs/install.md

Configuration

All configuration is done through an application.properties file a la spring-boot.

Configuration.md for details.

Contributing

Pull requests are welcome. We ask that all pull request are linked to an issue. The source code format should be in conformance with google guidestyles for java. this is checked for each PR, you can enforce it with mvn spotless:apply.

We also check that all commit are signed-off in accordance with DCO

All feature creation or update should be reflected in the documentation. All new configuration keys of the instance or realm should be documented.

Please make sure to update or create tests as appropriate.

License

Apache License, Version 2.0

sugoi-api's Issues

[JMS Store] Implement JMS store writer

We want to be able to push modification request to a jms queue for further processing.

No authentication on ldap

In https://github.com/InseeFrLab/sugoi-api/blob/main/sugoi-api-ldap-utils/src/main/java/fr/insee/sugoi/ldap/utils/LdapFactory.java
the user and password properties are not used on the connection, which prevents all update on ldap.

Fix inseeGroupDefaut

InseeGroupeDefaut is a ldap attribute on a User that can be read to define the right of a user on an application. InseeGroupeDefaut is described as [prop_]role_appli with prop optional. Users have a list of Habilitation which describe their inseeGroupeDefauts.
Two problems need to be fixed :

• only prop_role_appli can be read and not role_appli
• null Habilitation creates null_null_null inseeGroupeDefaut

Fix WriterStore interface

in WriterStore organization methods takes group argument instead of organization

Set Up CI

Retrieving and adding nested ressources when using LdapStores

[Tests] Add tests

• Unit test where possible.

• I think we also need some integration tests in a new maven module

[JMS Service] Implement JMS consumer

Should be able to work with ActiveMQ

Fix search with object mapper

Object mapper property should be a perfect match and not an * match. Search requests currently fail with openldap.

Tests errors in windows

When using windows with JDK 11.0.9 and building sugoi-api with mvn package, process exit with errors :

[ERROR] Caused by: org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was cmd.exe /X /C "C:\Users\USER\AppData\Local\Programs\jdk-11.0.9+11\bin\java -javaagent:C:\\Users\\USER\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.4\\org.jacoco.agent-0.8.4-runtime.jar=destfile=C:\\Users\\USER\\Documents\\dev\\sugoi-api\\sugoi-api-ldap-store-provider\\target\\jacoco.exec -jar C:\Users\USER\temp\surefire345198285959551297\surefirebooter11487559930431689607.jar C:\Users\USER\temp\surefire345198285959551297 2021-01-14T09-54-02_245-jvmRun1 surefire16528952380935067697tmp surefire_148524147527092363468tmp"
[ERROR] Process Exit Code: 0
[ERROR] Crashed tests:
[ERROR] fr.insee.sugoi.ldap.LdapWriterStoreTest
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:690)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter.access$600(ForkStarter.java:118)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:447)
[ERROR]         at org.apache.maven.plugin.surefire.booterclient.ForkStarter$2.call(ForkStarter.java:423)
[ERROR]         at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[ERROR]         at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[ERROR]         at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[ERROR]         at java.base/java.lang.Thread.run(Thread.java:834)

[Rest Services] Add rest endpoints

Add all needed endpoint to manage users and organizations.

[Stores] Complete interface

Add DAO operations about CRUD user/orga/groups/habs in ReaderStore and WriterStore interfaces

Complete LDAP Mappers

LDAP Mappers allow converting a java object to a ldap object and vice versa. There is a mapper for Address, Application, Group, Organization, Realm and User. For now only major or simple attributes such as the id is mapped. We need to test what the mapper can do and add missing mappings.

Add management role

We need to add some more management role :

• Manage an application -> have reader rights on a realm and management rights on the app (to add group or group members)
• Credential management -> reader rights + rights to update password for an user

[LDAP] Implement Reader and Writer

Implement CRUD methods in LDAP for user/orga/habs/groups

Add an event mecanism

We need to be able to execute actions in some case (Create User, Update Password, Delete User.....).

There could be several actions for the same events (send mail, add to log...).

Implements Rest endpoints

They return null at the moment

When 2 UserStorage have the same name Store beans are not overrided

When we execute
http://localhost:8080/tomcat1/v2/SSM/users?size=20&typeRecherche=et
and
http://localhost:8080/tomcat1/v2/domaine1/users?size=20&typeRecherche=et
we have the same results

Add support for search of type OR

Adding properties to configure LDAP UserStorage

For now the UserStorage configuration allows us to read an organization source and user source. We also want to choose the address source for all type of stores and for the ldap stores a user rdn, organization rdn, address rdn, appli rdn as well as a regex to retrieve the groups.
We suggest that a ldap UserStorage look like this, with ldap specific attributes in the properties map.

"userStorages": [
  {
    "name": "default",
    "userSource": "ou=contacts,ou=clients_domaine1,o=insee,c=fr",
    "organizationSource": "ou=organisations,ou=clients_domaine1,o=insee,c=fr",
    "addressSource": "ou=adresses,ou=clients_domaine1,o=insee,c=fr",
    "properties": {
      "userRDNBase": "uid",
      "organizationRDNBase": "uid",
      "groupRegex": "cn=*_%s",
      "addressRDNBase": "l",
      "appliRDNBase": "ou"
    },
    "readerType": null,
    "writerType": null
  }
]

[Store provider] Add Application methods in store interfaces

Implements JmsWriterStore and JmsListener

Swagger should not consider storage as required

It seems that Swagger won't consider a PathVariable as optional even if it has been specified (in @PathVariable or via swagger annotation @\Parameter(required=false))
Maybe we can write distinct endpoints ?

User storage model shouldn't be linked to ldap

Config should be a simple map, only generic attribute should appear.

[test] Setup cucumber test

Describe store interfaces

We have to precisely describe as javadoc the expected behavior of the methods defined in WriterStore and ReaderStore for the implementations to be consistent with one another.
It's for instance lacking of information on :

• the content of the objects returned (for example do we need to have all the user's group when we retrieve a user ?)
• informations on how should be treated requests on credentials
• the possibilities of the search methods (such as what the PageableResult is expected to be or what the typeRecherche is used for)
• what should be return when not found
• the exceptions that can be thrown

Implements all method in `sugoi-api-rest-old-service`

We need to be compatible with our old proprietary api.

Error ldap when updating user

Sugoi trie to update forbidden attributes like modifyTimestamp :

LDAPException(resultCode=19 (constraint violation), diagnosticMessage='modifyTimestamp: no user modification allowed', ldapSDKVersion=5.1.2, revision=420e41670eca6da60425a79cf400a49205397f61) 

Services implementations

in core module, fr.insee.sugoi.service package should contain the following service class :

• RealmService
• UserService
• OrganizationService
• ApplicationService

Those should implement all the operation needed on those object. At minimum :

• findById()
• findAll()
• create()
• update()
• delete()
• findByProperties(Map<String, String> properties)

We could also avoid using interface for those, as I do not see the need for another implementation.

Implement mail sender when password is reset

[LDAP] Select more generic objectclass and attributes to configure realms in ldap

Avoid using attributes 'inseePropriete" for instance

Authorization should be case insensitive

We should check Roles in a case insensitive way.

[Rest Services] Fix CORS configuration

Cors Configuration gives 403 with requests such as :

curl -H"Origin: nope.insee.fr" http://localhost:8080 -X OPTIONS -v

From the logs, this is because OPTIONS method is not allowed.

Add "see also" attribute to user model

should be an Uri towards some informations (multivalued)

Null Pointer Exception in decider when userStorage is null

In Preauthorize, if userStorage is null, we get a NPE in the decider.

[Ldap realm provider] Avoid ldap request everytime it is called

Maybe use caching

Fix update user fails because of address updating

LDAPException(resultCode=65 (object class violation), diagnosticMessage='Unable to modify entry 'l=testa,ou=adresses,ou=clients_domaine1,o=insee,c=fr' because the entry resulting from applying the modifications would have violated the provided schema: The entry contains abstract object class top that is not subclassed by any of the structural or auxiliary object classes included in the entry. The entry contains attribute l which is not allowed by its object classes and/or DIT content rule. The entry contains attribute inseeAdressePostaleCorrespondantLigne4 which is not allowed by its object classes and/or DIT content rule. The entry contains attribute inseeAdressePostaleCorrespondantLigne3 which is not allowed by its object classes and/or DIT content rule. The entry contains attribute inseeAdressePostaleCorrespondantLigne2 which is not allowed by its object classes and/or DIT content rule. The entry contains attribute inseeAdressePostaleCorrespondantLigne1 which is not allowed by its object classes and/or DIT content rule. The entry contains attribute inseeAdressePostaleCorrespondantLigne7 which is not allowed by its object classes and/or DIT content rule. The entry contains attribute inseeAdressePostaleCorrespondantLigne6 which is not allowed by its object classes and/or DIT content rule. The entry contains attribute inseeAdressePostaleCorrespondantLigne5 which is not allowed by its object classes and/or DIT content rule. The entry's RDN contains attribute l which is not allowed to be included in the entry.', ldapSDKVersion=5.1.2, revision=420e41670eca6da60425a79cf400a49205397f61)

Manage userstorage better

At the moment we expect to have only one user storage in a lot of place, this needs to be addressed.

We would need to manage a default usertorage by realm, maybe by an attribute "default" when there is more than one in a realm.

Implements credentials management

• change password : change the password when knowing the old one, and maybe verify some pasword policy (length...)
• reinit password : add a password to an user (regardless of the presence of an old one) with a flag to force the user to change it at next login (or not).

Fix full env distribution

Full env distribution doesn't work anymore since eec0390 (?).
Maybe caused by this :
java.nio.file.NoSuchFileException: /Users/cecile/Documents/Workdir/sugoi-api/sugoi-api-distribution/sugoi-api-distribution-full-env./../sugoi-api-distribution-war/target/sugoi-api.war
at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
at java.base/java.nio.file.Files.readAttributes(Files.java:1763)
at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1225)
at java.base/java.util.zip.ZipFile$CleanableResource.(ZipFile.java:727)
at java.base/java.util.zip.ZipFile$CleanableResource.get(ZipFile.java:844)
at java.base/java.util.zip.ZipFile.(ZipFile.java:247)
at java.base/java.util.zip.ZipFile.(ZipFile.java:177)
at java.base/java.util.jar.JarFile.(JarFile.java:348)
at java.base/java.util.jar.JarFile.(JarFile.java:319)
at java.base/java.util.jar.JarFile.(JarFile.java:285)
at org.apache.catalina.startup.Tomcat.getWebappConfigFileFromWar(Tomcat.java:1289)
at org.apache.catalina.startup.Tomcat.getWebappConfigFile(Tomcat.java:1269)
at org.apache.catalina.startup.Tomcat.addWebapp(Tomcat.java:776)
at org.apache.catalina.startup.Tomcat.addWebapp(Tomcat.java:738)
at org.apache.catalina.startup.Tomcat.addWebapp(Tomcat.java:249)
at fr.insee.sugoi.app.service.TomcatEmbeddedService.launchTomcat(TomcatEmbeddedService.java:91)
at fr.insee.sugoi.app.service.TomcatEmbeddedService.start(TomcatEmbeddedService.java:177)
at fr.insee.sugoi.app.SugoiTestService.lambda$startServers$2(SugoiTestService.java:121)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

[StoreProvider] How to get the beans

Wa may need a better way to get the store beans than by String.

Add write methods in RealmProvider interface

If we want to be able to modify the configuration of our service via the application we need to implement create, delete and update realm in RealmProvider interface and implementations.

The war distribution doesn't load the custom property file

When using the war distribution in tomcat and trying to load the configuration from $catalina.base/webapps/sugoi.properties, the file is not loaded.

Use spring boot actuator

https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-features.html

Swagger should be better documented

All controller methods should have a description.

Create documentation website

We need some documentation :

• for users who want to use sugoi (installation, configuration, usage)
• for developpers/contributors

We need it to be multilanguage/multiversions

We could use docsify like here to manage documentation in repos but also get only one website (like inseefrlab/sugoi to manage releases and docs).

Realm ldap configuration is not case insensitive

For instance, it expects inseepropriete in lower case.

Fix search with object mapper attribute

Object mapper property should be a perfect match and not an * match. Search requests currently fail with openldap.

Refactor dependency management

All dependencies should be declared in the parent to be used in children (either by BOM for spring boot, or directly in dependency-management).

Implement other methods on file store provider