zgzgorg / iam-backend

IAM Backend

An Identity Access Management (IAM) system using Google Workspace accounts.

Intended Users. This system is intended for use at Community Builder Toolbox, Inc., a California-based 501(c)(3) non-profit organization. It is currently incubated under ZaiGeZaiGu, a volunteer platform for Chinese in the SF Bay Area.

Installation

Set up the environment

Assuming you have GitHub CLI installed (possibly via brew install gh) and uses Conda as your environment manager, execute the following commands:

git clone zgzgorg/iam-backend # Clone the repo.
cd iam-backend
conda create -n zgiam python=3.8
conda activate zgiam
make develop

Bootstrap the database

You can skip this section if you're provided with a zgiam.sql file.

1. Run make update-schema. This will create an empty SQLite file at zgiam/zgiam.sql.

2. Open this file with a SQLite editor of your choice. We recommend DBeaver, which you can install via brew install --cask dbeaver-community (assuming you have Homebrew installed).

3. Insert a row to the table account. Provide the following required fields:

   1. email -- You must be able to receive emails via this email address.
   2. first_name
   3. last_name
   4. phone_number

4. Save and exit.

Run the server

Someone should've sent you an iam_sqlite.cfg. Place it under the repo's directory. Run:

IAM_CONFIG_PATH=$PWD/iam_sqlite.cfg python zgiam/app.py

Now, go to http://127.0.0.1:5000/api/v1/. You should see a page similar to this screenshot.

Configuration

Using a file. By default, iam-backend reads /etc/zgiam/zgiam.cfg for configs. The file supports a dialect of the INI file structure defined by the Python 3 standard library configparser. A sample zgiam.cfg file can be found at zgiam/conf/default_iam.cfg. You can override the default path via the environment variable IAM_CONFIG_PATH.

Using environment variables. All variables in this file are also overridable via environment variables. The overriding environment variable should follow the format of IAM_{section}_{option}. For example:

• IAM_CORE_DEBUG
• IAM_DATABASE_TYPE
• IAM_DATABASE_FILE_PATH
• IAM_DATABASE_HOST
• IAM_DATABASE_PORT
• IAM_DATABASE_USER
• IAM_DATABASE_PASSWORD
• IAM_DATABASE_DBNAME
• IAM_DATABASE_SQLALCHEMY_TRACK_MODIFICATIONS
• IAM_LOGGING_CONFIG_PATH

Styles, Conventions, and Standards

This repo adheres to the following practices:

• Semantic Versioning 2.0.0.
• Conversational Commits: A specification for adding human and machine readable meaning to commit messages. Configured with .commitlintrc.yml.
• Black: The uncompromising code formatter for Python. Takes priority over PEP 8.
• PEP 8: Style Guide for Python Code.
  • This repo uses both pycodestyle and flake8 to enforce PEP 8. They have each other's back.
• Python code should be typed. This repo uses both mypy (by Python makers) and pytype (by Google) as type checkers. They have each other's back.

Further, this repo uses these dev-cycle tools:

• A requirements file defines dependencies that are parsable to pip. pip-tools reads the .in files and generates pip-friendly requirements.txt.

• makefile defines most of the dev-cycle actions.

  • Many of these actions are automatically triggered with pre-commit hooks.

• pytest is for unit tests.

  • pytest-cov generates the .coverage file. It computes coverage from pytest unit tests.

• Codacy checks code quality and keep track of technical debt. It integrates well into GitHub reviews.

• Pylint is another Python code analyzer.

• Dependabot makes security updates.

• CodeQL (by GitHub) and LGTM (by Semmle) discover vulnerabilities. They have each other's back.

Structure

This repo uses these modules:

• Alembic is a lightweight database migration tool for usage with the SQLAlchemy Database Toolkit for Python.
• google-auth is the Google authentication library for Python.
• Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX.
• SQLAlchemy is a Python SQL toolkit.
• Blinker provides fast & simple object-to-object and broadcast signaling for Python objects.
• Flask is a web framework for Python. Flask depends on the Jinja template engine and the Werkzeug WSGI toolkit. We use these plugins of Flask:
  • Flask-RESTX enables one to define API endpoints via a class-method structure.
  • Flask-SQLAlchemy.
  • Flask-JWT-Extended.
  • Flask-Dance for OAuth.
  • Flask-Login provides user session management for Flask.

iam-backend's Issues

design group with google workspace group

Since we use google workspace(Gsuite) to manage our group mostly,
and we have the groups concept in our database in #3, we should be able to link the google group with.

Outcome:
design the process and document the flow with the ACL process

user login API

user needs API to login using a Google account.

Maybe we can use SSO, or some Google API to achieve this.

Design database initial schema

The schema may need to handle existing data in the google sheet and need to be able to have a simple ACL such as group. That can make user belong to which group(s)

apply lgtm and try

When I implement some CI/CD, I realized we may be able to use lgtm for our code analysis.
Let's tried to include that into our CI pipeline and see how it goes

implement group concept for account

As a user, may be a group admin, member, or manager.
We should define some sets of group roles.

1. a user can be a group A admin that should be able to do operations for the group, such as approval of a user who wants to join group A
2. a user can be an organization admin, that can be able to change declined user to approve and regroup to pending approval with the right group, and approve new user too.

Outcome:

1. design a way to know what group and role of the user with some API result
2. change the schema to include the group role concept
3. the group role should be as same as google workspace groups.

Implement Alembic for database schema version control

Since database schema is very important for the whole application, we want to have a record to control our schema

Let's implement the init version for schema control using Alembic.

Outcome:

1. be able to have schema version record in our code base,
2. be able to create migration easier (over Makefile)?
3. be able to apply the migration for the schema update (maybe need to consider backup method too?)

Implement user update API

after the read feature(#5) is done, we need to have API that can update the user information.
We need 2 API for this.

1. user can update user-self information.
2. an admin can update the user has been approval or not (again, don't worry about the ACL at the moment)
   after the user has been approved, we should have couple action happen here.
   1. generate a ZgID here
   2. create a Google workspace account here
   3. send out the welcome email to the user

Unit test if possible

Apply CodeQL and try

When I implement some CI/CD, I realized we may be able to use CodeQL for our code analysis.
Let's tried to include that into our CI pipeline and see how it goes

apply codeclimate and try

When I implement some CI/CD, I realized we may be able to use code climate for our code analysis.
Let's tried to include that into our CI pipeline and see how it goes

Implement app config

We should have a config file for applications include environment, config file, and default method.
The config file should be able to config the database, logging format and some Flask config.

Outcome:

1. be able to have some default config value
2. be able to read config file from default config path or environment variables.
3. be able to config logging format, default logging format applied

api docs description may be redundant with function docstring

Currently, some of the code is written like below:

    @api.doc(
        description="XXX",
    )  
    def post(self) -> None:
        """XXX"""
       ...

We may be able to take out the description keyword to simply this.
Currently, some of the code is written like below:

    @api.doc()  
    def post(self) -> None:
        """XXX"""
       ...

Implement user can read user infomation

After issue #4 is done, we should have another API be able to read user information.
But we also need to consider PII.
However, this is not goal for this issue, but we need 2 API for now

Outcome:

1. a API for admin user can see the full list of user information. (don't worry about admin user now, we will do ACL define who can use this API later
2. normal users can see user-self information

If possible, do some unit tests for it.

implement automation generate documentation

We should have automation generate documentation
Our docstring style follows google style.

Outcome:

1. generate docs file by make docs
2. make sure it can read google style docstring
3. can choose Markdown, HTML format
4. research some tools such as Sphinx, sphinx-markdown-builder, and recommonmark

Reference: https://github.com/kis87988/Python_Google_docstring/

change approved account only allow admin to do it

This may be a bug need to fix, the current code everyone can approve a newly registered account.
Fix it to only allow core admins can approve newly registered accounts.

implement account not approval list and declined list

As an admin user, need to be able to check who is not approved yet, and who is being declined

This may depend on the group API that needs group admin permission, but this is not the scope for this issue.(#54)

Outcome:

1. design an API that can show the not approval list for the account
2. design an API that can show declined list for the account

implement send welcome email after approving an account

As a user, after an admin approve the account register, the user should get a welcome email

The email can be a template of an HTML file or some other solution.

Outcome:

1. be able to send an email to a user
2. be able to have template format for the email format

rename module

I am still considering what is a good name for this module.

The module may be used for other org as well

Implement user register API

This API main goal is define API style and swag docs implementation
Outcome:

1. user can call this API to register a new user, but waiting for approval. This means this API will create a user under the user table.
2. API will extend read, update, and (delete) in the future
3. API Swag documentation contains code

If possible, do a unit test too. If not, create another issue to follow up unit test

change commitlintrc.yml to ts or js syntax for better support

We use commitlint for commit check. However, it looks like we don't have full config support for the yml config file.
I start looking into this due to dependabot/dependabot-core#2445 and trying to have ignores, but that looks like we need to create a function to do that, but I don't believe yml support that.

Since the origin commitlint is created by javascript, we can use js or ts(TypeScript) for better support.

Reference:

1. https://github.com/conventional-changelog/commitlint/blob/master/docs/reference-configuration.md

Outcome:

1. determinate we should use ts or js
2. convert current commitlintrc.yml to commitlintrc.ts or commitlintrc.js

Initial the project

This issue created after the PR close, keep for documentation

We should have an initial structure for IAM backend project.
The initial will contain some linter, GitHub, test, and githook config.

Outcome:

1. GitHub config workflow to run CI
2. linter config contains python, python typing, shell
3. githook for pre-commit, commit-message, and pre-push
4. Python requirements control

Implement init database schema

When issue #2 is done, we should be able to implement it.

Outcome:

1. implement database session be a module that we may be able to change back with Flask-Sqlalchemy or pure Sqlalchemy
2. define database model file that can be used for API later
3. define the database schema management using alembic

dependabot.yml use wrong ecosystem

config file set wrong package-ecosystem to npm

Let's make sure it works with pip-compile.
And we may need to separate the requirements*.txt to a folder, but if we are going to put that into a folder, please update MANIFEST.in include that folder with *.txt too.

check model and migration are match

Inspired by https://github.com/4Catalyzer/alembic-autogen-check
If we can have a way to check the schema with migration is not match, we can notice developer forget to create a migration.

I am not sure how to implement this yet. Some ideas such as we may be able to just save a schema.sql into a file, and check all has been created. If not, fail the test.

If we cannot implement this feature, feel free to close this issue.

apply codecov and try

When I implement some CI/CD, I realized we may be able to use codecov for our code analysis.
Let's tried to include that into our CI pipeline and see how it goes

implement account decline api

As an admin user, need to be able to decline user(s).

This may require the #54 to complete because only the admin users can do this.

This issue main goal is to have an API to decline account

Outcome:

1. design an API that can decline an account or accounts if they are admin

fix the model user rename to account

We rename the model user to account, some part of the code doesn't rename fully.

And we should fix the alembic hook issue too

outcome:

1. rename all models
2. regenerate the init schema part
3. fix the hook for Black in alembic
4. fix the test as well

apply codacy and try

When I implement some CI/CD, I realized we may be able to use Codacy for our code analysis.
Let's tried to include that into our CI pipeline and see how it goes

implement admin can change account infomation

as admin, should be able to change user account information.

We can make the account/info patch method to take the argument as an option.
If the requester is an admin, they can change other account information.
Otherwise, the argument default is login user.

Outcome:

1. modify account/info patch method to achieve this goal

add cache to the ci to speed up

Currently, we need to do pip install every CI run in the GitHub workflow. We should be able to cache(https://github.com/marketplace/actions/cache) for speed up.

implement account approve api

As an admin user, need to be. able approve user if admin in a google's ​admin group or the group owner.
However, the scope will need to involve group ACL. That is not the major goal for this issue.
We will have a follow-up issue for this(#54)

This issue main goal is to have an API that can approve an account, and after approval be able to create a google workspace account

Outcome:

1. design an API that can approve an account if they are admin
2. after approving an account, create a google workspace account

implement groups list

As a user, we would like to know what groups are in this org.
We should have an API to show a list of groups

Outcome:

1. develop a API to show a list of group

design user register workflow

The workflow should be able to split couple part, we should have a design workflow diagram in this issue and document/explain for future review.

Outcome:

1. documentation of the workflow
2. drew simple flow chart