gluufederation / gluu-passport-testing

Gluu Passport Stage Tests

Gluu-Passport Blackbox Testing

Goal

• Automate tests for all the passport flows

How we do it

Creating a stage environment like:

Starting

Clone this reppo to your test server / CI server

Setup

Setting up provider (idp/op) droplet

@TODO

Setting up passport droplet

• Setup your SP static IP (you need a static ip reserved, preffered a floating_ip) and host name

  • In the project root:

    vim setup/setup.properties
    

  • Locate the line with ip=
  • Update with the static IP address
  • Locate the line with hostname=
  • Update with your passport hostname

• Edit setup/templates/passport-central-config.json to configure your passport SPs

• In case you want certificates (recommended):

  • run:

    certbot --apache --agree-tos --force-renewal -m [email protected] -n -d <passporthost>
    

    (replace passporthost for your passport host, i.e. test.gluu.org)
  • compress etc/letsecrypt

    tar -czvf etcletsencrypt.tar /etc/letsencrypt
    

    (so you will have etcletsencrypt.tar in your setup folder)

• On a fresh droplet, create folder test-install-data:

ssh <yourhost> mkdir /test-install-data

• Copy files from setup folder to test-install-data that you just created on your fresh droplet:

scp -r ./setup/* <your-host>:/test-install-data/.

• Login to your droplet and run the prepare.sh file:

cd /
chmod -R 755 test-install-data
cd /test-install/data
./prepare.sh

• Check if flag file freshsnap was created:

ls /root

Starting test server

Just restart (shutdown -r) and gluu-server will be installed w/ testing-data. Follow it on /test-data/gluu.log

Configuration

Use test.conf for configuration. You can pass any file by using option -c <file>.

Configuration details are as given below:

Env	Description
PASSPORT_HOST	Use to set Host for Passport Droplet
PASSPORT_IP	Use to set floating IP for Passport Droplet
LATEST_DEV_SNAPSHOT_ID	It is Dev Passport Snapshot Id from which you want to create droplet for Passport
LATEST_STABLE_SNAPSHOT_ID	It is Stable Passport Snapshot Id from which you want to create droplet for Passport
PROVIDER_HOST	Use to set Host for SAML IDP Provider Droplet
PROVIDER_IP	Use to set floating IP for SAML IDP Provider Droplet
PROVIDER_SNAPSHOT_ID	It is SAML IDP Provider Snapshot Id from which you want to create droplet for SAML IDP Provider
CLIENT_HOST	It is use to set Host for Request Party Client i.e. auth-tdd-client
API_CLIENT_ID	Providers' gluu-server Admin REST API client_id (used for automated user creation/deletion)
API_CLIENT_SECRET	Providers' gluu-server Admin REST API client_secret (used for automated user creation/deletion)

Setting up test suite

Install Xvfb sudo apt install xvfb

Install poetry if you don't have: curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

Install dependencies: poetry install

If you don't have, install Firefox: apt install firefox

Setup ENV on env.sh file

run poetry run env.sh

Options

You can run env.sh with skip options:

• -s: skip droplet creation
• -t: skip tests
• -c <file>: Pass configuration file

Test logs / artifacts

Test Server will collect logs through ssh (ensure you have authorized test server to connectPROVIDER_HOST and PASSPORT_HOST through ssh)

Logs (such as idp-process.log, passport.log, oxauth_script.log, etc) will be stored ar server_artifacts/provider and server_artifacts/passport so they can be analyzed when needed. (i.e. to dig in a failing test case).

Your CI may handle to fetch and/or publish artifacts from this folder.

TO BE

In the near future

This is a nice milestone...

gluu-passport-testing's Issues

Script to destroy droplets after tests

We gonna shutdown / destroy droplets after tests

https://developers.digitalocean.com/documentation/v2/

Force "refresh" SPs metadata on IDP

BEFORE we starts tests, we need IDP to re-fetch metadata from URL.

This can be done restarting gluu-server

Suggestion: gluu-serverd restart on init.d, AFTER passport gluu server is up and running.

Passport configuration not being installed on 4.3.0

On Gluu Server 4.3.0 , passport providers are not being installed. It may be a template issue.

To reproduce:

• Start a VM on ubuntu 18
• Clone project
• run gluu.sh
• check LDAP for passport configuration is setup properly

Changing gluu-server version is kind of exaustive

Version downloaded here:

gluu-passport-testing/setup/gluu.sh

Line 11 in b69e936

echo "deb https://repo.gluu.org/ubuntu/ bionic-devel main" > /etc/apt/sources.list.d/gluu-repo.list

Should be able to be easily changed.

For example right now we are testing with latest stable.

As gluu is downloaded on droplet startup, I'm not sure what's the best way.
It would be nice to do it automatically, but then the droplet will need to be waiting for some kind of external (jenkins) input somewhere between startup and gluu package download.

Today the way to do it is:

• Creating the test droplet from snapshot manually
• Setup accordingly
• Clean existing gluu install
• create new snapshot
• change snapshot Id on config file at jenkins server

We can also create 2 snapshots, 1 for latest dev, and 1 for latest stable, and run both tests.

What do you think about this @kdhttps ?

[build] Change pip to poetry

pip have been buggy

Poetry is better.

• add poetry
• add needed dependencies to poetry (pyproject.toml)
• remove requirements.txt

Test passport PR with latest stable gluu-server build.

NOW it's like this:

• we can test new builds

we need to add this feature:

• Test passport PR with latest stable build.

As a developer
I want to test my changes with last passport build
So I can ensure quality

Scenario: Raising PR
Given I have a new branch
When I raise PR
Then I want to run stage integration tests w/ last gluu-Then server build

So what we need to do is:

• create snapshot with latest stable gluu-server version
• on init, droplet should upgrade passport version inside chmod
• create trigger on passport test jenkins

Create Jenkins job setup section on README.md

Step by step jenkins job configuration.

Github Action (version bump) failing when no increment

There is no impact but the latest push, if there is no increment, will show an x (failed).

It's a known bug commitizen-tools/commitizen#280

And I already raised PR: commitizen-tools/commitizen#284

Script to create droplet from snapshot

We gonna create 3 droplets from snapshots on digital ocean.

https://developers.digitalocean.com/documentation/v2/

Fresh instalation with existing data

We should be able to do a fresh gluu-server instalation and use data from an existing database (containing tests database)

Figure out how to do it

Document test.conf

Create instructions on setting up test.conf on README.md (Setup test-suite section)

Certificates are not being renewed

Certificates from PROVIDER are not being renewed.

We need to create a script to do the following:

I understand it may run each 20 days for safety.

A cron job may be setup or a jenkins job using the script.

My suggestion:

• Create droplets using the helper we already have
• ssh to PROVIDER and renew certificates (saving new tarfile, check:

  gluu-passport-testing/setup/gluu.sh

  Line 38 in b69e936

  tar -zxvf /test-install-data/etcletsencrypt.tar -C /opt/gluu-server

  )
• Prepare droplet to take snapshot (uninstall gluu-server - it was installed automatically when you created droplet. And create "flag" file) like here

  gluu-passport-testing/setup/prepare.sh

  Line 8 in b69e936

  touch /root/freshsnap

• Stop droplet through rest api or pydoautomator
• take snapshot
• save new snapshot ID to configuration file or env (as option, and in option we may need to specify configuration file name if not default.

---

Manual tests: (dry run may be used for testing renewal without currently renewing)

• Renew with default config file option
• Renew with custom name config file option
• Renew with env variable

---

• Documentation on README.md

Add tag `psp-stage` to droplets created

So we can enhance security.

Add test-case for: External OP Login

Add test-case for external OP.

Deploy tests in gluu digital-ocean and document step by step

Right now tests are being runned in @christian-hawk digital ocean cloud.

Waiting till "destroy droplets script" be done (#21) so we can fully automate the cloud droplets with the lowest cost possible (create droplet -> run tests -> destroy droplets)

We need to have a digital ocean api token, and we will:

• create droplet and setup idp/op with test data
• snapshot idp/op
• create droplet with init scripts to install gluu and passport testing branch
• snapshot droplet with init scripts
• destroy droplets
• setup tests
• setup jenkins (temporary running in testing instance)
• run tests

Start Gluu Passport Server before IDP/OP fetch metadata from Gluu Passport Servers provider metadata uri.

As Gluu Passport Metadata will be recreated on every test (every new test = new gluu passport install), we need IDP to fetch metadata from URI. So IDP/OP Gluu server is configured to get SP (passport) metadata from URL.

Shibboleth's FileBackedHTTPMetadataResolver is not clear about how to configure refresh from URI interval. Actually according to docs, several variables impacts on interval.

Check docs What triggers the metadata refresh process?

Maybe SP/IDP droplet creation can automatically refresh on start, but after that will wait for 6/8h (maybe there is a workaround for this).

So IF SP/IDP server starts IDP before Gluu Passport Server is ready, we will have to 'wait 8 hours' to do test lol

droplets are destoyed after every test

Sometimes we need droplet not to be destroyed after every test, so we can inspect logs and data.

Load Test Data from JSON

Load test data from JSON files

Load existing vars from config file CLI option

Option on env.sh script, load existing vars from ENV.

We should load this existing vars from ENV.

Something like --load-from-env, other suggestions are welcome.

deprecated apache templates folder

Gluu-server versions >= 4.3.0 uses apache/https_gluu.conf instead of only https_gluu.conf

delete_droplets is not working

During test, outputs:

ALL TEST FINISHED.
=====================================================================
Deleting droplets...
finished with success!!!

But droplets are NOT destroyed on Digital Ocean

SSL Error checking created droplet status

When running env.sh -t, after creating droplet, getting status:

HTTPSConnectionPool(host='t1.techno24x7.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))
Server still not ready...

refactor sh for idps and tests for be pre-selected provider

Setup IDP / OP Server

Setup t3 for IDP / OP Servers

Add test cases for error handling for gluu-passport #170 #173

Add test cases for GluuFederation/gluu-passport#170 and GluuFederation/gluu-passport#173

Test demo client needs to register OIDC client on gluu passport server

In my understanding this needs to be done on test demo client.

It's already possible to do it with

oidc-register https://<gluupassportdomain> https://<demotestappdomain>

Output json file has indormation needed

[TEST CASE] Fetch Configuration from OxAuth

Check If passport fetches configuration correctly from oxAuth
(maybe we will need to check the logs to see if configuration fetch was successful)

Suggested feature acceptance:

Feature: Fetch Configuration
 Get full configuration from oxauth

Scenario: Successful get configuration

Given passport is not running
  And oxauth is running
When passport is started
  And I wait for the fetch configuration time
Then configuration should be correctly fetched

Install fresh gluu-server with existing OpenDJ data

Problem on import-ldif: java out of memory, heap space

• Increase droplet memory to 16GB / 6 core
• Increase heap space to 6G on opendj java.properties (import-ldif) -Xmx6g
• chown ldap:ldap /opt/opendj/db/userRoot/*.*
• /opt/opendj/bin/import-ldif -n userRoot -l exactdatabackup_date.ldif --offline

** DATA IMPORTED **

• Trying to use setup.proerties, found error:

root@localhost:/install/community-edition-setup# ./setup.py -c
Detected OS ubuntu 18

Installing Gluu Server...
Detected OS  :  ubuntu
Detected init:  systemd
Detected Apache:  2.4

Installing Gluu Server...

For more info see:
  /install/community-edition-setup/setup.log  
  /install/community-edition-setup/setup_error.log


hostname                                        t1.techno24x7.com
orgName                                                      Gluu
os                                                         ubuntu
city                                                    Sao Paulo
state                                                          SP
countryCode                                                    BR
Applications max ram                                         6473
Install oxAuth                                               True
Install oxTrust                                              True
Backends                                        wrends, couchbase
Java Type                                                     jre
Install Apache 2 web server                                  True
Install Fido2 Server                                        False
Install Scim Server                                         False
Install Shibboleth SAML IDP                                 False
Install oxAuth RP                                           False
Install Passport                                             True
Install Casa                                                False
Install Oxd                                                 False
Install Gluu Radius                                         False


Proceed with these values [Y|n] y
Installing [#########################        ] Installing Gluu components: Couchbase   ***** Error caught in main loop *****
Traceback (most recent call last):
  File "./setup.py", line 5562, in do_installation
    self.install_gluu_components()
  File "./setup.py", line 2604, in install_gluu_components
    self.install_couchbase_server()
  File "./setup.py", line 4870, in install_couchbase_server
    self.create_couchbase_buckets()
  File "./setup.py", line 4793, in create_couchbase_buckets
    couchbaseClusterRamsize = (system_info['storageTotals']['ram']['quotaTotal'] - system_info['storageTotals']['ram']['quotaUsed']) / (1024*1024)
KeyError: 'storageTotals'



 Gluu Server installation successful! Point your browser to https://t1.techno24x7.com

• Open issue - talk to mustafa

• Workaround: comment couchbase installation on setup.py

• Trying to login as admin:
  PROBLEM when trying to login:

==> /opt/gluu-server/opt/gluu/jetty/oxauth/logs/oxauth.log <==
2020-08-05 20:19:25,642 ERROR [qtp296347592-20] [org.gluu.oxauth.service.ClientService] (ClientService.java:104) - javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
org.gluu.util.security.StringEncrypter$EncryptionException: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.

More Logs / Artifacts needed

We need to have logs from IDP (aka `PROVIDER'), from passport's instance, and test server (jenkins instance).

workaround: create droplet snapshot for other versions (4.2.1)

As we still have some development to be done till a more automated pipeline, and we need urgency for testing 4.2.1, what we gonna do for now is:

Create droplet similar to 4.2.0 snapshot, but change the script to install 4.2.1 instead of 4.2, and save this droplet snapshot w/ 4.2.1 information.

droplets are not being deleted when tests break

This is happening because we do not have a teardown function.