openmf / mojafos Goto Github PK

View Code? Open in Web Editor NEW

4.0 4.0 3.0 88 KB

A Deployable Package for Mifos/Fineract, Payment Hub EE, and Mojaloop (Mojafos)

License: Apache License 2.0

Shell 100.00%

fineract fintech mojaloop paymenthub

mojafos's People

Stargazers

Watchers

Forkers

vilol-04 shraddha761 abhinavcode13

mojafos's Issues

enable cleanly deleting cloud resources by removing application components and ingress

we need to destroy things very often in the reverse order in which they are installed. So for k3s or microk8s locally you can simply do a cleanup of the k8s server and then everything goes away BUT (and this is important) if you try to remove (say) an AWS EKS cluster that still has running apps and still has (especially) an active nginx and load balancer => it leaves an awful awful mess that takes a long time to clean up and leaves resources around that "cost money" !!!! So can I refer you top vNext beta mini-loop directory and the eks-setup tools in the vNext Beta that back out the apps and then back out the nginx and LB etc and then (and only then) do they remove infrastructure.

unable to login mifos environment

After deploying all the three applications I'm unable to loginto the mifos login page. When accessing the url http://1-communityapp.sandbox.fynarfin.io/, and provided the default credentials username: mifos password: password but it throws the error
"Couldn't connect to server. Make sure you are using correct settings."

Please help me how to loginto it, what credentials to use.

reduce memory footprint of Mojafos deployment

We want Mojafos to be simple and cheap to use over a good number of platforms to this end we want to reduce the memory footprint of he default deployment from the current (approx) 16GBs to as low as possible and ideally to 8GB.

If 8GB can be achieved then Mojafos can likely be run on a single Raspberry PI -4 with 8GB and though this might be a stretch goal it is one worth trying for.

Initial observation of deployment Mojafos (1xOHEE, 1xMifos and 1xPHEE instances) show that there are still some obvious things to target

reduce number of MySQL databases to 1 (down from 2 currently)
reduce number of kafka deployments to 1 (down from 2 currently)
reduce number of redis instances deployments to 1 (down from 3 currently)
reduce Java heap sizes (some are still Xms=1Gb)
potentially reduce monitoring (ElasticSearch and Kibana and make optional if this is possible and not going to affect function)

Note: the approach here is to avoid adding to the plethora of existing helm charts and values.yaml files already available in the Mifos Repos but instead use and adopt the existing configure.py facility from vNext to adjust the memory params and values.yaml files at deploy-time so that they are specific and limited to Mojafos requirements.

implement deployment of new mifos web-app instead of the community app.

Right now, mojafos deploys a Mifos X with the old community app. We need to make it so that it deploys the latest stable release of the web app.

Add instrumentation to Mojafos for memory,disk use deploy time taken etc

It would be really helpful for Mojafos users as wells as Mifos and Mojaloop vNext developers to see

deploy time taken for major Mojafos components
total deploy time i.e. how long from ./run.sh until all pods are in running state
memory used by each major component (Mifos,Mojaloop vNext, PHEE)
total memory used by Mojafos
total disk space used by Mojafos

Much of this instrumentation exists in functions in the bash scripts of vNext and can be readily adopted and tailored for use by Mojafos.

Update PHEE version in Mojafos to release 1.13.0

https://mifos.gitbook.io/docs/payment-hub-ee/release-notes/v1.13.0

remove interactive components of Mojafos

I think we need to eliminate the user interaction and Mojafos behaviour should be driven from the initial parameters

if you want to deploy or destroy vNext and 2 Mifos and PHEE we need to be able to do something like run.sh -m install -s ml_mf2_ph with the -s string determining the services and the counts to be made (see below for an outline). This is important as it means we can out this into CI/CD (circleCi) and do full system testing using Mojafos for vNext,Mifos and PHEE (which will be especially cool once we start running txns end-to end)
example parameter handling code

#!/bin/bash
# Default values
mode=""
ml=true
mf=true
ph=true
mf_count=1
# Usage function
usage() {
  echo "Usage: $0 -m [install|delete] [-s <string>] [-c <count>]"
  echo "Options:"
  echo "  -m: Mode (install or delete)"
  echo "  -s: Services to operate on (ml, mf, ph)"
  echo "  -c: Count for mf service (default: 1)"
  exit 1
}
# Parse options
while getopts ":m:s:c:" opt; do
  case $opt in
    m)
      mode=$OPTARG
      ;;
    s)
      ml=false
      mf=false
      ph=false
      services=($(echo $OPTARG | tr ',' ' '))
      for service in "${services[@]}"; do
        case $service in
          ml)
            ml=true
            ;;
          mf)
            mf=true
            ;;
          ph)
            ph=true
            ;;
          *)
            echo "Invalid service: $service"
            usage
            ;;
        esac
      done
      ;;
    c)
      mf_count=$OPTARG
      ;;
    \?)
      echo "Invalid option: -$OPTARG" >&2
      usage
      ;;
    :)
      echo "Option -$OPTARG requires an argument." >&2
      usage
      ;;
  esac
done
# Check if mode is provided
if [ -z "$mode" ]; then
  echo "Mode is required."
  usage
fi
# Perform actions based on mode
if [ "$mode" = "install" ]; then
  echo "Installing selected services:"
  $ml && echo "- Installing ml"
  $mf && echo "- Installing $mf_count mf"
  $ph && echo "- Installing ph"
elif [ "$mode" = "delete" ]; then
  echo "Deleting selected services:"
  $ml && echo "- Deleting ml"
  $mf && echo "- Deleting $mf_count mf"
  $ph && echo "- Deleting ph"
else
  echo "Invalid mode. Mode should be 'install' or 'delete'."
  usage
fi (edited)

[DMP 2024]: Mojafos Version 2 - DaaS providing deployable package of Mifos/Fineract, Mojaloop and Payment Hub EE DaaS

Ticket Contents

Description

Mifos & Fineract integrated with Mojaloop via Payment Hub EE provides a end to end open source architecture for a building a complete digital financial services solution including managing wallets and stores of value In Mifos and Fineract to orchestrating and initiating real-time payments via the Mojaloop APIs throught Payment Hub EE orchestration engine.

However, it takes significant time and effort to deploy and install each of these individual components which are comprised various microservices and libraries and dependencies. This presents a steep barrier for fintechs and financial institutions to evaluate and discover the potential of this powerful stack. This project aims to maintain and enhance this deployable package, Mojafos. A major focus in this second year will be storyboards to help an individual developer rapidly deploy this end to end environment. Each of the various individual solutions/projects have a number of different options available to ease deployment into the cloud including infrastructure as code, docker images, helm charts, terraform scripts, etc.

Recently Carnegie India published a white paper outlining a new approach for rapid global adoption of Digital Public Infrastructure (DPI) - DPI as a Service or DaaS model. Mojafos fits squarely into that deployment approach as it bundles together multiple DPGs into a deployable package that streamlines the deployment process.

Goals & Mid-Point Milestone

Goals

[1] [Update Mojafos to deploy latest version of Mojaloop]
[2] [Update Mojafos to deploy latest version of Fineract]
[3] [Update Mojafos to deploy latest version of Mifos X Web App ]
[4] [Update Mojafos to deploy latest version of Payment Hub EE]
[5] [Improved resource consumption of Mojafos Deployable Package
[6] [With upgraded versions and improved consumption of resources, Mojafos should be closer to production-quality]

Setup/Installation

https://github.com/openMF/mojafos/blob/master/README.md

Expected Outcome

In 2023, GSOC inten, Elijah Okello created a deployable package, Mojafos, that incorporates the various components of the end to end stack - Mifos/Fineract, Mojaloop, Payment Hub EE, Mifos Channel App (mobile wallet/mobile banking) Initial deployment would target Azure but a bonus goal would be to make it cloud-agnostic.

Expected outcome for the second version of Mojafos is to continue optimizing its deployability, upgrade to current stable versions of the DPGs contained, optimize resource consumption and extending the storyboard of use cases that can be supported. The initial version was only a proof of concept so the second version would be intended for production-level deployments, aligning this with the vision of DPI as a Service (DaaS)

Latest stable release of Fineract core banking system (1.9 or current)
Latest stable version of Mojaloop (vNext)
Latest stable release of Mifos X Web App (23.12 or later)
Latest upstream version of Payment Hub EE
Storyboard/Cookbooks for standard use cases in the form of BPMN diagrams.

P2P Transfer
Merchant payment via QR code
Merchant Request to Pay
G2P Payment into Mifos Mobile Wallet
P2G Payment from Account in Fineract
PISP Payment
Applicants are also welcome to inputting their own ideas on how to improve and enhance.

Acceptance Criteria

No response

Implementation Details

Helm Chart for Fineract deployment - https://github.com/fynarfin/fineract-env/tree/master/helm/fineractConnect your Github account

Helm Charts for Payment Hub EE deployment - https://github.com/openMF/ph-ee-env-labs/tree/master/helmConnect your Github account

Docker Compose for Mifos - (Fineract Back-End + Web App) - GitHub - openMF/mifos-x-containers: Quick Deployment tool for having a running, non persistent Mifos X environment for demonstration purpose

Docker Hub Image of Fineract - Docker

Docker

Fineract Technical Documentation - Fineract Platform Documentation

Miniloop - GitHub - mojaloop/mini-loop: Deployment utilities for Mojaloop

Payment Hub EE - Welcome - Mifos Payment Hub EE

Lab Environment Overview - Lab environment

Mockups/Wireframes

No response

Product Name

Mojafos

Organisation Name

The Mifos Initiative

Domain

Financial Inclusion

Tech Skills Needed

AWS, DevOps, Java, Microservices, Performance Improvement, Spring Boot

Mentor(s)

@elijah0kello
@tdaly61

Update readme to be clear on latest versions of subcomponents being used in build

The readme should be updated for releases to reference the versions being deployed of Fineract PH-EE and Mojaloop, unless it always pulls the latest from the master repo's in which case that should be stated.

Investigate making Mojafos work on arm64

vNext runs on arm64 just fine
MifosX looks like it just needs to specify the correct JVM version on docker build and the correct deploy time Postgres tags to enable it to run deployed from docker-compose or Kubernetes
PHEE needs investigation to see what needs to change to support ARM

Drive end to end transactions through PHEE->Mojaloop vNext -> Mifos (fineract)

Now that Mojafos is deploying all components correctly to a local Kubernetes cluster we want to drive transactions end-to-end and enable end users to do the same.

The steps anticipated to accomplish this are

Connect PHEE to Mojaloop vNext switch and vNext to Mifos
update to the latest PHEE version (see #18)
upload txn data to the bulk processing facility of PHEE

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

chore(deps): update helm release redis to v19

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

helmv3

src/mojafos/deployer/helm/infra/Chart.yaml

kafka 19.0.2

console 0.6.6

mongodb 13.3.1

mongo-express 3.1.1

redis 17.11.6

elasticsearch 19.9.2

mysql 9.4.5

Check this box to trigger a request for Renovate to run again on this repository

Error while deploying Mojaloop correctly

Problem

When I try to run the deployment using sudo ./run.sh -u $USER -m deploy -d true -a all -f 1 -e local. Mojaloop is not being deployed correctly. Here are the deployment logs:

============================
Infrastructure Deployed
============================

Deploying Mojaloop vNext application manifests
==> Creating namespace mojaloop 
namespace/mojaloop created
==> Namespace mojaloop created successfully.

mojaloop Repo exists deleting and re-cloning 
==> Repository cloned successfully.

Error: The specified folder does not exist.

Configuring Mojaloop Manifests 
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/fspiop-api-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/fspiop-api-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/account-lookup-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/account-lookup-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/quoting-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/quoting-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/settlements-api-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/settlements-api-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/settlements-api-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut/platform-configuration-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut/authorization-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/transfers-event-handler-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut/authentication-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/transfers-command-handler-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/transfers-command-handler-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/settlements-command-handler-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/settlements-command-handler-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/transfers-api-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/transfers-api-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/accounts-and-balances-coa-grpc-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/accounts-and-balances-coa-grpc-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: redis-master.infra.svc.cluster.localsed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/accounts-and-balances-coa-grpc-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/accounts-and-balances-builtin-ledger-grpc-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/settlements-event-handler-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/participants-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/participants-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: mongodb:\/\/root:[email protected]:27017\/sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/accounts-and-balances-builtin-ledger-grpc-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: redis-master.infra.svc.cluster.localsed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps/accounts-and-balances-builtin-ledger-grpc-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: http:\/\/mojafos-infra-elasticsearch.infra.svc.cluster.local:9200sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut/auditing-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut/auditing-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: http:\/\/mojafos-infra-elasticsearch.infra.svc.cluster.local:9200sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut/logging-svc-deployment.yaml: No such file or directory
Error updating the value.
New value: value: kafka.infra.svc.cluster.local:9092sed: can't read src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut/logging-svc-deployment.yaml: No such file or directory
Error updating the value.
Mojaloop Manifests were not edited successfully
Deploying files in /home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut
Directory '/home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/crosscut' not found.
Waiting for Mojaloop cross cutting concerns to come up
Proceeding ...
Deploying files in /home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps
Directory '/home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps' not found.
Deploying files in /home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/ttk
Directory '/home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/ttk' not found.
Deploying files in /home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps
Directory '/home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/apps' not found.
Deploying files in /home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/reporting
Directory '/home/ratnesh/Desktop/mifos/mojafos/src/mojafos/deployer/apps/mojaloop/packages/installer/manifests/reporting' not found.

============================
Mojaloop Deployed
============================

Additional Information

I started facing this error possibly after this commit.
I'm using Ubuntu 22.04.4

Screenshot

Update Mojafos to use Mojaloop vNext Beta

Mojafos currently uses an early build of vNext and updating to the stable and much more fully featured Beta that was released last year is a simple yet productive thing to do.

When Pedro and the vNext team provide an RC for Mojaloop vNext we should also move from Beta to RC.