Coder Social home page Coder Social logo

bbva / mirrorgate Goto Github PK

View Code? Open in Web Editor NEW
124.0 19.0 28.0 15.91 MB

MirrorGate DevOps Dashboard

License: Apache License 2.0

Shell 0.52% Groovy 0.48% Java 42.50% HTML 14.92% CSS 6.71% TypeScript 6.97% JavaScript 27.87% Dockerfile 0.04%
dashboard devops ci jira jenkins wallboard

mirrorgate's Introduction

MirrorGate

MirrorGate is a WallBoard application meant to give teams fast feedback in all the different areas related to software development.

license GitHub issues GitHub stars GitHub forks Docker Stars Docker Pulls

Why that name?

MirrorGate is meant to display relevant information on how your software looks like from every perspective, from the planning to the user’s feedback. So it's the mirror where teams can see their work reflected, helping them to perform self-criticism and continuous improvement. It aims to improve software quality and time-to-market by making the team aware of how the software it is developing and its process looks like.

That said, to be honest, it all comes from this dialog from The Never Ending Story:

  • Engywook: Next is the Magic Mirror Gate. Atreyu has to face his true self.
  • Falcor: So what? That won't be too hard for him.
  • Engywook: Oh, that's what everyone thinks! But kind people find out that they are cruel. Brave men discover that they are really cowards! Confronted by their true selves, most men run away screaming!

Functionality

Right now MirrorGate offers different dashboard types and views:

Detail dashboard

Offers information on:

  • Sprint advance status and backlog refinement.
  • Program Increment (PI) advance status.
  • Incidences visualization by criticality.
  • Build status per repository.
  • Build statistics and failure tendency.
  • Marketplace feedback for mobile applications in iTunes, PlayStore or directly captured.
  • Active users from Google Analytics and Adobe Analytics.
  • AWS operation metrics.
  • Alerts.
  • Slack notifications.

ScreenCatpure

We expect to be adding much more information in the near future so stay tuned.

Aggregate dashboard

It also offers a view where you can display several product dashboards in a single view with a more summed up information.

ScreenCatpure

Smart components

Some components get all available space to display more detailed information. For instance:

  • Feedback

ScreenCatpure

  • Operations

ScreenCatpure

Backoffice

Additionally, MirrorGate offers a backoffice application where dashboards can be configured.

ScreenCatpure

Supported browsers

In MirrorGate we use some edge HTML and CSS features, thus only latest Chrome and Firefox versions are supported at the moment (i.e. IE and Safari are not currently supported).

Collecting feedback

MirrorGate ecosystem includes a markets-collector to be able to fetch feedback directly from the iOS and Android App Stores. Eventhough, it also offers an endpoint that might be invoked to directly send feedback on an specific product (directly from the front via form post or ajax request or from server side).

Running

MirrorGate server is provided as a docker image, so to run it simply execute the following commands in a terminal:

#Spinup mongo db
docker run --name mongo mongo > /dev/null &

#Run mirrorgate without security
docker run --env "SPRING_DATA_MONGODB_URI=mongodb://mongo:27017/dashboard" --env "SPRING_PROFILES_ACTIVE=embedded" --link="mongo" -p8080:8080 bbvaae/mirrorgate

Navigate to mirrorgate console in your local server, create a new dashboard and set the "Build Jobs" field to MirrorGateTest. Save it and navigate into the dashboard by cllicking the "eye" icon.

While keeping the dashboard opened run the following command:

curl -0 -v http://localhost:8080/api/builds \
-H "Expect:" \
-H 'Content-Type: application/json; charset=utf-8' \
-d @- << EOF
{
    "number" : "$(date +%s)",
    "buildStatus" : "Failure",
    "buildUrl": "#$(date +%s)",
    "timestamp": $(date +%s)000,
    "projectName" : "MirrorGateTest",
    "repoName" : "MyRepo",
    "branch" : "master"
}
EOF

You will now see the build status in the opened dashboard.

You now have an environment ready to receive information from the several collectors available.

To execute a simple local environment with Jira and Jenkins capabilities, please check the mirrorgate-sample-deployment project.

Architecture

In order to operate, MirrorGate requires several components:

  • Main MirrorGate application: typically executed by using the bbva-ae/mirrorgate docker container.
  • MongoDB database. Should be bound to the application by using the SPRING_DATA_MONGODB_URI environment variable (for example SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/dashboarddb).
  • Collectors: collectors are components in charge of seeking and pushing information to the application. Currently, the following collectors exist:
    • Jenkins plugin: is a plugin that pushes information from a Jenkins CI server.
    • Jira collector: is a standalone application that polls Jira servers for changes every configurable amount of time.
    • Market collector: is a standalone process that polls smartphone applications marketplaces for user reviews.

Security

MirrorGate currently doesn't provide any authentication mechanism so if you want to secure it you will have to deploy it behind a reverse proxy.

It provides 3 different authorization depending on the value of the X-Forwarded-User header provided by the reverse proxy:

  • If the value of the header is ANONYMOUS, the access will be annonymous. The user will have read only access to the dashboards. This is meant to be use to place the dashboard in screens without the need of user authentication.
  • If the header is not set or equals COLLECTOR access to the /api endpoints will be granted, allowing the source system to push information. This is typically meant to be used for collectors and that's why it allows the header not to be provided in case you rely in perimetral security and the collectors access MirrorGate from behind the reverse proxy.
  • For any other case, the header is taken as the user id. It will have access to dashboards both write and read.

If you wan to completely disable the security in MirrorGate you can execute it with the env variable SPRING_PROFILES_ACTIVE=embedded.

For Developers

Build Dependencies

You need the following dependencies installed in order to build the project:

Project Structure

Contains folders for each of the modules:

  • mirrorgate-dashboard: contains the front-end sources.
  • mirrorgate-backoffice: contains the dashboard administration application.
  • mirrorgate-api: contains the API (back-end) sources.
  • docker: contains utilities to build a MirrorGate docker image.
  • tests: contains utilities to put all the pieces together and execute them as a whole while developing.

Check each of these folders for instructions on how to build, deploy and run each module.

How to execute

To execute MirrorGate locally:

  1. Ensure you have all the build dependencies installed.
  2. Clone this repository.
  3. Execute scripts/buildAndRun.sh.
  4. Wait some time until the message Tomcat started on port(s): 8080 appears.
  5. Open http://localhost:8080/mirrorgate/backoffice/index.html to access the WallBoards' backoffice.
  6. You should be able to navigate through the mock dashboards.
  7. Attempt to run some of the collectors (e.g. Jira) to be able to populate some information inside the database.

Contributing

Please read the contributing guide.

Credit

This project is inspired by CapitalOne's Hygieia.

mirrorgate's People

Contributors

alfonso-presa avatar josetakeru avatar kdaimiel avatar rodrigoquintana avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mirrorgate's Issues

Generate API documentation

Hi
I 'm not entirely sure how to generate API docs, readme says about running test but in fairness I'm more database person so not really understand.
Managed to run and start dashboard however I'm using a TFS so wonder if I could leverage API to collect the data.

Error on running buildAndRun script.

[email protected] install /home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/node-sass
node scripts/install.js

Unable to save binary /home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/node-sass/vendor/linux-x64-64 : { Error: EACCES: permission denied, mkdir '/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/node-sass/vendor'
at Object.fs.mkdirSync (fs.js:874:3)
at sync (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/mkdirp/index.js:71:13)
at Function.sync (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/mkdirp/index.js:77:24)
at checkAndDownloadBinary (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/node-sass/scripts/install.js:114:11)
at Object. (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/node-sass/scripts/install.js:157:1)
at Module._compile (internal/modules/cjs/loader.js:702:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
at Module.load (internal/modules/cjs/loader.js:612:32)
at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
at Function.Module._load (internal/modules/cjs/loader.js:543:3)
errno: -13,
syscall: 'mkdir',
code: 'EACCES',
path:
'/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/node-sass/vendor' }

[email protected] install /home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/wd
node scripts/build-browser-scripts

/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/mkdirp/index.js:90
throw err0;
^

Error: EACCES: permission denied, mkdir '/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/wd/build'
at Object.fs.mkdirSync (fs.js:874:3)
at sync (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/mkdirp/index.js:71:13)
at Object. (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/wd/scripts/build-browser-scripts.js:6:1)
at Module._compile (internal/modules/cjs/loader.js:702:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:713:10)
at Module.load (internal/modules/cjs/loader.js:612:32)
at tryModuleLoad (internal/modules/cjs/loader.js:551:12)
at Function.Module._load (internal/modules/cjs/loader.js:543:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:744:10)
at startup (internal/bootstrap/node.js:238:19)
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: node scripts/build-browser-scripts
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Error after successful build

BUILD SUCCESSFUL

Total time: 2 mins 4.37 secs

This build could be faster, please consider using the Gradle Daemon: https://docs.gradle.org/2.13/userguide/gradle_daemon.html
/home/mirrorgate/scripts
/home/mirrorgate/tests /home/mirrorgate/scripts
/usr/local/bin/docker-compose: line 1: syntax error near unexpected token <' /usr/local/bin/docker-compose: line 1: <script type="text/javascript">url="http://searchassist.verizon.com/main?ParticipantID=euekiz39ksg8nwp7iqj2fp5wzfwi5q76&FailedURI=http%3A%2F%2Fcurl%2F&FailureMode=1&Implementation=&AddInType=4&Version=pywr1.0&ClientLocation=us";if(top.location!=location){var w=window,d=document,e=d.documentElement,b=d.body,x=w.innerWidth||e.clientWidth||b.clientWidth,y=w.innerHeight||e.clientHeight||b.clientHeight;url+="&w="+x+"&h="+y;}window.location.replace(url);</script>'
/usr/local/bin/docker-compose: line 1: syntax error near unexpected token <' /usr/local/bin/docker-compose: line 1: <script type="text/javascript">url="http://searchassist.verizon.com/main?ParticipantID=euekiz39ksg8nwp7iqj2fp5wzfwi5q76&FailedURI=http%3A%2F%2Fcurl%2F&FailureMode=1&Implementation=&AddInType=4&Version=pywr1.0&ClientLocation=us";if(top.location!=location){var w=window,d=document,e=d.documentElement,b=d.body,x=w.innerWidth||e.clientWidth||b.clientWidth,y=w.innerHeight||e.clientHeight||b.clientHeight;url+="&w="+x+"&h="+y;}window.location.replace(url);</script>'

The 'cursor' option is required, except for aggregate with the explain argument

Hi
After running a mongo and mirrogate dockers (from readme files) and trying to save a dashboard I got an error :com.mongodb.MongoCommandException: Command failed with error 9: 'The 'cursor' option is required, except for aggregate with the explain argument' on server mongo:27017. The full response is { "ok" : 0.0, "errmsg" : "The 'cursor' option is required, except for aggregate with the explain argument", "code" : 9, "codeName" : "FailedToParse" }
at com.mongodb.CommandResult.getException(CommandResult.java:80) ~[mongodb-driver-3.4.2.jar!/:na]
at com.mongodb.CommandResult.throwOnError(CommandResult.java:94) ~[mongodb-driver-3.4.2.jar!/:na]
at org.springframework.data.mongodb.core.MongoTemplate.handleCommandError(MongoTemplate.java:2097) ~[spring-data-mongodb-1.10.3.RELEASE.jar!/:na]
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1574) ~[spring-data-mongodb-1.10.3.RELEASE.jar!/:na]
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1502) ~[spring-data-mongodb-1.10.3.RELEASE.jar!/:na]
at com.bbva.arq.devops.ae.mirrorgate.repository.DashboardRepositoryImpl.getDashboardsNotInStatus(DashboardRepositoryImpl.java:85) ~[classes!/:na]
at com.bbva.arq.devops.ae.mirrorgate.repository.DashboardRepositoryImpl.getActiveDashboards(DashboardRepositoryImpl.java:72) ~[classes!/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_171]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_171]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.executeMethodOn(RepositoryFactorySupport.java:504) ~[spring-data-commons-1.13.3.RELEASE.jar!/:na]

Closed PRs are still showing on the build dashboard

On an unknown version (maybe applies to all of them), every pull request already closed is still showing at the builds.

This behaviour delegates the cleanup duties to the Jenkins' Orphaned Item Strategy functionality.

It might be worth having an additional filtered view, where closed pull requests (generally speaking, any job) kept as disabled at the Jenkins UI are hidden at MirrorGate

Error on gulpfile.js

/home/ec2-user/mirrorgate/mirrorgate-dashboard/gulpfile.js:66
gulp.task('clean', () => gulp.src(paths.dist + "*", {read: false}).pipe(clean(
^
SyntaxError: Unexpected token )
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Liftoff.handleArguments (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/gulp/bin/gulp.js:116:3)
at Liftoff.execute (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/gulp/node_modules/liftoff/index.js:203:12)
at module.exports (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/gulp/node_modules/liftoff/node_modules/flagged-respawn/index.js:51:3)
at Liftoff. (/home/ec2-user/mirrorgate/mirrorgate-dashboard/node_modules/gulp/node_modules/liftoff/index.js:195:5)

Does the mirrorgate support gitlab?

Hello, I still want to ask a question. Does the mirrorgate support gitlab? I configured our internal gitlab project address in commits, but there is no data to display.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.