Coder Social home page Coder Social logo

seshat's Introduction

Seshat

A tool to extract metrics from agile tools such as Jira. It is heavilly influenced by the work of Troy Magennis (https://www.focusedobjective.com/)

Currently the following sources of information are supported:

  • Jira
  • Trello
  • CSV file
  • Clubhouse.io
  • Gitlab

Instalation

Requires Python 3.6 or later.

Install Python 3 and the pip package manager.

If you're using Windows, install Python from Anaconda distribution (https://www.anaconda.com/products/individual). This will install some essential packages that fail to install via the standard python distribution. No guarantees that it will work well in Windows!

Then run to install required modules:

 $ pip install -r requirements.txt   

Caveat: if you have both python and python3 install you will need to run pip3 instead of pip

Configuration

Two major configuration files exist, one for configuring logging (log_config.yml) and another for the app main configuration (config.yml) Also for each project/team a config file needs to be created and stored in the directory configured in config.yml.

Please check conf/ directory with several examples for each case:

  • Jira with oauth (server)
  • Jira with token (cloud)
  • Trello
  • Clubhouse.io
  • Gitlab
  • CSV file

config.yaml

This file provide basic needed information: config_dir: directory where configuration files for each project are stored cache_dir: the directory where cache files are stored (needs yet to be implemented) debug: run application in debug mode or not (applicable for the web interface). Could be set to true or false

log_config.yml

Change this file to your needs. By default it writes to /tmp/seshat.log

Project configuration files

Sample files for each type of project have been provided. You can just copy them and rename it to .yml adding your own data The application can support multiple files/projects. If you add several files, you will be able to select which one to be used in the "projects" menu. This menu will only appear if you have more than one *.yml file.

Jira

Jira passwords

One important note about jira passwords. Jira cloud version no longer support passwords and you need to create an API token instead and then copy&paste it to the password field. You can find more information about how to create API tokens here: https://confluence.atlassian.com/cloud/api-tokens-938839638.html

oauth

If using oauth authentication (more used for jira server) you will need to create your own certificates for the application and create an application link in you Jira instance. You can find more information about it here (Step 1): https://developer.atlassian.com/server/jira/platform/oauth/

To do the oauth dance and get the oauth token you can you can use the python jira-oauth packaget: https://pypi.org/project/jira-oauth/

Then use this to configure your access following the oauth example in conf/ directory

Trello

You need to get a trello API key and secret Check here for more information: https://trello.com/app-key

(in the future oauth support will be provided)

Clubhouse.io

You need to get a Clubouse API key

Gitlab

Only project issues are supported for now. You will need to get a Personal access token: https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html

Workflow

Configure here the workflow of your team/project. The Worklow provided in the example files are only examples and you will need to customized them to your needs.

To configure the workflow make sure that the name in the right is the same as the one in the left. For example

 To Do: To Do
 Done: Done

The reason for this is that I plan in the future to support mapping several states to the same step.

There is a "special" state called "Created". This one will show the date the card was created. In Jira, use this "Created" instead of the first status.

Usage

Starting

To run the application just type

 $python seshat.py 

(or just ./seshat.py in unix based OS).

After loading the data and starting with success, use your browser of choice and access http://localhost:8050/

Running in a server

To run the application in a server and access it from the outside, install a WSGI server (for example gunicorn) and don't forget to bind it to the public address.

For example:

 $ gunicorn --bind 0.0.0.0 seshat:server

Trivia

Why Seshat?

I'm not great with names in one hand and in another I would prefer to have something more interesting than "python agile metrics" or similar. So I looked for a name of a god that would make sense and found this in wikipedia:

 Seshat, under various spellings, was the ancient Egyptian goddess of wisdom, knowledge, and writing. 
 She was seen as a scribe and record keeper, and her name means she who scrivens (i.e. she who is the scribe), 
 and is credited with inventing writing.

seshat's People

Contributors

goncalovalverde avatar deepsourcebot avatar

Stargazers

 avatar

Watchers

 avatar  avatar

seshat's Issues

if ONLY ONE jira configuration file in conf dir the dashboard fails to load

IndexError: list index out of range

Traceback (most recent call last)
File "/home/oluapojuara/projects/repos/seshat/viewer/dash.py", line 402, in display_page
try:
team = int(request.cookies["team_metrics_idx"])
except BadRequestKeyError:
team = 0

    title = f"{self.app.title} : {self.projects[team].name}"

    if pathname:
        logging.debug("Changing page to " + pathname)
    if pathname == "/lead_cycle_time":
        return self.show_hist_dash(team), title

IndexError: list index out of range
Traceback (most recent call last):
File "/home/oluapojuara/projects/repos/seshat/viewer/dash.py", line 402, in display_page
title = f"{self.app.title} : {self.projects[team].name}"
IndexError: list index out of range


If we put two files for two jira projects it works! and menu dashboard we have the selection available to select the project we want to analyze

Throughput doesn't shows!

My team data doesn't use any sort of estimation...

output error bellow:

image


KeyError: 'Done'

Traceback (most recent call last)
File "/home/oluapojuara/seshat-master/viewer/dash.py", line 413, in display_page
return self.show_throughput_dash(team), title
File "/home/oluapojuara/seshat-master/viewer/dash.py", line 179, in show_throughput_dash
def show_throughput_dash(self, team=0):
tm = self.projects[team]
fig_throughput = tm.draw_throughput("all")
# TODO: improve this logic
if tm.has_story_points:
fig_velocity = tm.draw_velocity("Total")
fig_spoints_throughput = tm.draw_story_points()
else:
fig_velocity = {}
fig_spoints_throughput = {}

File "/home/oluapojuara/seshat-master/viewer/team_metrics.py", line 74, in draw_velocity
velocity = calculator.flow.velocity(self.cycle_data).resample("W").sum()
File "/home/oluapojuara/seshat-master/calculator/flow.py", line 38, in velocity
table = pd.pivot_table(
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/reshape/pivot.py", line 111, in pivot_table
grouped = data.groupby(keys, observed=observed)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 6515, in groupby
return DataFrameGroupBy(
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 525, in init
self.dropna = dropna

    if grouper is None:
        from pandas.core.groupby.grouper import get_grouper

        grouper, exclusions, obj = get_grouper(
            obj,
            keys,
            axis=axis,
            level=level,
            sort=sort,

File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/groupby/grouper.py", line 786, in get_grouper
raise KeyError(gpr)
KeyError: 'Done'
Traceback (most recent call last):
File "/home/oluapojuara/seshat-master/viewer/dash.py", line 413, in display_page
return self.show_throughput_dash(team), title
File "/home/oluapojuara/seshat-master/viewer/dash.py", line 179, in show_throughput_dash
fig_velocity = tm.draw_velocity("Total")
File "/home/oluapojuara/seshat-master/viewer/team_metrics.py", line 74, in draw_velocity
velocity = calculator.flow.velocity(self.cycle_data).resample("W").sum()
File "/home/oluapojuara/seshat-master/calculator/flow.py", line 38, in velocity
table = pd.pivot_table(
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/reshape/pivot.py", line 111, in pivot_table
grouped = data.groupby(keys, observed=observed)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/frame.py", line 6515, in groupby
return DataFrameGroupBy(
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 525, in init
grouper, exclusions, obj = get_grouper(
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/groupby/grouper.py", line 786, in get_grouper
raise KeyError(gpr)
KeyError: 'Done

File not found error, if CSV file configured!

2020-12-21 18:15:37,854 - root - DEBUG - - Reading data for CSV example
Traceback (most recent call last):
File "./seshat.py", line 24, in
data = reader.read_data(source_config)
File "/home/oluapojuara/seshat-master/reader/init.py", line 18, in read_data
cycle_data = reader.csv.read(config["csv"], config["Workflow"])
File "/home/oluapojuara/seshat-master/reader/csv.py", line 10, in read
cycle_data = pd.read_csv(csv_file)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/io/parsers.py", line 688, in read_csv
return _read(filepath_or_buffer, kwds)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/io/parsers.py", line 454, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/io/parsers.py", line 948, in init
self._make_engine(self.engine)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/io/parsers.py", line 1180, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/io/parsers.py", line 2010, in init
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 382, in pandas._libs.parsers.TextReader.cinit
File "pandas/_libs/parsers.pyx", line 674, in pandas._libs.parsers.TextReader._setup_parser_source
FileNotFoundError: [Errno 2] No such file or directory: 'safaricom_csv.csv'

oluapojuara@oluapojuara-Virtual-Machine:~/seshat-master/conf/Celfocus$ ll
total 36
drwxrwxr-x 2 oluapojuara oluapojuara 4096 dez 22 13:08 ./
drwxrwxr-x 3 oluapojuara oluapojuara 4096 dez 21 18:25 ../
-rw-rw-r-- 1 oluapojuara oluapojuara 16273 dez 21 13:05 safaricom_csv.csv
-rw-rw-r-- 1 oluapojuara oluapojuara 241 dez 21 19:34 safaricom_csv.yml-example

oluapojuara@oluapojuara-Virtual-Machine:~/seshat-master/conf/Celfocus$

Is this teh correct location?

ttributeError: 'RangeIndex' object has no attribute '_with_freq'

oluapojuara@oluapojuara-Virtual-Machine:~/seshat-master$ ./seshat.py
2020-12-22 13:25:14,903 - main - INFO - - Starting seshat. Let's do team magic!
2020-12-22 13:25:14,905 - root - ERROR - get_input - Reading input configuration from conf/Celfocus/celfocus_safaricom_jira_token.yml
2020-12-22 13:25:14,907 - root - DEBUG - - Reading data for Jira Token Safaricom Chama
2020-12-22 13:25:14,908 - root - INFO - init - Initializing Cache
2020-12-22 13:25:14,908 - root - DEBUG - get_data - Getting data from jira
2020-12-22 13:25:14,908 - root - DEBUG - get_data - Getting jira info cached
2020-12-22 13:25:14,908 - root - DEBUG - read - Reading from cache /tmp/seshat_cache_cd33f585cc4f9a6ee753e168601fa763.pkl
2020-12-22 13:25:14,910 - root - DEBUG - lead_time - Calculating lead time for Created
2020-12-22 13:25:14,916 - root - DEBUG - cycle_time - Calculating cycle time for start:Created and end:To Do
2020-12-22 13:25:14,917 - root - DEBUG - cycle_time - Calculating cycle time for start:To Do and end:In Progress
2020-12-22 13:25:14,919 - root - DEBUG - cycle_time - Calculating cycle time for start:In Progress and end:Waiting for Response
2020-12-22 13:25:14,920 - root - DEBUG - cycle_time - Calculating cycle time for start:Waiting for Response and end:Approval
2020-12-22 13:25:14,921 - root - DEBUG - cycle_time - Calculating cycle time for start:Approval and end:Done
Traceback (most recent call last):
File "./seshat.py", line 26, in
team_metrics = viewer.team_metrics.Team_Metrics(cycle_data, source_config)
File "/home/oluapojuara/seshat-master/viewer/team_metrics.py", line 22, in init
self.throughput = calculator.flow.throughput(self.cycle_data, self.end_column)
File "/home/oluapojuara/seshat-master/calculator/flow.py", line 32, in throughput
throughput = calculator.tools.group_by_date(cycle_data, end_column)
File "/home/oluapojuara/seshat-master/calculator/tools.py", line 15, in group_by_date
df = df.resample("D", on=index).sum()
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/resample.py", line 926, in f
return self._downsample(_method, min_count=min_count)
File "/home/oluapojuara/.local/lib/python3.8/site-packages/pandas/core/resample.py", line 1029, in _downsample
obj.index = obj.index._with_freq(self.freq)
AttributeError: 'RangeIndex' object has no attribute '_with_freq'

oluapojuara@oluapojuara-Virtual-Machine:~/seshat-master$

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.