Coder Social home page Coder Social logo

fiware-management-scripts's Introduction

FIWARE HelpDesk and Desks reminders script

License badge

This script was developed in order to facilitate the diary operations of the Jira and the synchronization with different tools. The Purpose is to check if there are issues open and send reminders to the owners in order to resolve them and synchronize Jira issues with the data from Askbot and StackOverFlow.

These scripts were originally developed by Manuel Escriche from Telefónica I+D and now is maintained by me.

Top

Build and Install

Requirements

The following software must be installed:

  • Python 3.9
  • pip
  • virtualenv

Installation

The recommended installation method is using a virtualenv. Actually, the installation process is only about the python dependencies, because the python code do not need installation.

  1. Clone this repository.
  2. Define the configuration file: ./Config/management.ini
  3. Create the virtualenv: virtualenv -ppython3.9 env
  4. Activate the python environment: source ./env/bin/activate
  5. Install the requirements: `pip install -r requirements.txt

By default, the service will take the content of the Environment variable CONFIG_FILE. If it is not specified, the service will take the values by default from the local configuration file file.

Top

Configuration

The scripts are searching the configuration parameters or in the '/etc/fiware.d' directory or in the environment variables. It tries to find if there is defined an environment variable whose name is 'CONFIG_FILE' to the 'management.ini' file. If the scripts cannot get this environment variable, it tries to find this file in '/etc/init.d'. In any other case, the scripts will give you an error.

One possible solution might be to provide a soft link in the /etc/fiware.d to the management.ini file in the corresponding ./Config folder, excuting the command:

ln -s /home/ubuntu/fiware-management-scripts/Config/management.ini management.ini

Last but not least, it is possible to create a cronjob to automatically execute the tests, just execute the following commands:

echo "# FIWARE Management Script" | crontab -
( crontab -l ; echo "00 2 * * MON /home/ubuntu/fiware-management-scripts/management.py -a Tech 2>/dev/null" ) | crontab -
( crontab -l ; echo "30 2 * * MON /home/ubuntu/fiware-management-scripts/management.py -a Lab 2>/dev/null" ) | crontab - 
( crontab -l ; echo "00 3 * * MON /home/ubuntu/fiware-management-scripts/management.py -a Other 2>/dev/null" ) | crontab - 
( crontab -l ; echo "30 3 * * MON /home/ubuntu/fiware-management-scripts/management.py -a Urgent 2>/dev/null" ) | crontab -
( crontab -l ; echo "00 4 * * MON /home/ubuntu/fiware-management-scripts/management.py -a Accounts 2>/dev/null" ) | crontab -

( crontab -l ; echo "30 4 * * * /home/ubuntu/fiware-management-scripts/management.py -a Askbot 2>/dev/null" ) | crontab -
( crontab -l ; echo "00 5 * * * /home/ubuntu/fiware-management-scripts/management.py -a Caretaker 2>/dev/null" ) | crontab -
( crontab -l ; echo "30 5 * * * /home/ubuntu/fiware-management-scripts/management.py -a Stackoverflow 2>/dev/null" ) | crontab -
( crontab -l ; echo "00 6 * * * /home/ubuntu/fiware-management-scripts/management.py -a FLUAs 2>/dev/null" ) | crontab -

Top

Docker and Docker Compose

For more details about the docker version of the service, take a look to the README content. In case that you need to access to the created image, execute the following command:

docker run -it <Docker Image ID> sh

Troubleshooting

Problems with Google Credentials

It is natural that the Google Access Token and Google Refresh Token expires after a while. If it is the Access Token the one that it expires the system can regenerate automatically a new Access Token using the Refresh Token. Nevertheless, if the Refresh Token is expired too, you need to request new Access Token and Refresh Token using the Client ID and Client Secret. You can use the script oauth2.py to generate and authorize an OAuth2 token.

  oauth2 [email protected] \
    --client_id=1038[...].apps.googleusercontent.com \
    --client_secret=[...] \
    --generate_oauth2_token

More details, take a look to the README content.

License

These scripts are licensed under Apache License 2.0.

fiware-management-scripts's People

Contributors

flopezag avatar jicarretero avatar

Watchers

James Cloos avatar  avatar  avatar

Forkers

fgalan

fiware-management-scripts's Issues

Problems with the number of requests to StackOverflow

We have problems with the number of requests to the StackOverflow... need to check it.

From StackOverflow scripts:

2018-01-17 06:05:59|INFO:stackoverflow questions= 810
2018-01-17 09:05:02|INFO:Getting the HelpDesk monitors data
2018-01-17 09:05:36|INFO:Getting the StackOverflow data
2018-01-17 09:05:36|INFO:loaded questions from file: FIWARE.Helpdesk.StackOverflow.pkl
2018-01-17 09:05:54|INFO:saved questions to file: /home/fla/fiware-management-scripts/HelpDesk/store/FIWARE.Helpdesk.StackOverflow.pkl
2018-01-17 09:05:59|INFO:questions=810, monitors=1642, matches=810
2018-01-17 09:05:59|INFO:Obtaining list of questions with no answer
2018-01-17 09:06:00|INFO:NOT available answered questions for synchronization with help desk
2018-01-17 09:06:00|INFO:Obtaining list of questions answers
2018-01-17 09:06:00|INFO:NOT available questions with accepted answer for synchronization with help desk
2018-01-17 09:06:00|INFO:Getting the final list of StackOverflow questions
2018-01-17 09:06:00|ERROR:Error in the request to StackOverflow: too many requests from this IP, more requests available in 11832 seconds
2018-01-17 09:06:00|INFO:helpdesk: # issues created = 0
2018-01-17 09:06:00|INFO:helpdesk: # issues transitions = 0
2018-01-17 09:06:00|INFO:helpdesk: # issues assignments = 0
2018-01-17 09:06:00|INFO:stackoverflow questions= 810

From mainhelpdesk.log

2018-01-17 09:00:02|ERROR:HTTPConnectionPool(host='backlog.fiware.org', port=80): Max retries exceeded with url: /api/enablersbook (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fc90dd7e190>: Failed to establish a new connection: [Errno 111] Connection refused',))
2018-01-17 09:00:02|ERROR:Unexpected error: <class 'requests.exceptions.ConnectionError'>
Traceback (most recent call last):
File "/home/fla/fiware-management-scripts/HelpDesk/fiware-main-helpdesk-caretaker.py", line 31, in
helpdesk = HelpDesk()
File "/home/fla/fiware-management-scripts/HelpDesk/desks/helpdesk.py", line 52, in init
self.enablersBook = find_enablersBook()
File "/home/fla/fiware-management-scripts/HelpDesk/desks/resourcesBooks.py", line 14, in find_enablersBook
answer = requests.get(url, auth=auth)

Log level in emailer class

By default the class Emailer is called with the logger.DEBUG details. it should be something that have to be configured in the configuration files.

Problems in the execution of the account-desk.py

2017-04-27 08:33:08|ERROR:{u'smartsdk_martel': (553, '5.1.2 The recipient address <smartsdk_martel> is not a valid RFC-5321\n5.1.2 addre
ss. e12sm1162681wrc.43 - gsmtp')}
Traceback (most recent call last):
File "/home/fla/fiware-management-scripts/DesksReminder/Basics/emailer.py", line 147, in _deliver
server.sendmail(self._sender, msg['To'], msg.as_string())
File "/usr/lib/python2.7/smtplib.py", line 747, in sendmail
raise SMTPRecipientsRefused(senderrs)
SMTPRecipientsRefused: {u'smartsdk_martel': (553, '5.1.2 The recipient address <smartsdk_martel> is not a valid RFC-5321\n5.1.2 address.
e12sm1162681wrc.43 - gsmtp')}

Problem with certain users

FIWARE: Reminders: Unknown Nickname for Ozdemir, Omer

Dear Reminders Admin,

Please, have a look at Ozdemir, Omer nickname because it wasn't found in the contact book.

Thanks in advance for cooperation!!

Connection unexpectedly closed

In some cases, the connection of the SMTP is closed unexpectly giving us the following information:

2017-01-17 13:23:22|ERROR:[-] Sending Mail Failed.
Traceback (most recent call last):
File "/home/fla/fiware-management-scripts/DesksReminder/Basics/emailer.py", line 147, in _deliver
server.sendmail(self._sender, msg['To'], msg.as_string())
File "/usr/lib/python2.7/smtplib.py", line 723, in sendmail
self.ehlo_or_helo_if_needed()
File "/usr/lib/python2.7/smtplib.py", line 543, in ehlo_or_helo_if_needed
(code, resp) = self.helo()
File "/usr/lib/python2.7/smtplib.py", line 403, in helo
(code, msg) = self.getreply()
File "/usr/lib/python2.7/smtplib.py", line 368, in getreply
raise SMTPServerDisconnected("Connection unexpectedly closed")
SMTPServerDisconnected: Connection unexpectedly closed

Not sure if it is a problem with the data or a problem with de code.

Problems with certain users

We receive a message about certain users due to we do not have them:

  • Radosław Adamkiewicz nickname
  • Santiago Martinez García nickname
  • Miguel Jiménez nickname

it wasn't found in the contact book.

Problems with Stackoverflow questions

Since mid of May, apparently there were some changes in Stackoverflow that produce a recreation of all the tickets again althought they could be old or answered and created previously in JIRA.

Problems with logs directory

If the directory logs does not exist inside the folder DesksReminder and HelpDesk the scripts produce an error:

Traceback (most recent call last):
File "/home/fla/fiware-management-scripts/DesksReminder/fiware-urgentDesk-weekly.py", line 24, in
level=log_level)
File "/usr/lib/python2.7/logging/init.py", line 1532, in basicConfig
hdlr = FileHandler(filename, mode)
File "/usr/lib/python2.7/logging/init.py", line 903, in init
StreamHandler.init(self, self._open())
File "/usr/lib/python2.7/logging/init.py", line 928, in _open
stream = open(self.baseFilename, self.mode)
IOError: [Errno 2] No such file or directory: '/home/fla/fiware-management-scripts/DesksReminder/logs/urgentdesk-weekly.log'

Deliver option should be in config file

Currently all the scripts are making use of a variable 'deliver' which is assigned a value True or False to send or not the email. This parameter should be defined in the configuration file and not defined in all the scripts.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.