zowe / zowe-chat

Zowe Chat Ops

Zowe Chat

Zowe Chat is a chatting application for you to operate z/OS itself including job, dataset, USS file, error code, console command etc. from channels of 3 popular chat tools including Mattermost, Slack, Microsoft Teams. Extendibility also is provided for users to create their own plugins to extend capabilities of Zowe Chat as plugins.

Content

• Features
• Projects
• Documentation
• Contribution guidelines
• Steps to build Zowe Chat
• Steps to run Zowe Chat server

Features

• Operate z/OS job
• Operate z/OS dataset
• Operate z/OS USS file
• Query z/OS error code
• Issue z/OS console command
• Extendibility with plugin

Projects

There are four projects below under Zowe Chat repository.

• @zowe/chat
• @zowe/zos
• @zowe/clicmd
• @zowe/webapp

Documentation

For detailed information about how to install, configure, and use Zowe Chat, see Zowe Chat Documentation.

Contribution guidelines

The following information is critical to working with the code, running/writing/maintaining automated tests, developing consistent syntax in your plug-in, and ensuring that your plug-in integrates with Zowe CLI properly:

For more information about ...	See:
General guidelines that apply to contributing to Zowe Chat	Contribution Guidelines
Documentation that describes the features of the Common Bot Framework	About Common Bot Framework

Steps to build Zowe Chat

• Environment variables

  • NODE_ENV

    Specifies the building environment for your mono-repo

  • RELEASE_TYPE

    Specifies the release type of your building result

  • RELEASE_VERSION

    Specifies the release version of your building result

• Build steps

  • Clone the repo
  • Go to the directory where Zowe Chat repo is cloned
  • Run the command below to build the whole workspaces

    npm run installAll
    npm run buildAll

    Note: You can also to build single project by specifying the detailed project name. e.g. use the command npm run build --workspace=@zowe/chat to build only chat project.

Steps to run Zowe Chat server

• Prerequisite: the connection with your chat tool and related bot app are ready

• Run from local laptop for development

  Note: Depending on chat tool requirement and your laptop network environment, not all chat clients will work when running the project locally (i.e. MSTeams)

  • Execute npm run buildAll to build all projects
  • Execute npm run startLocal to set up a local environment in the folder $PROJECT_ROOT/.build based on the previous build result

    Note: This will fail on first invocation.

  • Configure your local environment in the $PROJECT_ROOT/.build/zoweChat/config folder
  • Re-execute npm run startLocal command to start Zowe Chat server locally

    Note: This will not over-write changes to your configuration

  • Launch your chat tool client and chat with your bot

• Run from your xLinux or zLinux server for production

  • Execute npm run packagingAll to build and package the project

    Note: Your must set the three required environment variables (NODE_ENV, RELEASE_TYPE, RELEASE_VERSION) for packaging first

  • Upload your building result $PROJECT_ROOT/release/zowe-chat-v<version>-.tar.gz to your Linux server
  • Logon your Linux server, create one folder and unpack the building result there
  • Set and update required environment variables

     export ZOWE_CHAT_HOME=<your created folder>/zoweChat
     export ZOWE_CHAT_PLUGIN_HOME=<your created folder>/plugins
     export PATH=$PATH:$ZOWE_CHAT_HOME/bin

  • Update the plugin configuration file $ZOWE_CHAT_PLUGIN_HOME/plugin.yaml if necessary
  • Execute the commands below to install local dependencies

    cd $ZOWE_CHAT_HOME/node_modules/i18next
    npm link
    cd  $ZOWE_CHAT_PLUGIN_HOME/@zowe/clicmd
    npm link $ZOWE_CHAT_HOME
    npm link i18next
    cd  $ZOWE_CHAT_PLUGIN_HOME/@zowe/zos
    npm link $ZOWE_CHAT_HOME
    npm link i18next

  • Update Zowe Chat, chat tool and z/OSMF server configuration per your environment

    $ZOWE_CHAT_HOME/config/chatServer.yaml
    $ZOWE_CHAT_HOME/config/zosmfServer.yaml
    $ZOWE_CHAT_HOME/config/chatTools/<mattermost | msteams | slack>.yaml

  • Execute chatsvr start to start your Zowe Chat server
    • chatsvr status - Check your Zowe Chat server status
    • chatsvr stop - Stop your Zowe Chat server
    • chatsvr restart - Restart your Zowe Chat server
  • Launch your chat tool client and chat with your bot

zowe-chat's Issues

Implement hello world code for installer

Draft IDE setup

Documentation for setting up the project in various popular IDEs.

• VS Code
• ...

z/OS job plugin - implement sample event listener

z/OS job plugin - implement sample event listener

Implement architecture code for Zowe Chat - logging

Implement architecture code for Zowe Chat - logging

• Write code to provide logging for Zowe Chat

Manage z/OS datasets from chat tools

Who: Annette and the rest of the z/OS technical team

What: Can get notified about critical status from SDSF/Zowe Desktop/Zowe Explorer in their enterprise chat platform and can use the Zowe chatbot to query information about dataset and create / delete them directly from their team channel

Wow: Hence helps with collaboration among the team, helps to solve issues in less time with less context switches and provides a consolidated user experience together with data from other management domains

End-User Documentation

Zowe Chat Documentation and Publication

Enhance common bot framework to support Zowe Chat

Enhance common bot framework to support Zowe Chat

• TypeScript interfaces
• API format

Draft contribution guidelines

Implement architecture code for Zowe Chat - configuration

Implement architecture code for Zowe Chat - configuration:

• Create and read YAML configuration files for Zowe Chat server and 3 chat tools
• Define configuration interface and write configuration class

Enable authentication in Zowe Chat

Automated Pipeline Development

• Building:
  • Code scan: SonarQube / CodeQL
  • Write building script
  • Implement building pipeline
• Testing:
  • Write UT/FVT testing case
  • Implement testing pipeline
    automation

Dynamically loading installed plugins

Provide extensibility to Zowe Chat and dynamically load plugins installed in ZOWE_CHAT_PLUGIN_HOME directly.

CI/CD pipeline implementation - build and test

Investigate the requirements from Zowe V2

investigate the requirements from Zowe V2

z/OS job plugin - implement sample message listener

z/OS job plugin - implement sample message listener

Implement view and handler for slack, Mattermost, MSteams - submit/re-submit/stop jobs

CI/CD pipeline implementation - System Testing

Scan initial code contribution to see this repository.

The source code at https://github.com/Joe-Winchester/zowe-chat-staging should be ready for scanning by the LF tools, ready for moving across into this repository.

Implement architecture code for Zowe Chat - abstract listener for message

Implement architecture code for Zowe Chat - abstract listener for message and event

• Write abstract message listener

Create skeleton Node.js project for Zowe Chat

Create one skeleton Node.js project for Zowe Chat

Build Scripts

Enhance common bot framework for job management

Investigate possible installation and configuration methods for Zowe Chat

How to use Zowe Chat with the chat tools

Work with designers for defining user scenario and workflow

Designers need some technical support for helping understand workflow and providing sample case for design mockup, which can make our design more practical and fits different use cases.

What's more, we would like to validate design with customers, so real sample case is needed.

To-do:

• new scenario proposal
• User workflow design
• Provide real sample cases
• Design review

Determine development tech. stack

Determine development tech. stack

• Runtime: Node.js
• Programming language: TypeScript
• Building tool: NPM scripts
• Testing framework: Jest
• Library and SDK: 
  • CommonBotFramework 
  • Zowe Client SDK

Setup Required Infrastructure for Dev/Test

Sub-issues:

Chat Services/Servers

• Microsoft Teams [Plan A - Linux Foundation, Plan B - Internal Teams only?]
• Mattermost [Container/Kubernetes, x86 only]
• Slack [Zowe.Robot Apps - Will only need to deploy once]

Backend

• Test Deployment Target
• Dev Deployment Target(s)

Maifnrame

• z/OS SIngle Plex

Create Image and content for zowe.org site

@Phoebecdy91 to create the image that will be mashup of the slack JOBS scenario together with graphics for Slack/Teams/Mattersmost (to show the support for those three offerings). For inspiration see the python-sdk on zowe.org (that has three graphics) and other offerings that have functional graphics against them.

Collect feedback from chat survey for chatbot

---

Your feedback matters!

Currently we are working on improving the user experience of Zowe with new features. For better fit into the realities of users’ business scenarios and improve the user experience, we got a survey to understand your current workflow, scenarios, pain points and then see how the new feature could help.
This survey may take 3-4 minutes to complete and it can help us validate our design and assumption in advance. We really appreciate your time and your comments are completely anonymous.
https://www.surveymonkey.com/r/YCCSXJG
For non-mainframer, who might not directly interact with mainframe but consume mainframe data in your daily work, your feedback is also helpful: https://www.surveymonkey.com/r/N7TVB7X
Come and help us shape the future of Zowe and Z.

FVT test - write FVT test cases

Implement architecture code for Zowe Chat - create chat bot

Implement architecture code for Zowe Chat - create chat bot

• Write one class to create messaging app
• Write Zowe Chat chat bot class and leverage Common Bot Framework to create chat bot

Developer Onboarding

create the docs folder: refer to zowe-cli/docs/
readme file

build, test,
review pull request
merge branch

Create skeleton project for z/OS job plugin

Create skeleton project for z/os job plugin

Implement architecture code for Zowe Chat - abstract listener for event

Implement architecture code for Zowe Chat - abstract listener for event

• abstract event listener

Create blogpost announcement for the project on medium.com/zowe

To help get the word out about the incubator project starting, publish an article on medium.com/zowe

Investigate designer's mockup for job scenario

Investigate designer's mockup for job scenario to evaluate and determine how to enhance common bot framework

Investigate all possibles ways for Zowe Chat authentication

• Investigate all possible ways to link chat tool user ID with RACF ID: one configuration app in Zowe Desktop?
• Implement hello world code

Investigate and implement POC code for Zowe chat installer

• Investigate possible installation method
• Implement hello world code for installer
• investigate the requirements from Zowe V2

Query knowledge base error code for z/OS

Who: Annette and the rest of the z/OS technical team

What: Can query z/OS error code to get proposed action from z/OS documents

Wow: Hence helps with collaboration among the team, helps to solve issues in less time with less context switches and provides a consolidated user experience together with data from other management domains.

Quality Gates: SonarQube / CodeQL

Implement Zowe Chat authentication

Implement Zowe Chat authentication based on the previous investigation result ( issue #9).

Sort the plugins' listeners per the predefined priority

As a Zowe Chat user, I want zowe chat bot can match and process messages and events in the order of predefined priority. Also I can control how many responses can be displayed when there are multiple plugins can handle the same message or event.

Manage z/OS jobs from chat tools

Who: Annette and the rest of the z/OS technical team

What: Can get notified about critical status from JES/Zowe Desktop/Zowe Explorer in their enterprise chat platform and can use the Zowe chatbot to query information about job and start/stop them directly from their team channel

Wow: Hence helps with collaboration among the team, helps to solve issues in less time with less context switches and provides a consolidated user experience together with data from other management domains

Complete all processes for code contribution

Complete all processes for code contribution

• Wicked proposal is approved
• - Finish  internal code scan
• - Move common bot framework code to Zowe Chat repo
• - Finish OMP code scan

Implement view and handler for slack, Mattermost, MSteams - list jobs and view a job

Command:
List jobs
View a specified job.

Feedback collection / Usability Test

We would like to validate design with users/clients. So it can help us better iterate design before final release, making sure that we fit user needs.

Method:

• Clickable design mockup
• Feedback documentation

To-do:

• Recruit users, clients who might be interested in Zowe Chat and would love to give us feedback.
• Usability test to help us get feedback based on design mockup.

Goal:

• Validate the info we show meets user needs
• Validate interaction flow meets user behaviour

Organize Zowe Chat as monorepo

Here is the proposal to organize Zowe Chat as monorepo. Please review and add your comment.

Create skeleton project and architecture code

Architecture code

• Create skeleton project code
• Architecture code to support plug-in mechanism 

Issue z/OS commands from chat tools

Who: Annette and the rest of the z/OS technical team

What: Can use the Zowe chatbot to issue commands on connected z/OS systems while ensuring granular authentication and authorization using her personal user ID

Wow: Allowing her to make changes to systems and resolving issues or running pre-canned runbooks without having to leave the collaboration environment.

Design Mockup - Job Management

Job Management Scenario

• View status of a specific job: e.g. zowe zos-jobs view job-status-by-jobid j123
  Job detailed info, related jobs (with similar name)

• List jobs of specific user or the name with prefix: e.g. zowe zos-jobs list jobs -o "ibmu*" -p "myjo*"
  Job status summary, job name list with status, job details (next step)

• List jobs, displaying only the job ID of each job with table format: e.g. zowe zos-jobs list jobs --rff jobid --rft table
  Job status summary, table

• List jobs with execution data: e.g. zowe zos-jobs list jobs —exec-data
  Job status summary, job name list with status, execution data, job details (next step)

• List jobs with specific status: e.g. active, abended job, CC, JCL error, input (don’t have related CLI cmd so far)
  Job status summary, job name list with status, job details (next step)

• Create a new connection or View the Job by filtering
  Hint for how to filter jobs: Name related, Connection related, File Path, Content related, Response Format