paloaltonetworks / minemeld-webui

WebUI of MineMeld

minemeld-webui

WebUI for minemeld

BUILDING

Requirements

Not strictly required but suggested:

• virtualenv (https://pypi.python.org/pypi/virtualenv)
• nodeenv (https://github.com/ekalinin/nodeenv)

Procedure

Setup a virtual node env and activate it

nodeenv -v -n 0.12.2 --npm=2.14.7 --prebuilt -c venv
. ./venv/bin/activate

Clone the repo

git clone https://github.com/PaloAltoNetworks/minemeld-webui.git
cd minemeld-webui

Install the package dev deps

npm install

Add local node modules to the PATH

export PATH=$(npm bin):$PATH

Install the bower deps

bower install

Install typings type files

typings install

Check for known security issues on node packages

nsp check

Build the WebUI in the dist subdirectory

gulp build

TESTING

Use the following command to serve and test your local version of the WebUI during development:

gulp serve --url https://<IP of MineMeld VM>

minemeld-webui's Issues

Placeholder when config is empty

When there are no nodes in the CONFIG page, a placeholder should be created saying "Add nodes to the config". Or similar.

In rule view of syslog miners, HITS are not updated correctly

Refresh should be hit a couple of times to show the stats.

"Color" column should not be sortable

In both config and nodes tables.

Be responsive when minemeld WebAPI are not responding

When engine API restarts or system is starting WebUI just "hangs", loading bar is stuck and no message is shown. This should be modified to show a message to the user.

Add node - show description of selected prototype

When a prototype is selected in the add node view, the prototype description should appear somewhere in the page.

Remove bower

Switch to npm + webpack. Bower is deprecated.

Sorting states persistent across page views

State of the table sorting should be persistent across page views.

Warnings on NPM

On building the Web UI, npm emits the following warnings:

- Install npm packages...
npm WARN deprecated [email protected]: ...psst! Your project can stop working at any moment because its dependencies can change. Prevent this by migrating to Yarn: https://bower.io/blog/2017/how-to-migrate-away-from-bower/
npm WARN deprecated [email protected]: Please use gulp-clean-css
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: Typings is deprecated in favor of NPM @types -- see README for more information
npm WARN deprecated [email protected]: wrench.js is deprecated! You should check out fs-extra (https://github.com/jprichardson/node-fs-extra) for any operations you were using wrench for. Thanks for all the usage over the years.
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Use uuid module instead
npm WARN deprecated [email protected]: ReDoS vulnerability parsing Set-Cookie https://nodesecurity.io/advisories/130
npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated [email protected]: express 2.x series is deprecated
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: connect 1.x series is deprecated
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
npm WARN deprecated [email protected]: Use uuid module instead
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: this package has been reintegrated into npm and is now out of date with respect to npm
...
> [email protected] install /opt/minemeld/www/minemeld-webui-0.9.44/node_modules/node-sass
> node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v3.13.1/linux-x64-57_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/v3.13.1/linux-x64-57_binding.node":

HTTP error 404 Not Found

Hint: If github.com is not accessible in your location
      try setting a proxy via HTTP_PROXY, e.g.

      export HTTP_PROXY=http://example.com:1234

or configure npm proxy via

      npm config set proxy http://example.com:8080

Note that https://github.com/sass/node-sass/releases/download/v3.13.1/linux-x64-57_binding.node does not exist, it's not a proxy issue.

Support for renaming nodes

Users should be able to rename nodes. Data files and tables should be renamed as well.

WebUI should permit download of MineMeld CA certificate

Warnings on gulp build

When building the web ui from source, gulp spits a number of warnings:

- Gulp build...
[20:50:53] Using gulpfile /opt/minemeld/www/minemeld-webui-0.9.44/gulpfile.js
[20:50:53] Starting 'scripts'...
[20:50:54] Starting 'styles'...
[20:50:55] Starting 'partials'...
[20:50:56] Starting 'fonts'...
[20:50:56] Starting 'other'...
[20:50:57] gulp-inject 16 files into index.scss.
[20:51:08] Finished 'other' after 12 s
[20:51:33] Finished 'styles' after 39 s
[20:51:37] Finished 'fonts' after 41 s
[20:52:14] Time: 76739ms
          Asset     Size  Chunks             Chunk Names
index.module.js  1.77 MB       0  [emitted]  main

WARNING in ./src/app/nodedetail/localdb.controller.ts
[500, 9]: for (... in ...) statements must be filtered with an if statement
[67, 30]: expected parameter: 'result' to have a typedef
[97, 30]: expected parameter: 'result' to have a typedef
[364, 53]: expected parameter: 'key' to have a typedef
[475, 49]: expected parameter: 'key' to have a typedef


WARNING in ./src/app/nodedetail/nodedetail.info.controller.ts
[90, 5]: Declaration of public instance member function not allowed to appear after declaration of private instance member function


WARNING in ./src/app/nodedetail/proofpoint.controller.ts
[270, 24]: != should be !==


WARNING in ./src/app/nodedetail/recordedfuture.controller.ts
[138, 24]: != should be !==


WARNING in ./src/app/nodedetail/taxiiclient.controller.ts
[111, 65]: == should be ===


WARNING in ./src/app/nodedetail/yamlipv4.controller.ts
[449, 29]: forbidden bitwise operation


WARNING in ./src/app/dashboard/dashboard.controller.ts
[173, 19]: comment must start with lowercase letter


WARNING in ./src/app/system/system.controller.ts
[105, 34]: == should be ===
[125, 34]: == should be ===
[150, 34]: == should be ===
[199, 34]: == should be ===


WARNING in ./src/app/system/extensions.controller.ts
[528, 15]: comment must start with lowercase letter
[416, 56]: == should be ===
[509, 30]: == should be ===


WARNING in ./src/app/config/config.controller.ts
[205, 34]: == should be ===
[205, 68]: == should be ===
[610, 37]: == should be ===
[633, 28]: == should be ===
[726, 32]: == should be ===


WARNING in ./src/app/config/configimport.controller.ts
[403, 9]: for (... in ...) statements must be filtered with an if statement


WARNING in ./src/app/config/configadd.controller.ts
[111, 59]: == should be ===
[157, 24]: == should be ===
[187, 25]: == should be ===
[194, 31]: == should be ===


WARNING in ./src/app/admin/admin.users.controller.ts
[59, 13]: unused variable: 'lastUser'
[68, 31]: == should be ===


WARNING in ./src/app/indicatoradd/indicatoradd.controller.ts
[188, 29]: forbidden bitwise operation


WARNING in ./src/app/components/navbar/navbar.directive.ts
[79, 28]: != should be !==
[84, 45]: != should be !==


WARNING in ./src/app/components/suggestion/suggestion.directive.ts
[31, 9]: 'toastr' cannot be declared in the constructor
[32, 9]: '$modal' cannot be declared in the constructor
[93, 17]: '$modalInstance' cannot be declared in the constructor
[58, 19]: expected parameter: 'result' to have a typedef
[60, 28]: expected parameter: 'success' to have a typedef
[63, 26]: expected parameter: 'error' to have a typedef
[75, 19]: expected parameter: 'result' to have a typedef


WARNING in ./src/app/services/minemeldapi.ts
[197, 20]: == should be ===


WARNING in ./src/app/services/status.ts
[320, 12]: trailing whitespace


WARNING in ./src/app/services/prototype.ts
[16, 28]: comment must start with lowercase letter


WARNING in ./src/app/services/events.ts
[110, 63]: == should be ===


WARNING in ./src/app/services/traced.ts
[224, 21]: forbidden bitwise operation
[226, 38]: forbidden bitwise operation
[226, 38]: forbidden bitwise operation


WARNING in ./src/app/services/enginestatus.ts
[85, 50]: != should be !==
[111, 30]: != should be !==


WARNING in ./src/app/services/jobs.ts
[122, 35]: != should be !==
[126, 39]: != should be !==


WARNING in ./src/app/services/currentuser.ts
[51, 30]: == should be ===

[20:52:16] Finished 'scripts' after 1.37 min
[20:52:16] Starting 'inject'...
[20:52:16] gulp-inject 1 files into index.html.
[20:52:16] gulp-inject 1 files into index.html.
[20:52:16] Finished 'inject' after 170 ms
[20:52:17] Finished 'partials' after 1.35 min
[20:52:17] Starting 'html'...
[20:52:17] gulp-inject 1 files into index.html.
[20:52:44] dist/ maps/styles/app-4ff0e3d5ef.css.map 66.3 kB
[20:52:44] dist/ maps/scripts/app-299ade687d.js.map 532.66 kB
[20:52:47] dist/ maps/styles/vendor-c4f9a6a927.css.map 44.17 kB
[20:53:54] dist/ maps/scripts/vendor-c1a4d9da14.js.map 2.09 MB
[20:53:54] dist/ styles/app-4ff0e3d5ef.css 165.79 kB
[20:53:54] dist/ scripts/app-299ade687d.js 572.74 kB
[20:53:54] dist/ styles/vendor-c4f9a6a927.css 108.13 kB
[20:53:54] dist/ scripts/vendor-c1a4d9da14.js 1.51 MB
[20:53:54] dist/ index.html 1.12 kB
[20:53:54] dist/ all files 5.09 MB
[20:53:54] Finished 'html' after 1.62 min
[20:53:54] Starting 'build'...
[20:53:54] Finished 'build' after 153 μs

Unable to create a Custom Miner prototype

We want to create a new miner to download our IP address list.

We had tried to clone the blocklistDE.all prototype and when we saved it with another name, the miner is created on local/prototypes/minemeldlocal.yml

But we are unable to get it on the WebUI so we could not create a new node with that prototype.

We are running 0.9.52

Thanks in advace. Rafa.

Allow simple configuration of user-defined text parameters in node UI for extensions

If a Minemeld extension requires the user to specify some parameter in the node (i.e. a username, a password, a URL), a specific webui extension must be written that defines how the entire UI for that node should look like.

While this is powerful and flexible, it's also complex when writing extensions that require simple user input of text parameters (strings) in nodes.

This FR is for a simple method to allow extensions to specify simple configuration parameters (i.e. text strings) that the UI will present automatically to the user in the node configuration, without requiring the extension developers to write lots of webui code.

When modals disappear in CONFIG there is a black "flash"

When modals are dismissed in CONFIG, there is sort of black "flash".

To reproduce:

• load a config
• go to CONFIG
• click on an INPUT field
• click CANCEL

wrong tooltip in the ADMIN page for +

It says "Add Indicator" instead of "Add Admin"

Multiple sort criteria on tables

EOM :-)

Encode fields in datatables

Seems that datatables does not automatically escape HTML inside the fields.

Diff between candidate, running and committed configs

Support for loading old running config

Show author in prototype browser

Show name of the author in prototype browser

[Community Health Assessment] Changes needed

This issue was opened by a bot called Community Health (PANW) because this repo has failed too many community health checks.

Repo maintainers: Please take the time to fix the issues in the table to reach the target score. These improvements will help others find your work and contribute to it. This issue will update as your score improves until it hits the target score.

Click More info for instructions to fix each item.

Health Check	Pass	Score	More Info
Contains a meaningful README.md file	✅	20 / 20	More info
SUPPORT.md file exists	❌	0 / 20	More info
Repo has a description	✅	15 / 15	More info
Has a recognized open source license	✅	15 / 15	More info
Has a descriptive repo name	✅	15 / 15	More info
Required topics attached to repo	❌	0 / 15	More info
CONTRIBUTING.md file with contribution guidelines	❌	0 / 5	More info
Has custom issue and pull request templates	❌	0 / 5	More info

Current score: 65
Target threshold: 100
Total possible: 110

Limitation issue about inputs in output nodes

Hi,
i've seen there is a limitation about inputs I can choose from the web interface in output nodes, but I can bypass this limitation editing the config file of the nodes. I thin this is a web interface limitation.

Thank you,
Giorgio

Prisma Cloud IaC Scan - Error occurred during scan

Prisma credentials not configured properly. Please reinstall and complete the configuration.

In syslog miner rule view, hits counter is always 0 for some rules

Version: 0.9.18

How to reproduce:

• create a syslog miner
• create a rule with '-' in it
• send syslog messages matching the rule
• hits is 0

Corner in dialog titles are still round

Dialog corners are "square", but the dialog title still has round corners this leaves a white margin in top corners.

Fix errors on dashboard when stats are not available

Invalid value for attribute transform="translate(0.8004246284501062, NaN)"
Invalid value for attribute y1="NaN"
Invalid value for attribute y2="NaN"
Invalid value for attribute height="NaN"

Error showing dagpusher device list

TypeError: Cannot read property 'replace' of undefined

Implement a full prototype editor

The prototype editor should be able to:

• create a brand new prototype
• clone an existing prototype and modify its attributes

Add support for backend status messages

Coming from supervisord event listener.

Problems in EXPORT

2 bugs in EXPORT in the CONFIG view:

1. nodes wrapper object is missing
2. if inputs are not defined in the object, an empty inputs is produced.

Example:

ransomwaretracker_DOMBL:
  inputs:
  output: true
  prototype: ransomwaretracker.RW_DOMBL
  node_type: miner

Should be:

nodes:
  ransomwaretracker_DOMBL:
    output: true
    prototype: ransomwaretracker.RW_DOMBL
    node_type: miner

In yamlipv6.controller.ts check on IPv6 range is incomplete

Form should check that 1st address is lower than second address.

Support for client cert in TAXII Miner

Currently the only way to supply a client cert to TAXII Miner is via shell. We should support this via WebUI.

User should be able to:

• upload client certificate to TAXII Miner
• remove current client certificate

When views are changed, ongoing Ajax call should be canceled

Problem becomes evident when backend is slow or network latency is high.

Redis Feeds should open in a new window/tab

Better UX

Confirm not prevent deletion of last admin

WebUI should ask for confirm and do not prevent deletion of last admin user, this to support scenarios where local admins credentials are optional.

Show all the CPUs stats in the system page not only one

Group prototypes in the prototype browser

The number of prototypes is growing, we should start investigating a better way to handle browsing the prototype library.

Validate node names

Before sending them to backend and before rendering on the UI.

In graph view, when clicking on the current node the wrong info view is shown

To reproduce:

• in NODES click on a node
• click on GRAPH
• click on the current node
• info view doesn't show special fields

Create a graph based config page

Nodes are added and removed from the graph. Link between nodes are created using point and click.