c6fc / npk

A mostly-serverless distributed hash cracking platform

NPK - Increase your cred yield!

NPK is a distributed hash-cracking platform built entirely of serverless components in AWS including Cognito, DynamoDB, and S3. It was designed for easy deployment and the intuitive UI brings high-power hash-cracking to everyone.

'NPK' is an initialism for the three primary atomic elements in fertilizer (Nitrogen, Phosphorus, and Potassium). Add it to your hashes to increase your cred yield!

How it works

Let's face it - even the beastliest cracking rig spends a lot of time at idle. You sink a ton of money up front on hardware, then have the electricity bill to deal with. NPK lets you leverage extremely powerful hash cracking with the 'pay-as-you-go' benefits of AWS. For example, you can crank out 336 GH/s of NTLM for a mere $1/hr and scale it however you want. NPK was also designed to fit easily within the free tier while you're not using it! Without the free tier, it'll still cost less than $1 per MONTH to have online!

Features

1. Super easy install

Paste a one-liner into AWS CloudShell. Pretty easy.

source <(curl https://npkproject.io/cloudshell_install.sh)

If you'd like to use the dev branch to use beta features, use this one-liner instead:

source <(curl https://npkproject.io/cloudshell_install_dev.sh)

There are also Step-by-step instructions if you want them.

2. Intuitive campaign builder

Take the trial-and-error out of complex attack types with the intuitive campaign builder. With a couple clicks you can create advanced campaigns that even advanced Hashcat users would struggle to emulate.

3. Campaign price and coverage estimates

Take the guess-work out of your campaigns. See how far you'll get and how much it will cost before starting the campaign.

4. Max price enforcement and runaway instance protection

GPU instances are expensive. Runaway GPU instances are EXTREMELY expensive. NPK will enforce a maximum campaign price limit, and was designed to prevent runaway instances even with a complete failure of the management plane.

5. Multi-Tenancy & SAML-based single sign-on

NPK supports multiple users, with strict separation of data, campaigns, and results between each user. It can optionally integrate with SAML-based federated identity providers to enable large teams to use NPK with minimal effort.

6. Data lifecycle management

Configure how long data will stay in NPK with configurable lifecycle durations during installation. Hashfiles and results are automatically removed after this much time to keep things nicely cleaned up.

Easy Install

ProTip: To keep things clean and distinct from other things you may have in AWS, it's STRONGLY recommended that you deploy NPK in a fresh account. You can create a new account easily from the 'Organizations' console in AWS. By 'STRONGLY recommended', I mean 'seriously don't install this next to other stuff'.

Note: If you have an older version of NPK that you deployed without the one-liner, you'll need to destroy it before installing the new version

1. Log into the AWS Console for the account you want to deploy to.

2. Click the AWS CloudShell button in the top right corner.

3. Paste in the one-liner: source <(curl https://npkproject.io/cloudshell_install.sh)

4. Use the wizard to complete the configuration

When the deploy finishes, you'll be dropped to a custom prompt, which indicates that NPK is deployed and CloudShell is connected to it.

If you said 'no' at the end of the wizard, you can run npm run deploy from this prompt to finish the deployment.

See https://github.com/c6fc/npk/wiki/Detailed-NPK-Settings for more details about advanced configurations, or https://github.com/c6fc/npk/wiki/Configuring-SAML-SSO for help configuring SAML SSO.

Connect to an existing installation

Note: If you have an older version of NPK that you deployed without the one-liner, you'll need to destroy it before installing the new version

To connect to an existing NPK installation (which is needed to modify or uninstall NPK), log into the AWS account where NPK resides, click the CloudShell icon, and paste in the one-liner:

source <(curl https://npkproject.io/cloudshell_install.sh)

CloudShell will now connect to NPK (which may take a minute or two), after which you'll drop to a new prompt that looks like this:

You're now connected to your NPK installation. This can be performed by any user in the AWS account with admin rights, and can be performed in any region.

Modify Install

You can change the settings of an install without losing your existing campaigns. Use the instructions above to connect to your NPK installation, then edit npk-settings.json as necessary and run npm run update. It's that easy!

cloudshell-user$ source <(curl https://npkproject.io/cloudshell_install.sh)
@c6fc/npk> vim npk-settings.json
< ... change your settings however you need >
@c6fc/npk> npm run update

Uploading your own dictionaries and rule files

Once NPK has been deployed, administrative users can use the NPK console to upload wordlists and rule files using the 'Dictionary Management' link in the sidebar. NPK supports plain-text and gzipped dictionaries.

Uninstall

You can completely turn down NPK and delete all of its data from AWS very easily. Just attach your CloudShell to NPK, then run npm run destroy:

cloudshell-user$ source <(curl https://npkproject.io/cloudshell_install.sh)
@c6fc/npk> npm run destroy

Official Discord Channel

Have questions, need help, want to contribute or brag about a win? Come hang out on Discord!

npk's Issues

NPK Deployment / Custom Domain Issue

I'm deploying NPK with custom domain and npk-settings.json configs are ignored and NPK is deployed with default Cloudfront domain. Let me know if there is anything below that could could cause issue and I am on latest release of NPK.

{
"backend_bucket": "npk-terraform-quickdeploy-",
"campaign_data_ttl": 604800,
"georestrictions": ["US", "CA", "GB"],
"campaign_max_price": 50,
"useCustomDNS": true,
"route53Zone": "",
"dnsNames": {
"www": [
""
],
"api": [
""
]
},
"awsProfile": "default",
"criticalEventsSMS": "",
"adminEmail": "",
"useSAML": true,
"sAMLMetadataFile": "/npk/NPK.xml"
}

Cannot find module 'yargs'

Hello,
I just run the install command : source <(curl https://npkproject.io/cloudshell_install.sh)
And I have the following error. I got the same error if I do npm install yargs

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[+] Installing Node.js prerequisites. This can take up to two minutes, and may appear frozen. DON'T INTERRUPT IT.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


npm WARN deprecated [email protected]: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm ERR! code 1
npm ERR! path /aws/mde/npk/node_modules/@hanazuki/node-jsonnet
npm ERR! command failed
npm ERR! command sh -c cmake-js build --CDCMAKE_EXPORT_COMPILE_COMMANDS=1
npm ERR! [
npm ERR!   '/aws/mde/nvm/versions/node/v17.0.1/bin/node',
npm ERR!   '/aws/mde/npk/node_modules/.bin/cmake-js',
npm ERR!   'build',
npm ERR!   '--CDCMAKE_EXPORT_COMPILE_COMMANDS=1'
npm ERR! ]
npm ERR! Not searching for unused variables given on the command line.
npm ERR! -- The C compiler identification is GNU 7.3.1
npm ERR! -- The CXX compiler identification is GNU 7.3.1
npm ERR! -- Check for working C compiler: /usr/bin/cc
npm ERR! -- Check for working C compiler: /usr/bin/cc -- broken
npm ERR! -- Configuring incomplete, errors occurred!
npm ERR! See also "/aws/mde/npk/node_modules/@hanazuki/node-jsonnet/build/CMakeFiles/CMakeOutput.log".
npm ERR! See also "/aws/mde/npk/node_modules/@hanazuki/node-jsonnet/build/CMakeFiles/CMakeError.log".
npm ERR! info TOOL Using Unix Makefiles generator.
npm ERR! info CMD CONFIGURE
npm ERR! info RUN [
npm ERR! info RUN   'cmake',
npm ERR! info RUN   '/aws/mde/npk/node_modules/@hanazuki/node-jsonnet',
npm ERR! info RUN   '--no-warn-unused-cli',
npm ERR! info RUN   '-G',
npm ERR! info RUN   'Unix Makefiles',
npm ERR! info RUN   '-DCMAKE_JS_VERSION=6.3.0',
npm ERR! info RUN   '-DCMAKE_BUILD_TYPE=Release',
npm ERR! info RUN   '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/aws/mde/npk/node_modules/@hanazuki/node-jsonnet/build/Release',
npm ERR! info RUN   '-DCMAKE_JS_INC=/home/cloudshell-user/.cmake-js/node-x64/v17.0.1/include/node',
npm ERR! info RUN   '-DCMAKE_JS_SRC=',
npm ERR! info RUN   '-DNODE_RUNTIME=node',
npm ERR! info RUN   '-DNODE_RUNTIMEVERSION=17.0.1',
npm ERR! info RUN   '-DNODE_ARCH=x64',
npm ERR! info RUN   '-DCMAKE_EXPORT_COMPILE_COMMANDS=1'
npm ERR! info RUN ]
npm ERR! CMake Error at /usr/share/cmake3/Modules/CMakeTestCCompiler.cmake:52 (message):
npm ERR!   The C compiler
npm ERR! 
npm ERR!     "/usr/bin/cc"
npm ERR! 
npm ERR!   is not able to compile a simple test program.
npm ERR! 
npm ERR!   It fails with the following output:
npm ERR! 
npm ERR!     Change Dir: /aws/mde/npk/node_modules/@hanazuki/node-jsonnet/build/CMakeFiles/CMakeTmp
npm ERR!     
npm ERR!     Run Build Command:"/usr/bin/gmake" "cmTC_c48e2/fast"
npm ERR!     /usr/bin/gmake -f CMakeFiles/cmTC_c48e2.dir/build.make CMakeFiles/cmTC_c48e2.dir/build
npm ERR!     gmake[1]: Entering directory `/aws/mde/npk/node_modules/@hanazuki/node-jsonnet/build/CMakeFiles/CMakeTmp'
npm ERR!     Building C object CMakeFiles/cmTC_c48e2.dir/testCCompiler.c.o
npm ERR!     /usr/bin/cc    -o CMakeFiles/cmTC_c48e2.dir/testCCompiler.c.o   -c /aws/mde/npk/node_modules/@hanazuki/node-jsonnet/build/CMakeFiles/CMakeTmp/testCCompiler.c
npm ERR!     Linking C executable cmTC_c48e2
npm ERR!     /usr/bin/cmake3 -E cmake_link_script CMakeFiles/cmTC_c48e2.dir/link.txt --verbose=1
npm ERR!     /usr/bin/cc      CMakeFiles/cmTC_c48e2.dir/testCCompiler.c.o  -o cmTC_c48e2 
npm ERR!     collect2: fatal error: cannot find 'ld'
npm ERR!     compilation terminated.
npm ERR!     gmake[1]: *** [cmTC_c48e2] Error 1
npm ERR!     gmake[1]: Leaving directory `/aws/mde/npk/node_modules/@hanazuki/node-jsonnet/build/CMakeFiles/CMakeTmp'
npm ERR!     gmake: *** [cmTC_c48e2/fast] Error 2
npm ERR!     
npm ERR! 
npm ERR!   
npm ERR! 
npm ERR!   CMake will not be able to correctly generate this project.
npm ERR! Call Stack (most recent call first):
npm ERR!   CMakeLists.txt:3 (project)
npm ERR! 
npm ERR! 
npm ERR! ERR! OMG Process terminated: 1

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/cloudshell-user/.npm/_logs/2022-05-06T15_15_53_581Z-debug-0.log
node:internal/modules/cjs/loader:936
  throw err;
  ^

Error: Cannot find module 'yargs'
Require stack:
- /aws/mde/npk/bin/index.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/aws/mde/npk/bin/index.js:5:15)
    at Module._compile (node:internal/modules/cjs/loader:1095:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1147:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/aws/mde/npk/bin/index.js' ]
}

Node.js v17.0.1

Deprecation warning in tf output

╷
│ Warning: Argument is deprecated
│ 
│   with local_file.ssh_key,
│   on keys.tf.json line 14, in resource.local_file.ssh_key:
│   14:                 "sensitive_content": "${tls_private_key.ssh.private_key_pem}"
│ 
│ Use the `local_sensitive_file` resource instead.
│ 
│ (and 2 more similar warnings elsewhere)

Less is not installed on base image, causing quickdeploy.sh to fail

I pulled the latest docker image (at the time of this issue) and attempted to run the quick deploy script. I received the following error message.

[!] The provided profile is not valid. Try again.

Debugging the script, the cause is this line;

aws --profile $profile sts get-caller-identity > /dev/null

Attempting this in isolation results in the following error.

npm run deployroot@5290a070ec47:~/npk/terraform# aws sts get-caller-identity

[Errno 2] No such file or directory: 'less': 'less'

The root cause is AWS CLI requires less to be installed. Performing an apt-get update and apt-get install less resolves the issue. Can we get this integrated into the base image?

Cheers.

Add link to campaign from dashboard

1. Any way this message can be removed or updated when campaign is running "No status reports received. Nodes may have errored out" with something more accurate?
   

2. Manage Campaigns page should also include "Start" function since right now you can only execute "Cancel / Delete" functions from that page.

3. Any way to remove hyperlink to AWS EC2 instance that is shown on dashboard and add hyperlink to related campaign?
   

How to use an existing potfile

I already have a large pot file with a lot of cracked hashes, and I want to use it with NPK, sharing it with all campaigns to increment the potfile and to avoiding cracking the same hash twice, there is a way I could do it?

AWS Cli v2 Not installed in Docker Alpine linux

I used the docker container last night to test out a quick deployment and got presented with a few errors. The first error gave me a issue with not having AWS CLI v2 installed in the docker container. I see you also check to see if AWS CLI v2 is installed in the deployment scripts or you error the script out. I was able to get this fixed following this link:

https://stackoverflow.com/questions/61918972/how-to-install-aws-cli-on-alpine

The code I was able to run to install aws-cli-2 on alpine linux is the following:
apk --no-cache add \ binutils \ curl \ && GLIBC_VER=$(curl -s https://api.github.com/repos/sgerrand/alpine-pkg-glibc/releases/latest | grep tag_name | cut -d : -f 2,3 | tr -d \",' ') \ && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \ && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \ && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \ && apk add --no-cache \ glibc-${GLIBC_VER}.apk \ glibc-bin-${GLIBC_VER}.apk \ && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip \ && unzip awscliv2.zip \ && aws/install \ && rm -rf \ awscliv2.zip \ aws \ /usr/local/aws-cli/v2/*/dist/aws_completer \ /usr/local/aws-cli/v2/*/dist/awscli/data/ac.index \ /usr/local/aws-cli/v2/*/dist/awscli/examples \ && apk --no-cache del \ binutils \ curl \ && rm glibc-${GLIBC_VER}.apk \ && rm glibc-bin-${GLIBC_VER}.apk \ && rm -rf /var/cache/apk/*

NPK Instance Region Admin Settings

Allow admins to configure which regions NPK can utilize when users are selecting ec2 instance. We tried to implement this via AWS policy, but that broke some stuff with NPK campaign starting and NPK terraform code deployment. More info can be found here #51

Uploaded wordlist stuck in to_process

Allow admin to see all campaigns

Hi,
Is there the possibility to let an admin see all campaigns?

Also, is there the possibility to add users to a campaign as the owner, so we can have the progress shared?

Kind regards,
Sh4d0w

Hashes won't Ingest

Trying to run NTLM hashes.

Did as instructed with just the hash and one per line.

WPA / WAP2 Hash File Issue

Running into error below when trying to create campaign with hash type wpa/wpa2 hccapx file.

ERROR Unknown parameter type "require". Must be "request_descriptors", "response_descriptors", "request_parameters" or "response_parameters".

Unsupported regions shown for instance families

Got error message when starting an campaign "Failed to Request spot fleet" and below is CloudWatch log I got. Any idea why this would happen?

2022-04-07T04:30:53.995Z INFO Failed to request spot fleet. InvalidSpotFleetRequestConfig: Invalid: (InstanceType: g5g.xlarge with Os: Linux/UNIX and EC2-VPC),(InstanceType: g5g.xlarge with Os: Linux/UNIX and EC2-VPC),(InstanceType: g5g.xlarge with Os: Linux/UNIX and EC2-VPC)
at Request.extractError (/var/task/node_modules/aws-sdk/lib/services/ec2.js:50:35)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:688:14)
at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/var/task/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/var/task/node_modules/aws-sdk/lib/request.js:690:12)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:116:18) {
code: 'InvalidSpotFleetRequestConfig',
time: 2022-04-07T04:30:53.994Z,
requestId: '',
statusCode: 400,
retryable: false,
retryDelay: 72.47257896273831
}

Dictionary + rules produces different results than dictionary alone.

I have a file of 97 hashes that I uploaded and a file of 41 passwords that are all legit, and they all crack, but adding a rule set file to it only cracked 1 out of 97.

Changing Primary Region when you don't have quota

Hi !

As you asked me, there is an issue when a user try to change the primary region when he doesn't have the right GPU quota :

npm run configure

? Which region do you plan to use most? eu-west-1
? What is the admin user's email address? npk@xxx
? Enter an SMS-enabled phone number for critical events, including country code: xxx
? Max campaign price in USD: 50
? NPK is configured. Deploy now? Yes
[+] Retrieved all active regions
[+] Retrieved quotas.
[+] Retrieved per-region instance support.
[+] Retrieved availability zones.

[*] All prerequisites finished. Generating infrastructure configurations.
[+] Using bootstrap bucket sonnetry-shnprltxj-1651576289
Trace: Error: Error parsing Jsonnet file: JsonnetError: RUNTIME ERROR: field does not exist: eu-west-1
        terraform.jsonnet:410:16-56     thunk <a>
        terraform.jsonnet:(408:33)-(411:6)      function <anonymous>
        terraform.jsonnet:(408:33)-(411:6)      thunk <value>
        std.jsonnet:952:9-14    thunk <v>
        std.jsonnet:919:10      thunk <a>
        std.jsonnet:919:10-19   function <anonymous>
        std.jsonnet:919:10-19   function <aux>
        std.jsonnet:952:5-23    function <anonymous>
        terraform.jsonnet:(408:14)-(411:11)     object <anonymous>
        terraform.jsonnet:(405:15)-(413:5)      object <anonymous>
        terraform.jsonnet:(404:16)-(414:4)      object <config>
        jsonnet/lambda.libsonnet:(6:12)-(13:5)  object <anonymous>
        jsonnet/lambda.libsonnet:(5:24)-(14:4)  object <anonymous>
        jsonnet/lambda.libsonnet:(4:12)-(64:3)  object <anonymous>
        terraform.jsonnet:(399:37)-(462:4)      object <anonymous>
        During manifestation

    at exports.Sonnet.render (/aws/mde/npk/node_modules/@c6fc/sonnetry/src/index.js:346:10)
    at async deploy (/aws/mde/npk/bin/index.js:224:3)
    at async Object.handler (/aws/mde/npk/bin/index.js:487:20)
    at deploy (/aws/mde/npk/bin/index.js:226:11)
    at async Object.handler (/aws/mde/npk/bin/index.js:487:20)

[!] Failed to generate NPK configurations.

As you said : I need to fix the primaryRegion prompt to only show regions where there's sufficient quota.

Have a nice day ;)

Possibly incorrect hashcat syntax

Absolutely love what you are doing so please keep it up!

I had a look at the hashcat command being used:
./hashcat/hashcat.bin --quiet -O 4 -m 1600 -a 0 hashes.txt wordlist/NetgearKiller.7z wordlist/Top-10k.7z wordlist/Top-1M.7z wordlist/acdc.7z wordlist/probable-v2-wpa-top4800.7z wordlist/rockyou.7z

From what I understand -O is just a switch and does not take any arguments, but you have 4 straight after it. Unless I'm completely missing something, are you meaning -O -w 4?

That would make more sense to me.

Let me know what you think.

Set campaign max price, external reference

Would be great to see some of the improvements below in order to improve user experience.

1. Toggling features on and off isn't working as intended since users have to click outside of the toggle to get the feature enabled. You can replicate this by creating a new campaign and within new campaign page click on the toggle to enable advance mode.
2. It would be great if selecting wordlist follows the same UI as rules. This will keep consistency between different configs.
3. Having pagination within Events section would be nice feature.
4. Being able to filter within Events section from admin side in order to report against usage of NPK per user and spending.
5. Being able to set campaign max price directly from Admin settings.
6. Ability to provide custom external reference within NPK campaign. This will allow consultants to tag campaign with either project name or project ID. This can be then used to report against usage per project or even integrate NPK with other systems.

Thanks and let me know if you have any questions.

NPK's version of hashcat does not support bcrypt-sha512 hashes

When specifying the hash mode specific to bcrypt-sha512 hashes, the NPK new-campaign page returns false and will not count hashes uploaded in this format.

Start Campaign Error - UnauthorizedOperation

Getting below error when starting a new campaign. Can you replicate this on your side since nothing changed on our side when it comes to deployment.

Unable to parse response as json

Having the same problem from Coalfire-Research#4

New install, tried Chrome and Firefox, and the campaign does not get created after the error. Also same thing where the response is blank.

I went in to check out the Lambda logs like you mentioned in that issue, but I don't see the "proxy_api_handler" function? Any thoughts?

Thanks!

Wordlist in to_process folder after upload

Hey I just uploaded rockyou2021 to my npk. Is this normal? Do I need to run a command to process this file so I can see it in my word lists?

Error processing gzip uploads

Seems like processing large wordlist (1GB +) after uploading it via aws s3 cp s3://npk-dictionary-us-west-2-REDACTED/to_process/wordlist/ pauses after 15 minutes and doesn't resume. Is this due to lambda timeout or something else?

Thanks,

terraform: Error installing provider "aws": openpgp: signature made by unknown entity

I was presented with a terraform error when running the deploy.sh script from the docker image:

Error installing provider "aws": openpgp: signature made by unknown entity

This seems to be due to the current version you are using in the docker image. I was able to get this fixed by running the following command and then re-running the deply.sh script:
terraform init -verify-plugins=false

A fix would be to update terraform to a newer version that has the newly updated GPG keys.

I found this information in the following URL:
https://stackoverflow.com/questions/67368339/error-installing-provider-aws-openpgp-signature-made-by-unknown-entity

NPK Deployment / Upgrade Issue

Getting prompted with below when trying to deploy / upgrade NPK with latest commit. Do I need to specify anything when prompted or is this a bug?

var.dictionary-us-east-1
Enter a value:

var.dictionary-us-east-1-id
Enter a value:

Support for new GPU instances (G4)

Hi there,

Some time ago I proposed a way to include the new G4 instances to NPK since they were more cost effective as a pull request in the old repo: Coalfire-Research#140

The PR was incomplete so it couldn't be merged, but maybe we could resume this task. I could do it myself with some guidance about how to fully implement new instance types (which files need to be modified). #

Failed to request spot fleet. AuthFailure.ServiceLinkedRoleCreationNotPermitted

NPK build through campaign creation looked great but attempting to start a campaign resulted in a "Failed to request spot fleet" error.

Cloudwatch log group for /aws/lambda/execute_campaign showed the following authorization error after spot fleet request:

Failed to request spot fleet. AuthFailure.ServiceLinkedRoleCreationNotPermitted: The provided credentials do not have permission to create the service-linked role for EC2 Spot Fleet.

Addressed by requesting a spot fleet service linked role for the AWS account:
aws iam create-service-linked-role --aws-service-name spotfleet.amazonaws.com
aws iam create-service-linked-role --aws-service-name spot.amazonaws.com

Campaign was successfully started after building role.

How do I get instances to show up from multiple Regions?

Hi, I got approved for 4 spot instances in us-west-2 and want to use the G5 but they are not showing up when trying to start a new campaign. I re-ran the install script after they approved it. Is there something else I need to do?

In the wiki, I see you can choose from multiple Regions and would like mine set up the same.

Error: Error parsing response JSON.

Hello,

We are on the newest version of NPK and have no errors during install, but we have this error when submitting campaigns and starting them. Very rarely will the campaigns will submit successfully and some times they will start, but the error happens more often then not.

This was with a a custom wordlist (crackstation) and 450 NTLM hashes.

Let me know what other info you need!

New Feature - Campaign Notifications

It would be great if NPK would notify users when below events occurred.

1. Campaign job completed (include number of hashes cracked).
2. Campaign job cancelled (specify reason).

Thanks

User Login Experience Improvement

It would be great if NPK asked for user email address first then show SSO option or password option to login depending on users current auth provider.

Hashcat wrapper doesn't tolerate network failures

Noticed that my job would just kill the EC2 instance and after looking at output.log file below log showed up.

Any idea why this would happen?

Error sending status update to API Gateway
Error: getaddrinfo EAI_AGAIN api.npk.DOMAIN.com
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:72:26) {
errno: -3001,

Deployment Error: Unable To Authenticate. Long Term Credentials For Profile Are Invalid. AWS Profile Credential Verification Error

c6fc,
I am attempting a manual deployment of npk with the step by step installation instructions listed under:

https://github.com/c6fc/npk/wiki/Step-by-step-Installation

I was able to do a verification log in check to AWS successfully. I am able to run "aws sts get-caller-identity" which shows the account that I want to use. I then run "npm run deploy", and I get the following authentication error:

[!] Credential validation failed with error: InvalidClientTokenId: The security token included in the request is invalid.
[!] Unable to authenticate; [Error: Long term credentials for profile [xxx-xxxx] are invalid: Error: AWS profile credential verification error]

I have already tested that the AWS CLI works, so from the error above, it mentions "Error: Long term credentials for profile are invalid". The only place that where are expiration appears in is ~/.aws/credentials file under "aws_expiration="

This seems to only be an issue in the deployment process ("npm run deploy"). Since I was able to test my access to AWS successfully, AWS is not the issue. At this point, this error seems to be associated to NPM (not AWS) complaining that it does not like the AWS credentials (that it was able to find). NPM does not seem to be able to validate the long term credentials and throws an error saying that "AWS profile credential verification error".

I have not found much online to help with this error. Could this be due to how the deployment (with npm and/or nodejs) is attempting to authenticate? Is there anything in the deployment code that I can modify to fix the error referenced above?

Please advise.

vv78acc

Order Validation Failed

Hi,

I get the following error when creating a campaign :
Error: Content Type text/plain; charset=UTF-8 not permitted. Use text/plain.

The request is created on line 1524 in /site-content/angular/controllers/npkMainCtrl.js and calls the cognito provider in /site-content/angular/providers/cognitoProvider.js line 18.

The headers are provided by this line (22) params.headers = this.cognitoSigner.sign(params);. This cognitoSigner variable is a "awsSignWeb.AwsSigner" object.

No idea if this is bug in NPK or AWS changed something.

I'm using the latest onliner from the main branch to install NPK.

Could you check it out ?

Thanks,

NPK v3 Testing Bugs / Improvements

Found couple of issues below that could cause confusion.

1. Events Campaign started date is not correct.
   

2. We've noticed in one event that campaign cost was not correct. I'm not sure what caused this, but it would be great to look into if cost values on campaign and event metrics is correct.
   

3. Favorite hashes settings doesn't make sense to allow multiple values since new UI only shows default hash type.

4. It would be great to add large file upload UI to new campaign user interface.
   

Thanks,

spot_monitor trowing errors at record pace

Every invocation of the spot_monitor lambda throws the following errors:

2022-11-11T20:40:46.828Z	56d0e64e-0a6f-4f66-857b-576273621e67	INFO	TypeError: Cannot read property 'instances' of undefined
    at /var/task/main.js:87:22
    at Array.forEach (<anonymous>)
    at /var/task/main.js:72:31
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Promise.all (index 0)
    at async Runtime.exports.main [as handler] (/var/task/main.js:100:3)

2022-11-11T20:30:53.928Z	ad91e5e7-ba38-452a-90dd-2d03718bbd6f	ERROR	Invoke Error 	{
    "errorType": "Error",
    "errorMessage": "[!] Failed to retreive spot instance statuses: TypeError: Cannot read property 'instances' of undefined",
    "stack": [
        "Error: [!] Failed to retreive spot instance statuses: TypeError: Cannot read property 'instances' of undefined",
        "    at intoError (/var/runtime/Errors.js:27:12)",
        "    at postError (/var/runtime/CallbackContext.js:21:47)",
        "    at callback (/var/runtime/CallbackContext.js:42:7)",
        "    at /var/runtime/CallbackContext.js:113:16",
        "    at Runtime.exports.main [as handler] (/var/task/main.js:104:10)",
        "    at runMicrotasks (<anonymous>)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)"
    ]
}

/lambda_functions/spot_monitor/main.js#L72

Getting the matching email every minute is not my favorite thing.

{"version":"0","id":"20c1c82d-1f7f-168a-9d40-bf697b8b84e9","detail-type":"Scheduled Event","source":"aws.events","account":"XXXXXXX","time":"2022-11-11T20:54:14Z","region":"us-east-2","resources":["arn:aws:events:us-east-2:XXXXXXXXX:rule/npkSpotMonitor"],"detail":{}}

There are no open spot requests, or running instances.

┌──(user㉿devbox)-[~]
└─$ aws ec2 describe-spot-instance-requests --filters Name=state,Values=open,active
{
    "SpotInstanceRequests": []
}
                                                                                                                                                                                                                                            
┌──(user㉿devbox)-[~]
└─$ aws ec2 describe-instances --filters Name=instance-state-name,Values=pending,running 
{
    "Reservations": []
}

Update

Issue appears to have been caused by a campaign that ended (keyspace exhausted), but wasn't properly cleaned up. Canceling the campaign via the ui has (for the moment) stopped the errors, and given my inbox some peace. Not sure what cosmic bit-flip resulted in that campaign not being properly closed down.

Modify upload page wording to include .7z files

When uploading wordlists or rules to the dictionaries, the following text is present at the bottom of the upload page:

Files must be plain-text or Gzipped.
Gzipped files must use '.gz' file extension. Tarballs are not supported.

However, it is possible to upload .7z files and have them properly processed.

Can the text be changed to include .7z as an acceptable file extension? This will prevent someone (definitely not me though) from unzipping a .7z file and rezipping it as .gz.

Better detail in critical event notifications

Detail should probably be anything other than an empty object.

{"version":"0","id":"91a68cd8-1041-0d52-8ded-1e2a7722c930","detail-type":"Scheduled Event","source":"aws.events","account":"XXXXX","time":"2023-02-15T22:50:14Z","region":"us-east-2","resources":["arn:aws:events:us-east-2:522820226885:rule/npkSpotMonitor"],"detail":{}}

NPK CloudShell Deployment / SSO Improvement

When deploying NPK with SSO XML file NPK does not save the SSO XML file in S3 bucket used to hold configs. This causes deployment to fail when next users run the NPK update as XML file is missing from CloudShell local file system.

Is it possible to save XML file in S3 bucket used to hold NPK config?

Also, let me know if XML file is depreciated and if you have preferred process users should follow when setting up SSO.

Thanks,

New Campaign - Invalid Date Attack Breakdown

When selecting certain wordlists uses are getting below attack breakdown. Do I need to re-upload wordlists in order for NPK to process these in back end?

NPK Instance Type Compatibility?

Hello,

I've noticed that our prod NPK instance now shows Radeon Pro v520, but when selecting that instance NPK doesn't show any attack breakdown. Could it be that this instance isn't supported by NPK or is it just you don't have GPU specs integrated into NPK for this instance?

CloudShell Improvement

Please provide option for end users to specify if they want to just pull down current config or load current config and pull latest version of NPK when executing source <(curl https://npkproject.io/cloudshell_install.sh).

npm WARN tar TAR_ENTRY_ERROR ENOSPC: no space left on device, write

Using AWS CloudShell, I run the one-liner and it gets to the point where it prompts me for region, email, phone, price, etc. So far so good, once it gets ready to deploy I start to get tons of the following error:
null_resource.npm_install-spot_monitor (local-exec): npm WARN tar TAR_ENTRY_ERROR ENOSPC: no space left on device, write
and the end of it looks like this:

Any suggestions on expanding storage for CloudShell?

Hash files could be handled better

When I add a hash file to a campaign, it persists across the entire experience until that file is deleted using File Management. When I go to begin a new campaign and add a new hash file, I end up with the old file and new file in my new campaign. Could there be a feature added to select which files you wish to use for any given campaign during creation?

NPK / CloudShell Deployment Question

I just deployed NPK in dev environment using new CloudShell deployment process without any issues. However, I'm not sure many users know that CloudShell local files do expire after 190 days if there is no activity within that user's CloudShell.

Do you think best practice would be to pull down /aws/mde/npk directory or do you have better ways to go about not losing NPK local files?
Also, previous Docker deployment allowed other users to pull down the container and make changes / re-deploy NPK, but with CloudShell seems like only way is to pull down /aws/mde/npk directory and store that in S3.

Notification Read Config

What is the purpose of notificationread within NPK?

Add a mature potfile

What is the best way to add our existing potfile?

Unable to Launch instances (launchSpecUnusable)

Hi,
I get this error when trying to launch any type of instances:

Info (launchSpecUnusable) :
p2.xlarge, ami-0e02647cb673cb89a, Linux/UNIX, eu-central-1c, requested instance type is not supported in the requested Availability Zone
Error (spotFleetRequestConfigurationInvalid) :
p2.xlarge, ami-0e02647cb673cb89a, Linux/UNIX: We currently do not have sufficient p2.xlarge capacity in the Availability Zone you requested

I upgraded my quotas and my p2 instances limits and can launch a p2.xlarge (on ami-0e02647cb673cb89a) instance manually on the EC2 console.
Is this a eu-central-1 region problem ?

Thanks

Deployment / CloudShell Default Directory

Would it be possible to use CloudShell default directory /home/cloudshell-user as a location where NPK stores all of the files as its going through deployment? I'm not sure if you had a reason on why you didn't do this initially.

Resume campaigns, allow limited users to upload dictionaries

It would be helpful for NPK to allow regular users to upload their own wordlists / rules via UI.

Also, "Clone" feature to allow uses to clone Campaign configs and re-run the job again if previous job failed.

AWS account for npk

Hello,

never used AWS before... but now it's time because wanted to test c6fc npk :)

What kind of account do I have to sign-up at Amazon?

Thank you very much!

Joe