gchq / cyberchef-server Goto Github PK

We are trying to use docker image, followed all the steps mentioned in https://github.com/gchq/CyberChef-server#docker, after running the final command, it is complaining about node and supportive dependencies. We tried to resolved a couple of but it moved forward and again showing dependencies issue (un supported node version).

Could anyone assist regarding the same.

Describe the bug
No data is ever returned when executing recipes where operations return ArrayBuffer type. The affected examples are recipes containing the following modules (non-exhaustive list - this is what I've checked):

• Encode text
• Generate QR Code
• Generate Image

To Reproduce
To reproduce this issue, execute any of the following:

• curl -X POST -H "Content-Type:application/json" -d '{"input":"Test", "recipe":[{"op":"Encode text","args":["UTF-32BE (12001)"]}]}' localhost:3000/bake
• curl -X POST -H "Content-Type:application/json" -d '{"input":"Test", "recipe":[{"op":"Generate Image","args":["Greyscale",8,64]}]}' localhost:3000/bake
• curl -X POST -H "Content-Type:application/json" -d '{"input":"Test", "recipe":[{"op":"Generate QR Code","args":["PNG",5,4,"Medium"]}]}' localhost:3000/bake

From these tests, it could be concluded that any op that, by default, returns ArrayBuffer type could be affected.

Expected behavior
The expected behavior is receiving a result JSON, which returns the correct result.

The observed behavior is, in all these cases, receiving an empty result. The result looks like this:

{"value":{},"type":"ArrayBuffer"}

Notes

This happens even if the output type is explicitly stated, like so:
curl -X POST -H "Content-Type:application/json" -d '{"input":"Test", "recipe":[{"op":"Encode text","args":["UTF-32BE (12001)"]}],"outputType":"ArrayBuffer"}' localhost:3000/bake

However, the correct result is returned if another output type is selected, like so:
curl -X POST -H "Content-Type:application/json" -d '{"input":"Test", "recipe":[{"op":"Encode text","args":["UTF-32BE (12001)"]}],"outputType":"string"}' localhost:3000/bake

In this case, this is the returned result: {"value":"\u0000\u0000\u0000T\u0000\u0000\u0000e\u0000\u0000\u0000s\u0000\u0000\u0000t","type":"string"}

Node version:
v18.12.1

outputType is an optional parameter that can be specified when POSTing to /bake. When outputType is set as "file" or 7, the application should return a file.

Describe the solution you'd like

• if outputType=file in the request, the application should:

  • set Content-Disposition to attachment in the response
  • only return the dish value as the payload, so that if you pipe the response to a file, it should only contain the value.

• this behaviour should apply to multipart and body POSTs

For reference (with multipart at least), the response when outputType=7 is:

type: "FILE",
value: {
    data: {
        data: [
        84,
        104,
        101,
        32,
        99
        ],
        type: "Buffer"
    },
    lastModified: 1593784118511,
    name: "unknown",
    type: "application/unknown",
}

test for outputType = file is:

• expect a file of binary data out (e.g. png). Write it to x.png and open it. it should be valid.

It would be nice to have native Docker support.

I've submitted a PR for this here: #7

Describe the bug
When Choosing PNG as the output type when generating an image, unexpected data is returned. The tested modules include:

• Generate QR Code
• Generate Image

The returned data is not valid PNG data; instead, it appears to hold some intermediary encoder/decoder JavaScript code (see below).

To Reproduce
Execute any of the following:

• curl -X POST -H "Content-Type:application/json" -d '{"input":"Test", "recipe":[{"op":"Generate QR Code","args":["PNG",5,4,"Medium"]}], "outputType":"string"}' localhost:3000/bake
• curl -X POST -H "Content-Type:application/json" -d '{"input":"Test", "recipe":[{"op":"Generate Image","args":["Greyscale",8,64]}],"outputType":"string"}' localhost:3000/bake

Note: I'm forcing the "string" output type because of this bug I previously reported, but the same behavior can be observed for "File" and "List" types.

Expected behavior

The expected behavior is getting the result in some form of viewable PNG image, either binary PNG data or an HTML viewable representation of the PNG image.

The actual behavior is the return data containing this:

{"value":"\"use strict\";\nvar Buffer = require(\"safer-buffer\").Buffer;\n\n// Single-byte codec. Needs a 'chars' string parameter that contains 256 or 128 chars that\n// correspond to encoded bytes (if 128 - then lower half is ASCII). \n\nexports._sbcs = SBCSCodec;\nfunction SBCSCodec(codecOptions, iconv) {\n    if (!codecOptions)\n        throw new Error(\"SBCS codec is called without the data.\")\n    \n    // Prepare char buffer for decoding.\n    if (!codecOptions.chars || (codecOptions.chars.length !== 128 && codecOptions.chars.length !== 256))\n        throw new Error(\"Encoding '\"+codecOptions.type+\"' has incorrect 'chars' (must be of len 128 or 256)\");\n    \n    if (codecOptions.chars.length === 128) {\n        var asciiString = \"\";\n        for (var i = 0; i < 128; i++)\n            asciiString += String.fromCharCode(i);\n        codecOptions.chars = asciiString + codecOptions.chars;\n    }\n\n    this.decodeBuf = new Buffer.from(codecOptions.chars, 'ucs2');\n    \n    // Encoding buffer.\n    var encodeBuf = new Buffer.alloc(65536, iconv.defaultCharSingleByte.charCodeAt(0));\n\n    for (var i = 0; i < codecOptions.chars.length; i++)\n        encodeBuf[codecOptions.chars.charCodeAt(i)] = i;\n\n    this.encodeBuf = encodeBuf;\n}\n\nSBCSCodec.prototype.encoder = SBCSEncoder;\nSBCSCodec.prototype.decoder = SBCSDecoder;\n\n\nfunction SBCSEncoder(options, codec) {\n    this.encodeBuf = codec.encodeBuf;\n}\n\nSBCSEncoder.prototype.write = function(str) {\n    var buf = Buffer.alloc(str.length);\n    for (var i = 0; i < str.length; i++)\n        buf[i] = this.encodeBuf[str.charCodeAt(i)];\n    \n    return buf;\n}\n\nSBCSEncoder.prototype.end = function() {\n}\n\n\nfunction SBCSDecoder(options, codec) {\n    this.decodeBuf = codec.decodeBuf;\n}\n\nSBCSDecoder.prototype.write = function(buf) {\n    // Strings are immutable in JS -> we use ucs2 buffer to speed up computations.\n    var decodeBuf = this.decodeBuf;\n    var newBuf = Buffer.alloc(buf.length*2);\n    var idx1 = 0, idx2 = 0;\n    for (var i = 0; i < buf.length; i++) {\n        idx1 = buf[i]*2; idx2 = i*2;\n        newBuf[idx2] = decodeBuf[idx1];\n        newBuf[idx2+1] = decodeBuf[idx1+1];\n    }\n    return newBuf.toString('ucs2');\n}\n\nSBCSDecoder.prototype.end = function() {\n}\n\u0000\"use strict\";\nvar Buffer = require(\"safer-buffer\").Buffer;\n\n// Single-byte codec. Needs a 'chars' string parameter that contains 256 or 128 chars that\n// correspond to encoded bytes (if 128 - then lower half is ASCII). \n\nexports._sbcs = SBCSCodec;\nfunction SBCSCodec(codecOptions, iconv) {\n    if (!codecOptions)\n        throw new Error(\"SBCS codec is called without the data.\")\n    \n    // Prepare char buffer for decoding.\n    if (!codecOptions.chars || (codecOptions.chars.length !== 128 && codecOptions.chars.length !== 256))\n        throw new Error(\"Encoding '\"+codecOptions.type+\"' has incorrect 'chars' (must be of len 128 or 256)\");\n    \n    if (codecOptions.chars.length === 128) {\n        var asciiString = \"\";\n        for (var i = 0; i < 128; i++)\n            asciiString += String.fromCharCode(i);\n        codecOptions.chars = asciiString + codecOptions.chars;\n    }\n\n    this.decodeBuf = new Buffer.from(codecOptions.chars, 'ucs2');\n    \n    // Encoding buffer.\n    var encodeBuf = new Buffer.alloc(65536, iconv.defaultCharSingleByte.charCodeAt(0));\n\n    for (var i = 0; i < codecOptions.chars.length; i++)\n        encodeBuf[codecOptions.chars.charCodeAt(i)] = i;\n\n    this.encodeBuf = encodeBuf;\n}\n\nSBCSCodec.prototype.encoder = SBCSEncoder;\nSBCSCodec.prototype.decoder = SBCSDecoder;\n\n\nfunction SBCSEncoder(options, codec) {\n    this.encodeBuf = codec.encodeBuf;\n}\n\nSBCSEncoder.prototype.write = function(str) {\n    var buf = Buffer.alloc(str.length);\n    for (var i = 0; i < str.length; i++)\n        buf[i] = this.encodeBuf[str.charCodeAt(i)];\n    \n    return buf;\n}\n\nSBCSEncoder.prototype.end = function() {\n}\n\n\nfunction SBCSDecoder(options, codec) {\n    this.decodeBuf = codec.decodeBuf;\n}\n\nSBCSDecoder.prototype.write = function(buf) {\n    // Strings are immutable in JS -> we use ucs2 buffer to speed up computations.\n    var decodeBuf = this.decodeBuf;\n    var newBuf = Buffer.alloc(buf.length*2);\n    var idx1 = 0, idx2 = 0;\n    for (var i = 0; i < buf.length; i++) {\n        idx1 = buf[i]*2; idx2 = i*2;\n        newBuf[idx2] = decodeBuf[idx1];\n        newBuf[idx2+1] = decodeBuf[idx1+1];\n    }\n    return newBuf.toString('ucs2');\n}\n\nSBCSDecoder.prototype.end = function() {\n}\n\u0000Test\u0000\u0000éê\u001bc¼Ì0ZhxT\u0000\u0000\u0000\u0000\u0000\rIHDR\u0000\u0000 ... Unicode characters

Upon beautifying, the initial code looks like this (then it is repeated after a null byte and then the binary data starts):

"use strict";
var Buffer = require("safer-buffer").Buffer;

// Single-byte codec. Needs a 'chars' string parameter that contains 256 or 128 chars that
// correspond to encoded bytes (if 128 - then lower half is ASCII). 

exports._sbcs = SBCSCodec;
function SBCSCodec(codecOptions, iconv) {
    if (!codecOptions)
        throw new Error("SBCS codec is called without the data.")
    
    // Prepare char buffer for decoding.
    if (!codecOptions.chars || (codecOptions.chars.length !== 128 && codecOptions.chars.length !== 256))
        throw new Error("Encoding '"+codecOptions.type+"' has incorrect 'chars' (must be of len 128 or 256)");
    
    if (codecOptions.chars.length === 128) {
        var asciiString = "";
        for (var i = 0; i < 128; i++)
            asciiString += String.fromCharCode(i);
        codecOptions.chars = asciiString + codecOptions.chars;
    }

    this.decodeBuf = new Buffer.from(codecOptions.chars, 'ucs2');
    
    // Encoding buffer.
    var encodeBuf = new Buffer.alloc(65536, iconv.defaultCharSingleByte.charCodeAt(0));

    for (var i = 0; i < codecOptions.chars.length; i++)
        encodeBuf[codecOptions.chars.charCodeAt(i)] = i;

    this.encodeBuf = encodeBuf;
}

SBCSCodec.prototype.encoder = SBCSEncoder;
SBCSCodec.prototype.decoder = SBCSDecoder;


function SBCSEncoder(options, codec) {
    this.encodeBuf = codec.encodeBuf;
}

SBCSEncoder.prototype.write = function(str) {
    var buf = Buffer.alloc(str.length);
    for (var i = 0; i < str.length; i++)
        buf[i] = this.encodeBuf[str.charCodeAt(i)];
    
    return buf;
}

SBCSEncoder.prototype.end = function() {
}


function SBCSDecoder(options, codec) {
    this.decodeBuf = codec.decodeBuf;
}

SBCSDecoder.prototype.write = function(buf) {
    // Strings are immutable in JS -> we use ucs2 buffer to speed up computations.
    var decodeBuf = this.decodeBuf;
    var newBuf = Buffer.alloc(buf.length*2);
    var idx1 = 0, idx2 = 0;
    for (var i = 0; i < buf.length; i++) {
        idx1 = buf[i]*2; idx2 = i*2;
        newBuf[idx2] = decodeBuf[idx1];
        newBuf[idx2+1] = decodeBuf[idx1+1];
    }
    return newBuf.toString('ucs2');
}

SBCSDecoder.prototype.end = function() {
}

Notes:
Other types, like PDF and SVG, seem to be working fine.

Node version:
v18.12.1

Is your feature request related to a problem? Please describe.
I want to be able to copy & paste a deeplink from the CyberChef UI and use it to bake a recipe in CyberChef-server

Describe the solution you'd like

• Given some URL like https://gchq.github.io/CyberChef/#recipe=ROT13(true,false,14)&input=VGhyb3cgVGhyb3cgQnVycml0bw from the cyberchef ui
• I can take the "query" parameter section #recipe=ROT13(true,false,14)&input=VGhyb3cgVGhyb3cgQnVycml0bw and use it in cyberchef-server to give me the same result.

To deliver this we need to:

1. Make the Cyberchef node API accept chef-format recipe config
2. make the server parse the above type of input and use it in a bake operation.

Describe the bug
I get a status 500 every time I try submitting data > 197700. That's not a specific number, but just my closest frame of reference based on sources used.

To Reproduce
Attempt to bake with the following python function:

def query_cyberchef(source_data):
headers = {
'Content-Type': 'application/json',
}
encodedBytes = base64.urlsafe_b64encode(source_data.encode("utf-8"))
encodedStr = str(encodedBytes, "utf-8")
data = {"input":encodedStr, "recipe":[{ "op": "From Base64","args": ["A-Za-z0-9-_=", 'true'] },{ "op": "Remove whitespace","args": ['true', 'true', 'true', 'true', 'true', 'false'] },{ "op": "CTPH","args": [] }]}
response = requests.post('http://localhost:3000/bake', headers=headers, data=json.dumps(data))
return response

Expected behavior
I would expect a Status code 200 with the has as the CTPH in the response. This works on smaller files and I don't see anything other than the content-length that is different.

Node version:
14.15.5
Same results with 10.23.3

Additional context
Error: failed with status code 500
at ServerResponse.onResFinished (/home/redrobin/CyberChef-server/node_modules/pino-http/logger.js:51:33)
at ServerResponse.emit (events.js:327:22)
at ServerResponse.EventEmitter.emit (domain.js:467:12)
at onFinish (_http_outgoing.js:766:10)
at callback (internal/streams/writable.js:513:21)
at afterWrite (internal/streams/writable.js:466:5)
at afterWriteTick (internal/streams/writable.js:453:10)
at processTicksAndRejections (internal/process/task_queues.js:79:21)

Is your feature request related to a problem? Please describe.
Currently when you POST to /bake it returns a type and a value. If someone wanted to write the converted input to file, they would need to do some post-processing to extract the value from the returned data.

There could be a valuesOnly flag or property that the user could set, and then only the value would be returned.

Describe the solution you'd like
if valuesOnly is truthy
return body only contains dish.value
outputType is still honoured

Describe the solution you'd like
Pre-build container image like here

Describe the bug
When viewing http://localhost:3000 or http://$host:3000, the page is blank, and errors are noticed in Chrome Dev tools. The underlying server seems to work fine and respond to requests -- it's just the Swagger docs that are unviewable.

To Reproduce
Follow documented instructions to run CyberChef-server from the README ( I actually had to perform an npm run prod vs npm run to start the project).

Expected behavior
Swagger documentation is displayed.

Node version:
10.18.1

Additional context
I'm not seeing any CSS files in the CyberChef-server folder. I'm not terribly familiar with Node, so please forgive my ignorance with regard to how things are built/managed.

Chef currently rejects Cross Origin requests. Given that the intended use case is to be widely available resource for frontends to call on, it should be permissive with cross origin requests.

Is there any way to return the output of a base64 op as ASCII text vs integer values?

For example, if I do the following:

curl -XPOST -H "Content-Type: application/json" http://192.168.1.203:3000/bake -d '{"input":"dGVzdA==", "recipe":{"op":"From Base64"}}'

I get the following as the response:

[116,101,115,116]

How would I modify the request to receive test as the response?