mattosaurus / pgpcore Goto Github PK
View Code? Open in Web Editor NEW.NET Core class library for using PGP
License: MIT License
.NET Core class library for using PGP
License: MIT License
The DecryptFile function doesn't decrypt files that have been encrypted and signed using EncryptFileAndSign, the following error is returned.
Encrypted message contains a signed message - not literal data.
Hi @tkaradimitris, DecryptFileAndVerify is intended for decrypting and verifying encrypted and signed files which is why it requires both a public and private key. There's currently no functionality for extracting the clear signed content as I'd figured these message would mostly be being read by humans.
If this is something that you need then feel free to submit a PR if you're able, or if not I'll add it to my work list and get to it when I have a chance.
Thank you @mattosaurus for the answer and the great work. I am not sure what I need to use from BouncyCastle to extract content from signed files / streams, but if I manage to find examples from other solutions I will try to submit a PR.
Originally posted by @tkaradimitris in #40 (comment)
Hi.
I'm trying to encrypt/decrypt a large file (~7gb) but I'm getting a error
at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at Org.BouncyCastle.Bcpg.BcpgOutputStream.PartialFlush(Boolean isLast)
at Org.BouncyCastle.Bcpg.BcpgOutputStream.WritePartial(Byte[] buffer, Int32 off, Int32 len)
at PgpCore.Utilities.PipeStreamContents(Stream input, Stream pOut, Int32 bufSize)
at PgpCore.PGP.EncryptStream(Stream inputStream, Stream outputStream, IEnumerable`1 publicKeyStreams, Boolean armor, Boolean withIntegrityCheck)
at PgpCore.PGP.<>c__DisplayClass25_0.<EncryptStreamAsync>b__0()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
--- End of stack trace from previous location where exception was thrown ---
at PgpCore.PGP.EncryptStreamAsync(Stream inputStream, Stream outputStream, Stream publicKeyStream, Boolean armor, Boolean withIntegrityCheck)
at PgpProcessor.Encrypt(String pubKeyPath, String inputFilePath, String outputFilePath) in C:\AgodaGit\aes\dotnet\PgpProcessor.cs:line 15
at aes.Program.Main(String[] args) in C:\AgodaGit\aes\dotnet\Program.cs:line 33
at aes.Program.<Main>(String[] args)
Created a bufferSize to output buffer but didn't work
using System.IO;
using System.Threading.Tasks;
using PgpCore;
public class PgpProcessor
{
private const int bufferSize = 8 * 1024;
public static async Task Encrypt(string pubKeyPath, string inputFilePath, string outputFilePath)
{
using (PGP pgp = new PGP())
using (FileStream inputFileStream = new FileStream(inputFilePath, FileMode.Open))
using (Stream outputFileStream = File.Create(outputFilePath, bufferSize))
using (Stream publicKeyStream = new FileStream(pubKeyPath, FileMode.Open))
await pgp.EncryptStreamAsync(inputFileStream, outputFileStream, publicKeyStream, false, false);
}
public static async Task Decrypt(string privKeyPath, string password, string inputFilePath, string outputFilePath)
{
using (PGP pgp = new PGP())
using (FileStream inputFileStream = new FileStream(inputFilePath, FileMode.Open))
using (Stream outputFileStream = File.Create(outputFilePath, bufferSize))
using (Stream privateKeyStream = new FileStream(privKeyPath, FileMode.Open))
await pgp.DecryptStreamAsync(inputFileStream, outputFileStream, privateKeyStream, password);
}
}
Thanks!
Create IPGP interface object
Hey,
i'm tryng to encrypt a single file txt usign an .asc file whith the private key but the method ReadSecretKey generate an exception:
Org.BouncyCastle.Bcpg.OpenPgp.PgpPublicKeyRing found where PgpSecretKeyRing expected
The user gives me this .asc file but i don't know what's the password.
please some help
Such as with using gpg
in terminal.
gpg --output text.txt.gpg --encrypt --recipient [email protected] txt.txt
This issue is the reason of the behaviour in #44 .
The synchronous versions of the public methods (e.g. EncryptStream
) are just calls to the async version of that method (e.g. EncryptStreamAsync
) suffixed with .Wait()
.
Example:
EncryptStreamAsync(inputStream, outputStream, publicKeyStreams, armor, withIntegrityCheck).Wait();
You should never do this because it causes deadlocks, let me try to explain why:
Wait
will synchronously block until the task completes. So the current thread is literally blocked waiting for the task to complete. When the code reaches an await
statement, it returns an uncompleted task. When this Tasks completes, it wants to synchronize with the request context from where the Task is initiated. It will wait until this thread comes available. Because the Wait()
method blocks the current thread, it will never become available: deadlock.
As a general rule, you should use "async all the way down"; that is, don't block on async code.
What you should do is make the synchronous methods truly synchronous, or, remove them so the calling clients are forced to use the async methods.
There is another option though, but it is more or less a hack: suffix every async call with ConfigureAwait(false)
. This will prevent the task from synchronizing back with the current request context, avoiding a deadlock.
Below is my code, we genreated the pgp key pair using Go anywhere software, the code is reading the public key of the sender but giving error on reading my private key.The error is "Org.BouncyCastle.Bcpg.OpenPgp.PgpPublicKeyRing found where PgpSecretKeyRing expected". Please help its urgent.
PGPDecrypt test = new PGPDecrypt(@"d:\PGP_Keys\TestKeys\INT328.txt.pgp",
@"d:/PGP_Keys/TestKey/My secret key.asc",
"XXXX",
path1,
@"d:/PGP_Keys/TestKey/Sender public key.asc", true);
FileStream fs = File.Open(@"d:\PGP_Keys\TestKeys\INT328.txt.pgp", FileMode.Open);
test.decrypt(fs, path1);
fs.Close();
public class PgpEncryptionKeys
{
public PgpPublicKey PublicKey { get; private set; }
public PgpPrivateKey PrivateKey { get; private set; }
public PgpSecretKey SecretKey { get; private set; }
/// <summary>
/// Initializes a new instance of the EncryptionKeys class.
/// Two keys are required to encrypt and sign data. Your private key and the recipients public key.
/// The data is encrypted with the recipients public key and signed with your private key.
/// </summary>
/// <param name="publicKeyPath">The key used to encrypt the data</param>
/// <param name="privateKeyPath">The key used to sign the data.</param>
/// <param name="passPhrase">The (your) password required to access the private key</param>
/// <exception cref="ArgumentException">Public key not found. Private key not found. Missing password</exception>
public PgpEncryptionKeys(string publicKeyPath, string privateKeyPath, string passPhrase,bool toEncrypt)
{
if (!File.Exists(publicKeyPath))
throw new ArgumentException("Public key file not found", "publicKeyPath");
if (!File.Exists(privateKeyPath))
throw new ArgumentException("Private key file not found", "privateKeyPath");
if (String.IsNullOrEmpty(passPhrase))
throw new ArgumentException("passPhrase is null or empty.", "passPhrase");
PublicKey = ReadPublicKey(publicKeyPath);
SecretKey = ReadSecretKey(privateKeyPath, toEncrypt);
PrivateKey = ReadPrivateKey(passPhrase);
}
#region Secret Key
/// <summary>
/// Return the last key we can use to decrypt.
/// Note: A file can contain multiple keys (stored in "key rings")
/// </summary>
private PgpSecretKey GetLastSecretKey(PgpSecretKeyRingBundle secretKeyRingBundle)
{
return (from PgpSecretKeyRing kRing in secretKeyRingBundle.GetKeyRings()
select kRing.GetSecretKeys().Cast<PgpSecretKey>()
.LastOrDefault(k => k.IsSigningKey))
.LastOrDefault(key => key != null);
}
private PgpSecretKey ReadSecretKey(string privateKeyPath, bool toEncrypt)
{
using (Stream keyIn = File.OpenRead(privateKeyPath))
using (Stream inputStream = PgpUtilities.GetDecoderStream(keyIn))
{
PgpSecretKeyRingBundle secretKeyRingBundle = new PgpSecretKeyRingBundle(inputStream);
PgpSecretKey foundKey = toEncrypt ? GetFirstSecretKey(secretKeyRingBundle): GetLastSecretKey(secretKeyRingBundle);
// PgpSecretKey foundKey = GetFirstSecretKey(secretKeyRingBundle);
if (foundKey != null)
return foundKey;
}
throw new ArgumentException("Can't find signing key in key ring.");
}
/// <summary>
/// Return the first key we can use to encrypt.
/// Note: A file can contain multiple keys (stored in "key rings")
/// </summary>
private PgpSecretKey GetFirstSecretKey(PgpSecretKeyRingBundle secretKeyRingBundle)
{
foreach (PgpSecretKeyRing kRing in secretKeyRingBundle.GetKeyRings())
{
PgpSecretKey key = kRing.GetSecretKeys().Cast<PgpSecretKey>().Where(k => k.IsSigningKey).FirstOrDefault();
if (key != null)
return key;
}
return null;
}
#endregion
#region Public Key
private PgpPublicKey ReadPublicKey(string publicKeyPath)
{
using (Stream keyIn = File.OpenRead(publicKeyPath))
using (Stream inputStream = PgpUtilities.GetDecoderStream(keyIn))
{
PgpPublicKeyRingBundle publicKeyRingBundle = new PgpPublicKeyRingBundle(inputStream);
PgpPublicKey foundKey = GetFirstPublicKey(publicKeyRingBundle);
if (foundKey != null)
return foundKey;
}
throw new ArgumentException("No encryption key found in public key ring.");
}
private PgpPublicKey GetFirstPublicKey(PgpPublicKeyRingBundle publicKeyRingBundle)
{
foreach (PgpPublicKeyRing kRing in publicKeyRingBundle.GetKeyRings())
{
PgpPublicKey key = kRing.GetPublicKeys()
.Cast<PgpPublicKey>()
.Where(k => k.IsEncryptionKey)
.FirstOrDefault();
if (key != null)
return key;
}
return null;
}
#endregion
#region Private Key
private PgpPrivateKey ReadPrivateKey(string passPhrase)
{
PgpPrivateKey privateKey = SecretKey.ExtractPrivateKey(passPhrase.ToCharArray());
if (privateKey != null)
return privateKey;
throw new ArgumentException("No private key found in secret key.");
}
#endregion
}
I have a method that encrypts a Stream
:
public async Task<Stream> EncryptAsync(Stream streamToEncrypt, string publicKey) { using var pgp = new PGP(); Stream outputStream = new MemoryStream(); using (streamToEncrypt) using (var publicKeyStream = publicKey.ConvertToStream()) { await pgp.EncryptStreamAsync(streamToEncrypt, outputStream, publicKeyStream, true, true); outputStream.Seek(0, SeekOrigin.Begin); return outputStream; } }
It is being used by a web endpoint so encoding the encrypted data is likely the best way to reduce formatting issues etc. (e.g. Base64). What is a good way to encode the encrypted Stream
without hitting the constraints of multiple Byte[]
copies and MemoryStream
size constraints?
HI I am facing error Org.BouncyCastle.Bcpg.OpenPgp.PgpPublicKeyRing found where PgpSecretKeyRing expected.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using PgpCore;
using System.IO;
using System.Text;
namespace TEC_CitiEncryption
{
public class Class1
{
static void Main(string[] args)
{
using (PGP pgp = new PGP())
{
String search_file = @"C:\UAT Testing\file.txt";
String outputFile_file = @"C:\UAT Testing\Notfile.pgp";
String password = "1AS5D32eafsd5f3alasdkjf39ldk339j7yid9";
String public_File = @"C:\Users\syaphkg5axInstall\Documents\PGP Key Ankur\DLIP-PROD.asc";
String private_File = @"C:\Users\syaphkg5axInstall\Documents\PGP Key Ankur\DAN India IT AX TEAM.asc";
// Encrypt and sign file
pgp.EncryptFileAndSign(search_file, outputFile_file,public_File, private_File, password, true, true);
}
}
}
}
I am sure that I am passing the correct keys.
Update Arrange and Teardown methods to use uniquely named files to prevent access issues.
Have you thought about following Microsoft's guidance on providing strong-named assemblies for libraries? For some, strong-named libraries are required (e.g. COM objects), and StrongNamer does not work perfectly or in all scenarios. FYI, Portable.BouncyCastle is strong-named.
A tip to help those that strong-names causes issues: your AssemblyVersion does not have to change for each release. All 2.x.x.x releases can have AssemblyVersion 2.0.0.0, etc.
✔️ CONSIDER strong naming your library's assemblies.
✔️ CONSIDER adding the strong naming key to your source control system.
✔️ CONSIDER incrementing the assembly version on only major version changes to help users reduce binding redirects, and how often they're updated.
https://docs.microsoft.com/en-us/dotnet/standard/library-guidance/strong-naming
As suggested in #14, add a method to verify a signed file.
Hi, i can sign the stream
using (var inputStream = new MemoryStream(messageToSign))
using (var outputStream = new MemoryStream())
using (Stream privateKeyStream = new FileStream(privateKey, FileMode.Open, FileAccess.Read))
using (PGP pgp = new PGP())
{
await pgp.SignStreamAsync(inputStream, outputStream, privateKeyStream, pass);
return signature.ToArray();
}
but i cannot verify even with original message that it was signed.
i found out only
using (var inputStream = new MemoryStream(signature))
using (Stream publicKeyStream = new FileStream(publicKey, FileMode.Open, FileAccess.Read))
using (PGP pgp = new PGP())
{
return pgp.VerifyStreamAsync(inputStream, publicKeyStream);
}
the problem is that in this i only verify that signature is signed with that public key, and not that the original message (messageToSign) is signed in that signature..
what is the best practice to verify the signed message?
Hi,
First of all, great job on the library, it certainly takes the pain out of using the BouncyCastle APIs directly.
It looks like others have run into this issue but it seems that I can't sign and verify (+ encrypt/decrypt variants) when using keys generated using gpg. I am however able to encrypt and decrypt without signing/verifying.
I'm using the gpg command line tool and
gpg --generate-key
To generate the key pairs. Then the following commands to export the keys so I can use them in my test app:
gpg --export -a --openpgp [email protected] > test_pub.asc
gpg --export-secret-key -a --openpgp [email protected] > test.asc
I figured it may be a compatibility issue with PGP hence the --openpgp
flag but this doesn't seem to make a difference.
Keys generated with https://pgpkeygen.com/ do work as expected.
Also, on a different topic, would you be open to a PR that makes the library truly async as currently the sync calls are just being wrapped in a Task.Run?
If the contents of the encrypted stream isn't a valid encrypted value format, the decrypt function throws a unhelpful null reference error.
Example scenario: attempting to decrypt a file that is already decrypted.
Hi!
Could you add a license? Perhaps MIT, like ChoPGP?
Help me please.!! I want to example Decrypt not asking passPhrase [C#]
There is no documentation on DecryptStreamAndVerify nor any unit tests.
Hi,
I'm getting the Secret key for message not found when i try to decrypt a encrypt and signed file.
I have attach my project so you can try.
I have a private key which i have generated and a public key from my external partner where i have to send the encrypt and signed file.
Then encryption part works fine, but when I try to decrypt the encrypt and signed file I get the metioned error.
I have checked that i'm using the right private key, password and i'm using the same private key for decryption which I used to encrypt.
I have attached my C# project, just build it.
Try this flow:
PS. I have not attach the private and public key, because we use these keys in real test environment.
now
1.Point to the same private key
2. Start the console app
3. Press D for decryption
4. You will get the error.
Hope you can tell me why i'm getting this error.
Calling code:
using (var inputMemoryStream = new MemoryStream(Encoding.UTF8.GetBytes(unsignedString)))
using (var outputMemoryStream = new MemoryStream())
using (var privateKeyStream = new MemoryStream(Encoding.UTF8.GetBytes(key)))
{
pgp.SignStream(inputMemoryStream, outputMemoryStream, privateKeyStream, password);
signedString = (new StreamReader(outputMemoryStream)).ReadToEnd();
}
Given "foobar" for unsignedString, SignStream hangs at the point of writing out to literalOut, whereas unsignedString=string.Empty returns instantly (with string.Empty).
Am I doing anything wrong? No error decoding my key, which was my main reason for testing this before handing it real values. Could it be the key itself?
Add options for key type creation.
Hi.
I want to add a possibility to encrypt file with multiple public keys and sign it with one private key. Is it possible with PgpCore?
I am using this library to encrypt a zip file. The code is very simple:
using (PGP pgp = new PGP())
{
pgp.EncryptFile(zipFileFullname, encryptedFileFullname, publicKeysFileFullname);
}
It generally works well, but today I noticed that if I use an expired public key, it successfully encrypts the file regardless of the key being expired, without any warning. I have been looking for some parameters to be passed or some methods to call to verify if the keys are valid, but I didn't find anything.
Isn't the encryption supposed to fail if the encryption keys expired?
After hours of debugging, I've tracked down an inconsistency in this library compared to gpg (on Windows at least):
Any text that contains a line which ends in spaces (i.e. " \n", " \r\n" or similar) will produce a file which can be verified with PgpCore (i.e. VerifyClearStream
), but will fail validation with gpg (bad signature).
Example:
var text = "Test \nString";
using var inputStream = new MemoryStream(Encoding.ASCII.GetBytes(text));
using var outputStream = new MemoryStream();
using var keyStream = File.OpenRead("key.pgp");
var keyPassword = "<boing>";
var pgp = new PGP()
{
CompressionAlgorithm = Org.BouncyCastle.Bcpg.CompressionAlgorithmTag.Uncompressed,
FileType = PGPFileType.Binary,
HashAlgorithmTag = Org.BouncyCastle.Bcpg.HashAlgorithmTag.Sha512,
PublicKeyAlgorithm = Org.BouncyCastle.Bcpg.PublicKeyAlgorithmTag.RsaSign,
PgpSignatureType = 0
};
pgp.ClearSignStream(inputStream, outputStream, keyStream, keyPassword);
File.WriteAllBytes("borked.txt.asc", outputStream.ToArray());
var gpg = Process.Start("gpg", "--verify borked.txt.asc");
gpg.WaitForExit();
Console.WriteLine(gpg.ExitCode);
This will return 1 for any text
which has a line with trailing whitespaces, and 0 for those without. As far as I can tell, there have been some arguments in the PGP/GPG community in the past about how this should be handled (with Binary
as proposed solution to avoid this issue altogether), but none of the options in PgpCore seem to be able to create a clear signed file which can be verified by gpg.
I need this method
public async Task EncryptStreamAndSignAsync(Stream inputStream , Stream outputStream, Stream publicKeyStream, Stream privateKeyStream, string passPhrase, bool armor = true, bool withIntegrityCheck = true)
{
await Task.Run(() => EncryptStreamAndSign(inputStream, outputStream, publicKeyStream, privateKeyStream, passPhrase, armor, withIntegrityCheck));
}
Hi @mattosaurus ,
Thanks for the great library. Trying to setup sample provided on README.md using the same code. Found below issues. Can you guide me on fixing them?
Code:
pgp.GenerateKey(@"C:\TEMP\keys\public.asc", @"C:\TEMP\keys\private.asc", "[email protected]", "password");
Issues:
Sample public key(extension renamed to txt for Github to support upload):
public.asc.txt
Sample private key(extension renamed to txt for Github to support upload):
private.asc.txt
When generating a key, if the out-file already exists then the output file can become malformed.
pgp.GenerateKey("public.txt", "private.txt", "username", "password", 1024);
pgp.GenerateKey("public.txt", "private.txt", "username", "password", 512);
This example generates a long key, then a shorter key to the same file. The resulting file:
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: BCPG C# v1.8.5.0mEsEXc185AECAKdAfTMUCLXi2vRsz0RwOx2OMJuDUAS69kMyYxAYpZkorM/wqoTs
DkL3/X3Eqm0NNqN6BU9EUPzOFord9Le/uTMABRO0CHVzZXJuYW1liFwEEAECAAYF
Al3NwzQACgkQBD+fFOKPU+jIXAH9GyYQCEExNSBoBt4D0yiNi4fb1hjppp0CwfNI
EpHQ5cGgAKzcJ1ks1qlvOzLJfwBkDknGN1uxsRJ7G154vhFnJA==
=gfJE
-----END PGP PUBLIC KEY BLOCK-----
N7Uml6jYu4LJ3id/USTkFBuUOLwnndcsA
/vUtA9PiYg4uFIICGRBvW9EzS/QDgUK8FfJofDXNs2xBMUBiZJV2HBjnLZByC4X3
8BXwbc/NcnhR83ND9xAAw4f/YrpL
=vu/P
-----END PGP PUBLIC KEY BLOCK-----
I would expect that writing the key to the file would use File.WriteAllText() or something similar which overwrites the whole file.
I found where I would make an adjustment:
Lines 1112 to 1113 in 31479ef
Hello @mattosaurus ,
Is it possible to return the decrypted data as a Stream (accessing outputStream) rather than as a file as you did in the test case?
// Test File example
using (FileStream inputFileStream = new FileStream(contentFilePath, FileMode.Open))
using (Stream outputFileStream = File.Create(encryptedContentFilePath))
using (Stream publicKeyStream = new FileStream(publicKeyFilePath1, FileMode.Open))
using (Stream privateKeyStream = new FileStream(privateKeyFilePath1, FileMode.Open))
pgp.EncryptStreamAndSign(inputFileStream, outputFileStream, publicKeyStream, privateKeyStream, password1);
using (FileStream inputFileStream = new FileStream(encryptedContentFilePath, FileMode.Open))
using (Stream outputFileStream = File.Create(decryptedContentFilePath1))
using (Stream privateKeyStream = new FileStream(privateKeyFilePath1, FileMode.Open))
pgp.DecryptStream(inputFileStream, outputFileStream, privateKeyStream, password1);
string decryptedContent = File.ReadAllText(decryptedContentFilePath1);
//My attempt at accessing the stream directly
//outputStream is null
Stream outputStream = new MemoryStream();
Stream privateKeyStream = GenerateStreamFromString(privateKey);
var length = privateKeyStream.Length;
var length2 = req.Body.Length;
using (PGP pgp = new PGP())
{
using (Stream inputStream = req.Body)
using(outputStream)
using (privateKeyStream)
{
inputStream.Seek(0, SeekOrigin.Begin);
privateKeyStream.Seek(0, SeekOrigin.Begin);
outputStream.Seek(0, SeekOrigin.Begin);
pgp.DecryptStreamAsync(inputStream, outputStream, privateKeyStream, passPhrase);
log.LogInformation($"decryption complete");
outputStream.Seek(0, SeekOrigin.Begin);
log.LogInformation($"seek complete");
var length3 = outputStream.Length;
log.LogInformation($"check stream: {length3}");
StreamReader reader = new StreamReader(outputStream);
string decryptedData = reader.ReadToEnd();
log.LogInformation($"read complete: {decryptedData}");
dynamic data = JsonConvert.DeserializeObject(decryptedData);
log.LogInformation($"convert complete:{data}");
return data != null
? (ActionResult)new OkObjectResult(data)
: new BadRequestObjectResult("Data not found");
//return outputStream;
}
}
Current standard is 2048. there should probably be a workaround that I am not aware.
Feature request:
A way to get a public key's fingerprint from a public-key stream or file.
Hello,
I'm starting C# development so i'm very bad.
This is my code :
PGP pgp = new PGP();
Stream inputFileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("test_basic_messsage"));
Stream outputFileStream = new MemoryStream();
Stream publicKeyStream = new FileStream(Utils.app_directory + @"keys\public.asc", FileMode.Open);
pgp.EncryptStream(inputFileStream, outputFileStream, publicKeyStream);
StreamReader reader = new StreamReader(outputFileStream);
string text = reader.ReadToEnd();
MessageBox.Show(outputFileStream.Length.ToString()); //Messagebox to see outputFileStream length (it display 890)
MessageBox.Show(text); //Messagebox to see pgp Message (it display nothing)
MessageBox.Show(text.Length.ToString()); //Messagebox to see pgp message length (it display 0)
I'm trying to store a pgp message in a string variable but i cannot, with this code, outputFileStream.Length
equals to 890 but reader.ReadToEnd()
equals to nothing.
But when i replace Stream outputFileStream = new MemoryStream();
by Stream outputFileStream = File.Create(Utils.app_directory + @"keys\content.pgp")
it's working because i see the pgp message in the content.pgp file...
Sorry for my bad english, i'm french...
Hi, is there any possibility to add encrypt/decrypt memory stream methods in current realization?
Thanks)
Hi, I am trying to decrypt a Stream using an Azure function app in which the function is http triggered and the data to be decrypted is passed into req.body.
The passphrase and key are stored in a vault, I have verified that they work using gpg4win, and that the app can access them and they print out correctly.
Can you help me figure out why I am seeing the error "Secret Key for message cannot be found" when the key, passphrase, and data are being passed correctly?
using System.IO;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using PgpCore;
using System.Threading.Tasks;
using Microsoft.Azure.KeyVault.Models;
using System.Net.Http;
using Microsoft.Azure.Services.AppAuthentication;
using Microsoft.Azure.KeyVault;
using System;
using System.Text;
using System.Collections.Concurrent;
using Microsoft.Extensions.Logging;
namespace AzureFunctionsPGPDecrypt
{
public static class PGPDecrypt
{
private static readonly HttpClient client = new HttpClient();
private static ConcurrentDictionary<string, string> secrects = new ConcurrentDictionary<string, string>();
[FunctionName(nameof(PGPDecrypt))]
public static IActionResult Run(
[HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]
HttpRequest req, ILogger log)
{
log.LogInformation($"C# HTTP trigger function {nameof(PGPDecrypt)} processed a request.");
string privateKey = Environment.GetEnvironmentVariable("secretkey", EnvironmentVariableTarget.Process);
string passPhrase= Environment.GetEnvironmentVariable("secretphrase", EnvironmentVariableTarget.Process);
log.LogInformation($"SecretKey: {privateKey}");
log.LogInformation($"SecretPassPhrase: {passPhrase}");
if (privateKey == null)
{
return new BadRequestObjectResult("Please pass a private key secret identifier on the query string");
}
Stream outputStream = new MemoryStream();
Stream privateKeyStream = GenerateStreamFromString(privateKey);
var length = privateKeyStream.Length;
var length2 = req.Body.Length;
log.LogInformation($"Length private key stream: {length}");
log.LogInformation($"Length body: {length2}");
//verifying that the streams were not empty
using (PGP pgp = new PGP())
{
using (Stream inputStream = req.Body)
using(outputStream)
using (privateKeyStream)
{
pgp.DecryptStreamAsync(inputStream, outputStream, privateKeyStream, passPhrase);
return new OkObjectResult(outputStream);
}
}
}
private static Stream GenerateStreamFromString(string s)
{
MemoryStream stream = new MemoryStream();
StreamWriter writer = new StreamWriter(stream);
writer.Write(s);
writer.Flush();
stream.Position = 0;
return stream;
}
}
}
I'm trying to PGP decrypt a very large file via an Azure function using PgpCore, however whilst the output is processed quite quickly for test files (for example a 512Mb file gets decrypted in less than a minute) decrypting very large files (for example 20GB) fails.
We have a timeout on the Azure function of 6 hours, but the process never seems to complete.
The decryption code is quite simple:
using (Stream inputFileStream = await storageAccess.OpenFileStreamAsync(container, encryptedFilePath))
{
using (Stream outputFileStream = await storageAccess.OpenWritableStreamAsync(container, outputFilePath))
{
using (Stream privateKeyStream = await GetPrivateKeyStream())
{
using (PGP pgp = new PGP())
{
logging.LogInformation($"Decrypting {encryptedFilePath} to {destinationDirectory}");
pgp.DecryptStream(inputFileStream, outputFileStream, privateKeyStream, password);
logging.LogInformation($"Decryption complete for {encryptedFilePath}");
}
}
}
}
Where OpenWritableStreamAsync returns a blockBlob.OpenWriteAsync() and OpenFileStreamAsync returns a blockBlob.OpenReadAsync()
I've looked and I can't find any reference of the maximum size file that can be processed, let alone how to change this, or any examples of decrypting using buffered input streaming.
Is there something failing internally here, or is it just a case of taking over 6 hours or filling the available memory?
Thanks
When using "EncryptyStreamAndSign" or "EncryptFileAndSign" in Azure Functions I am hitting an error,
[Error] Executed 'Functions.HttpTrigger1' (Failed, Id=6486b525-e10f-4571-b721-85ef66ffecc8)
Org.BouncyCastle.Bcpg.OpenPgp.PgpPublicKeyRing found where PgpSecretKeyRing expected
I don't know how to fix this?
As far as I can tell I am using BouncyCastle version 1.8.2
Break Arrange and Teardown methods out of test files into TestFactory
We are using DecryptStreamAsync()
to decrypt a FileStream into a MemoryStream but when we do this with a 24MB FileStream we end up with a 1GB MemoryStream. However when decrypt a smaller FileStream (ie. 7.4KB) we end up with a similar-sized MemoryStream. What would cause this?
Attached are screenshots of the variables in question. encryptedData
is the FileStream and result
is the MemoryStream.
Thanks!
Hi, I'm sorry if this is a repeat of issue #6 but I just wanted to clarify.
Does PgpCore have the ability to decrypt signed files? In issue #6 it specifies that it cannot decrypt files that were created from the EncryptFileandSign method, but does that extend to any files that are encrypted and signed period?
In your read me file you provide an example of decrypting a signed file but it doesn't show where it would acquire reference to the public key so I figured it couldn't hurt to ask.
Thanks in advance for your insight.
Add more examples into readme rather than just relying on tests.
Not too sure if anyone have experience this, I've tried to test if the PGP works on large files using the settings below.
using (PGP pgp = new PGP()
{
CompressionAlgorithm = Org.BouncyCastle.Bcpg.CompressionAlgorithmTag.Uncompressed,
SymmetricKeyAlgorithm = Org.BouncyCastle.Bcpg.SymmetricKeyAlgorithmTag.Aes256,
PublicKeyAlgorithm = Org.BouncyCastle.Bcpg.PublicKeyAlgorithmTag.RsaSign,
HashAlgorithmTag = Org.BouncyCastle.Bcpg.HashAlgorithmTag.Sha512,
FileType = PGPFileType.Binary,
PgpSignatureType = PgpSignature.BinaryDocument
})
So far for smaller files (<1GB) I'm getting exactly same size after the encrypt and decrypt process. but for 1 of my sql server db backup that is sized at 6GB, there's a 53KB mismatched, and resulting in a corrupted file.
Make async methods properly async rather than just wrapping synchronous actions.
I'm using the following code for signing a file:
using PgpCore;
using System;
using System.IO;
namespace GpgTest
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Begin");
string basePath = "";
string fileBare = Path.Combine(basePath, @"testData\content.txt");
string fileEncrypted = Path.Combine(basePath, @"testData\content.txt.encrypt.pgp");
string fileSigned = Path.Combine(basePath, @"testData\content.txt.signed.pgp");
Directory.CreateDirectory("keys");
Directory.CreateDirectory("testData");
File.WriteAllText(fileBare, "MyTestContent");
using (PGP pgp = new PGP())
{
string KeyPub = Path.Combine(basePath, @"keys\public.asc");
string KeyPriv = Path.Combine(basePath, @"keys\private.asc");
string KeyPasswd = @"password";
// Generate keys
pgp.GenerateKey(KeyPub, KeyPriv, "[email protected]", KeyPasswd);
pgp.SignFile(fileBare, fileSigned, KeyPriv, KeyPasswd, true, true);
}
Console.WriteLine("Done");
}
}
}
This produces a file with the following contents:
-----BEGIN PGP MESSAGE-----
Version: BCPG C# v1.8.5.0
kA0DAAIBR2nEQUeYkD4Byx5iC2NvbnRlbnQudHh0XZNmFE15VGVzdENvbnRlbnSI
rwQAAQIAGQUCXZNJ9BIcZW1haWxAZXhhbXBsZS5jb20ACgkQR2nEQUeYkD4+JgP/
abO2vPjGj03J9AZe1OenyPdm/l+qN3xh++no8lqTW2t/KInhE8bnQyo2bNXLB+l0
FWt3AQRfMQYPRZ2wn6jcC+MolYc9dTQYbccsjv1nEeLOi5IbJ+i/Lk7h1B0EV2xU
xGXKNUGrs26eFVeFdLFkuZfnaHT+TVV1W3vYmpUPO6I=
=KIlz
-----END PGP MESSAGE-----
I would expect a file like the following (Signature/ Structure taken from http://apt.postgresql.org/pub/repos/apt/dists/bionic-pgdg/InRelease):
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
MyTestContent
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEuXsK/KoaR/BE8kSgf8x9RqzMTPgFAl2Mt24ACgkQf8x9RqzM
TPiFhw/8DM18ObaqnkX50G/T05EF4fTvbbfDIjSpHOP6eNCThC+FViuL74gcfSTv
FW2JtozWHo9Bm4V2AnoNCYSdc3cXIMcq8DVtaSY/ccR4s3CT6yo5oDKZT7/l5fkK
RJLlQuQdJzUKHEAzGx0jZQqY/E/jtNqS+BvIYlpPjwk3BLEzW4jqf5Lokf8eEgM5
ffnRE1plf8I8ciDcrUWsaQKGCPDAMFraqriT7pkB7y3rjrP77oFtU3VsZBUc2JS5
29A9/6fTLIGwSjcbXksQ88FXzENvE+MqIR4zojGLrg/vOlU08FjUuZUkAAhdQ0G6
PLlOBhn90JypFTfADfaGKK4DzUX1xOhQhbNNwXTDimkwjLYJb0waeen8vI0F4nnh
4I75fmBC4/mLLL0WnOYV3f12bFuZrA77+ispeUfsE24ZYiG68sVri7P1IWfAtMoZ
TaIWKblUZ2ZLOb223qEJNVvGGxuiJe3hXeifsFG4m/b9IPdiU8ZMe5knLbjNz3Sg
Plax2VVyESWA4kkol7TwhsjkfN1Mj4g5hrD6VckQ4M8UaXr2EvxpbjnIWMGrW7WM
fCOwiBfqiH9GMvTLpbgy/frMbMr/7Oc/4EOVLzHfid5yNd74aE5Z8rPUQhjNm1ah
j38FiIajeGq9/sFc9Xs2L6CNotaPNo3F0XcJpQLAXEPHwbID23k=
=GYF1
-----END PGP SIGNATURE-----
Looking at the ArmoredOutputStream from the underlying library, it is capable to create files with that structure:
https://github.com/bcgit/bc-csharp/blob/7248688e6f513cbdde1ccc1d39904cb964b0c88a/crypto/src/bcpg/ArmoredOutputStream.cs#L196
DoWrite("-----BEGIN PGP SIGNED MESSAGE-----" + nl);
case PacketTag.Signature:
type = "SIGNATURE";
break;
Even if the current verison of the library should only save the signature (I do not get it from the function itself), this file should begin with "BEGIN PGP SIGNATURE" instead of "BEGIN PGP MESSAGE". But as the file is always larger than the encrypted one, I would suspect, that the content signature are inside the file.
Decryption fails with error when I try to decrypt a file that has binary contents instead of the normal -----BEGIN PGP MESSAGE----- text file. I have checked the private key and public key are correct, also tried to decrypt it with openpgp js module with binary option and it worked there. But doesn't work with PgpCore, kindly help me out, I am using this in Azure function. See code below. Thanks.
public virtual async Task Decrypt(Client client, Stream blobStream, CloudBlobStream decryptedStream, ILogger log)
{
try
{
using (PGP pgp = new PGP())
{
Stream privateKeyStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(client.PgpConfig.PrivateKey));
log.LogInformation($"::::::x {pgp.FileType.ToString()}"); // this prints binary for all files
await DecryptStreamAsync(pgp, blobStream, decryptedStream, privateKeyStream, client.PgpConfig.Passphrase);
await decryptedStream.CommitAsync();
}
}
catch (Exception ex)
{
log.LogError($"Error while decrypting file {ex.Message} {ex.StackTrace}");
exception = ex;
}
}
public virtual async Task DecryptStreamAsync(PGP pgp, Stream stream, CloudBlobStream decryptedStream, Stream privateKeyStream, string passphrase)
{
await pgp.DecryptStreamAsync(stream, decryptedStream, privateKeyStream, passphrase);
}
Error while decrypting file unknown object in stream Reserved at Org.BouncyCastle.Bcpg.OpenPgp.PgpObjectFactory.NextPgpObject() in D:\a\1\s\crypto\src\openpgp\PgpObjectFactory.cs:line 119 at Org.BouncyCastle.Bcpg.OpenPgp.PgpObjectFactory.AllPgpObjects() in D:\a\1\s\crypto\src\openpgp\PgpObjectFactory.cs:line 136 at PgpCore.PGP.DecryptAsync(Stream inputStream, Stream outputStream, Stream privateKeyStream, String passPhrase) at PgpCore.PGP.DecryptStreamAsync(Stream inputStream, Stream outputStream, Stream privateKeyStream, String passPhrase) at SofyImportV3.common.PgpDecrypter.DecryptStreamAsync(PGP pgp, Stream stream, CloudBlobStream decryptedStream, Stream privateKeyStream, String passphrase) in /Users/kamesh/Desktop/Sofy/SofyUnit/azure_functions/SofyImportV3/common/PGPDecrypter.cs:line 166 at SofyImportV3.common.PgpDecrypter.Decrypt(Client client, Stream blobStream, CloudBlobStream decryptedStream, ILogger log) in /Users/kamesh/Desktop/Sofy/SofyUnit/azure_functions/SofyImportV3/common/PGPDecrypter.cs:line 131
Currently BouncyCastle.NetCore nuget is not official and don't have functions like
PgpUtilities.WriteFileToLiteralData
and DotNetUtilities.ToRSAParameters
Please change this dependency to official Portable.BouncyCastle.
Used the following code fragment to perform a test of PgpCore. It fails with, "Org.BouncyCastle.Bcpg.OpenPgp.PgpException: 'Checksum mismatch at 0 of 2'" when attempting to decrypt the file.
// Generate keys
pgp.GenerateKey(@"C:\TEMP\keys\public.asc", @"C:\TEMP\keys\private.asc", "[email protected]", "passwo0d");
// Encrypt file
pgp.EncryptFile(@"sample.pdf", @"sample_pdf_encrypted.pgp", @"C:\TEMP\keys\public.asc", true, true);
// Decrypt file
pgp.DecryptFile(@"sample_pdf_encrypted.pgp", @"sample_pdf_UNencrypted.pgp", @"C:\TEMP\keys\private.asc", "passw0rd");
Add an internal keystore to PGP allowing keys to be stored on initialisation.
How to choose desired keys if multiples allowed?
I've tried utilizing this library within Azure functions and having issues with the encryption methods for Streams.
When I EncryptStream, the output does not seem to be formatted correctly because i am unable to decrypt, getting an exception because PgpObjectFactory.NextPgpObject is null.
If I utilize EncryptFile and DecryptFile everything works.
Even if I EncryptFile and then convert that to a stream and DecryptStream, it works.
I'm going to use this in Azure Functions integrated with Azure storage for the files and Azure KeyVault for my pgp keys/passphrase/etc so the stream methods are what I want to get working.
Any idea what my issue is?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.