matheusgomes28 / base64pp

A modern C++ implementation of Base64 encoding and decoding.

Hello, I'm Mat!

I blog about coding things. I also like to contribute to cool open-source projects out there!

• 🔭 I’m currently working on improving the quality of popular git repos I have, and beefing up the content on my website & Youtube channel.
• 🌱 I’m currently learning cloud development, advanced CI/CD, ML things.
• 👯 I’m looking to collaborate on anything that peaks my interest!
• 🤔 I’m looking for help with maintaining my Base64 C++ implementation.
• 💬 Ask me about C++ or anything coding related.
• 📫 How to reach me: I'm always online on my discord channel!
• 😄 Pronouns: he/him
• ⚡ Fun facts: I was born in Sao Paulo, Brazil, and lived there for 12 years. I'm getting into DJ-ing with my new Pioneer ddj!

Contribution Chart

base64pp's Issues

Provide alias for "run-clang-tidy-13" -> "run-clang-tidy"

Minor improvement on the Docker image so that we can call run-clang-tidy instead of run-clang-tidy-13 on the static-analysis.sh script. This would make it possible for users of the library to just run that script as well.

TODOs 👍

• Modify Dockerfile to provide alternative alias for run-clang-tidy-13 📦
• Modify the static-analysis.sh script to call run-clang-tidy now. 🔢

Add coverage to tests

Add coverage reporting to unit tests. Difficult task!

TODOS 📑

• Compile with --coverage in gcc. 📔
• Integrate gcov and lcov to read the profiling data. 👓
• Deploy coverage HTML to artifacts. 📫

Improve the README.md build instructions

As #4 recently fixed the CI, and #5 started the upgrade of docker images, we now have:

• Conan 2.x in the build.
• Doxygen documentation automatic builds.

More specifically, the conan build documentation is out of date! So this needs to be reflected in the documentation.

Abstract ugly powershell artifact code into script

The following raw snippet feels out of place in the main YAML file

      - pwsh: |
          .\scripts\build_win_x64.ps1
          if(!$?) { Exit $LASTEXITCODE }
          New-Item -ItemType Directory -Force -Path "$(Build.SourcesDirectory)/artifacts"
          7z a -ttar "$(Build.SourcesDirectory)/artifacts/build_win_64.tar"  "build/"
          if(!$?) { Exit $LASTEXITCODE }
          7z a -tgzip "$(Build.SourcesDirectory)/artifacts/build_win_64.tar.gz" "$(Build.SourcesDirectory)/artifacts/build_win_64.tar"
          if(!$?) { Exit $LASTEXITCODE }
        displayName: 'Compile Code & Tests'
        workingDirectory: $(Build.SourcesDirectory)
        failOnStderr: true

Research a way to overcome all these if(!$?) { Exit $LASTEXITCODE } checks.

[CI/CD] Make sure coverage artifact upload happens even when coverage threshold fails

Make sure that the coverage artifact upload fails even when the coverage threshold fails, because we need to see the coverage report anyway.

This example shows that we cannot access coverage when the threshold check fails on the job before.

Add display names to the docs and testing stage jobs

The current pipeline doesn't have names for the jobs in the {docs, test} stages:

TODOS 👍

1. Add displayName field to jobs in the docs stage. 📖
2. Add displayName field to jobs in the tests stage. 🧪

Fix the inheritance of CMAKE_CXX_FLAGS in CMakePresets.json

As I casually looked at the presets, I realised that the union of the CMAKE_CXX_FLAGS never happens between presets and their parent's variables.

I have no idea what happens if two preset parents (like unix-shared and unix-rel) define the same variable, but it definitely does not unify them.

Perhaps use something like the CXX_FLAGS_INIT in one parent, and the normal CXX_FLAGS in the other, that way they will combine at configuration time, to produce the intended flags. For example, the unix-shared-rel should have flags -fPIC as well as -O3 ... .

Add pipeline `clang-tidy` job. 🧹

Previously I thought that adding the clang-tidy script to the failfast stage would be a good idea in #28 .

However, clang-tidy can take a long time to run. In addition, it also depends on the compile_commands.json generated from the build stage.

TODOs 👍

• Generate compile_commands.json from the CMake build stage.
• Add clang-tidy checking.

Fix pipeline so only PRs trigger a run

Currently, when a PR is up, a push to the PR branch will trigger a pipeline run, and so will the PR itself. This causes two pipelines to run on the same code:

Ideas on fixing this:

• Only trigger pipeline runs on PRs 🛑
• Check if Azure pipelines can spot whether the commit is in a PR, if it is, only run PR pipeline 👍

Add support for building a DLL / shared lib

Add CMake flags for switching between library types:

• Build static by default.
• Build a shared library if BUILD_SHARED is defined.
• Expose functions if building shared lib.
• Use -fvisibility=hidden and -fvisibility-inlines-hidden on gcc builds.

Write CMakePresets.json To Improve CI & User builds

Since we're using a recent enough version of CMake, we can bring in the CMakePresets.json support for the CI pipelines & also to make it easier for users to build this project locally.

You can read more about the CMakePresets.json documentation here.

Presets TODOs 📖 👍

• support unix builds with flags -Werror, -Wall, -Wextra, & -pedantic. 🐧
• support ninja builds with flags -Werror, -Wall, -Wextra, & -pedantic. 🪟 🐧
• support windows builds with vs2019 and flags /W4, /Wx, and /Wall. 🪟
• support windows builds with vs2022 and flags /W4, /Wx, and /Wall. 🪟

All the builds above should have {Debug,Release} settings.

Add Shared Library Build to CI

Build the shared library version of base64pp on the CI.

TODOs:

• Call the Windows build template with extra -DBUILD_SHARED enabled.
• Install the library and push artifacts.

Enable -Werror and /Wx flags in the pipeline

Enable warnings as errors as the CMakePresets.json was recently introduced in #16 .

TODOS 📖

1. Add -Werror to the *-unix-* presets in CMakePresets.json 🐧
2. Add /Wx to the vs* presets in CMakePresets.json 🪟
3. Make sure all compilation errors are fixed in the pipeline ⚒️

Add Windows Build & Tests To Pipeline

We're currently only building for Ubuntu 18.04 in the pipeline. Would be ideal to build on MSVC compilers to catch other potential errors and bugs.

TODOS:

• Figure out what image to use. These are the azure available ones. 💻
• Create Windows build stage. 🪟
• Use chocolatey to install conan on pipeline. 🍫
• Create Windows test stage. 🧪

Change the GoogleTest target names to lowercase in the CMake

In tests/CMakeLists.txt, change the line

 target_link_libraries(base64pp_tests PRIVATE base64pp GTest::GTest)

to

 target_link_libraries(base64pp_tests PRIVATE base64pp gtest::gtest)

so we confirm to the standards around the repo. 💯

Remove unused "ninja" presets

Currently, we're not using the ninja* {configure,build}` presets anywhere.

TODOS: 📖

1. Remove the ninja[-windows] configure presets from CMakePresets.json.
2. Remove the ninja[-windows] build presets from CMakePresets.json.

add std::string / std::string_view overload to base64pp::encode()

As base64 is often used to encode text data, it's likely that the input data is present in a std::string.
Encoding data from a std::string with base64pp currently looks like this:

std::string str = "Hello World";
std::string encoded = base64pp::encode({reinterpret_cast<std::uint8_t const *>(str.data()), str.size()});

With a proper overload, it could be simplified like that:

std::string str = "Hello World";
std::string encoded = base64pp::encode(str);

Don't need to tar.gz all the artifacts

Azure pipeline already zips the artifacts in the published artifact directory, gziping them will not improve the compression.

TODOS 📑

• Remove tar and gz calls in the Ubuntu builds before publishing artifacts.
• Remove 7zip calls in the windows builds before publishing artifacts.
• Remove tar and gz calls to decompress artifacts.

Run tests with AddressSanitizer

Make sure that the unit tests are running with ASan, to catch any potential leaks / overflows.

TODOS:

• Wait for Windows builds 🪟
• Add debugging builds, so we can print stack trace on asan errors 🧪
• Make sure unit tests are running with ASan 🖥️

Resources:

1. Article on using address sanitizer in gcc.
2. Address sanitizer for Windows builds.

Integrate Conan dependencies with CMAKE_TOOLCHAIN_FILE

Instead of using the current CMAKE_MODULES_PATH, let's use CMAKE_TOOLCHAIN_FILE to conform with modern conan & vcpkg ways of integration.

This should also make it easier to use vcpkg i n the future, as it's the same tolchain mechanism. 🤓

Is this a typo?

see this line, should it be below?

-   if ((c >= 'a') && ('z'))
+   if ((c >= 'a') && (c <= 'z'))

Add Fuzz testing to encoder API

Fuzz test encoding and decoding API's. Perhaps some bugs will be caught!

A good, lightweight fuzzing alternative is LibFuzzer, which required building with Clang.

Install the library files and headers with CMake

Need to install the library with CMake, so users can call cmake install

Suggested structure:

• ${INSTALL_DIR}/lib holds the library files (.a, .lib). 📖
• ${INSTALL_DIR}/include holds the public header files. 🎧

Relevant video: Deep CMake for library writers.

Add CI badges to README.md

So far, we can add:

• Azure badge
• CodeCov

Remove unused VS2019 Presets

We're not using the "vs2019" presets anywhere. Remove them from CMakePresets.json

TODOs: 📖

1. Remove the vs2019 configure presets from CMakePresets.json
2. Remove the vs2019 build presets from CMakePresets.json

Remove absolute paths from pipeline's conan install script

Change line

sed -i "s/build_type=Release/build_type=${BUILD_TYPE}/g" /home/vsts_azpcontainer/.conan2/profiles/default

to something like

sed -i "s/build_type=Release/build_type=${BUILD_TYPE}/g" "$(conan profile show)"

Create a build template for the Pipeline

Create build template for the pipeline build jobs. Most of them are the same, the only thing that differs is the BUILD_TYPE for cmake and conan.

TODOS 👍

• Create template build for Ubuntu
• Create template build for Windows

Sort our the artifacts pushed for the build stages

Make sure that each artifact for each stage only has the files that concern that stage.

TODOS 👍

• Make sure docs stage only publish the docs, not the build files as well. 📜
• Make sure windows build stages only output the .tar.gz file. 🪟

Library does not decode unpadded base64

Decoding eyJuYW1lIjoiSm9obiBEb2UifQ does not work because the validation checks if the length is %4.
Since the string is unpadded (which seems to be optional - haven't read the spec) the library fails to decode the string.

Passing it through
https://base64.guru/tools/repair

reveals the padded value which is decodable.
eyJuYW1lIjoiSm9obiBEb2UifQ==

I assume the library should either pad the value or handle non %4 length inputs.

Add CMake install step to CI

After adding the CMake installation support in #77, make sure that works properly in the CI. This could be the start of a deployment task, where the pipeline releases the installed files for both {STATIC,SHARED} builds.

TODOS:

1. Add install call to current {DEBUG,RELEASE} builds. 🚧
2. Publish the {DEBUG,RELEASE} installed dirs as artifacts. 🗝️

Update documentation for the "decode" function

As #84 and #85 have recently been fixed and merged, the base64pp::decode function accepts unpadded strings, so things like JWTs can be decoded without having to re-pad the strings.

This issue is to make sure that this new behaviour is outlined in the docs. Also, make sure that the action is ran once the changes go in.

[Conan] Add CI/CD to test conan package

Make sure that the conan package added in conan/conanfile.py can be built with conan create ... and a consumer project can use the generated package.

TODOs:

1. Create a "deployment" stage with a job for deploying this conan package.
2. In the conan deploy job, call conan create ... with the correct versioning parameters.
3. In the conan deploy job, create and build a consumer "hello world" project that uses the Base64 functions from the conan dependency.

Investigate writing a Conan v2 package recipe

Here are some conference talks worth watching:

1. Conan in practice
2. What changed in Conan 2

It's also worth seeing if there are free ways to host the conan recipe for an open-source package.

[Conan] consolidate package configurations

Make sure that we have somewhat the correct data files and structure to be accepted into the conan-centre-index

TODOs:

• Add a couple of version support with the conandata.yml and the config.yml. 🔧
• Apply patches to support older compilers: the main issue is the span header. 🩹
• Write some test packages for this. Maybe support Conan v1 too? 💪

This is a reference PR by one of the active conan devs.

Add "doxygen-awesome-css" as a submodule instead of tracked files

Add the doxygen-awesome-css as a submodule instead of copied files in the repo.

TODOS 📖

• Add the doxygen-awesome-css repo as submodule.
• Modify the pipeline so the docs generation stage inits submodules before building docs.

Need to add lcov and gcov to containers

These tools are necessary for #55

Don't build docs by default

Make sure the pipeline isn't configuring the docs generation for every build. This could save time as doxygen wouldn't be found and the doxygen targets wouldn't be generated.

TODOs 👍

• Make the BUILD_DOCS option default to False in the main CMakeLists.txt

Upgrade clang-format and clang-tidy in Dockerfile

Mistakenly, the clang tools added to the docker image was version 6.0, and they do not contain the --dry-run options needed for outputting linting errors in the pipeline.

Add support for unpadded base64 strings

Add support for optionally padded Base64 strings, one example mentioned in #84 .

TODOS 👍

1. Add logic to automatically pad strings if there are missing = characters at the end of decoding inputs.
2. Modify tests to include cases where unpadded strings occurs.

Investigate using pipx instead of choco on Windows

Investigate if there are time gains when switching to pipx when installing conan for Windows.

Motivation 🔢

Currently, the windows build pipeline takes way too long, most of that is basically just choco installing things.

Add licience please

Hello! Your lib looks solid. But I'm not sure if i can use it. It looks pretty match that you would like to share it, but still...

So could you specify any known licience on this, please?

Set up stage dependency in the pipelines

By default, all Azure Pipelines stages depend on its predecessor stage. This prevents certain stages from running in parallel, i.e. docs and tests.

TODOs 👁️

• Figure which stages can be running in parallel.
• Apply the dependsOn field to all the stages.

Disable test building by default

In order to improve the build times in the pipeline, removing some of the need for conan installation could reduce the Windows build times by minutes.

Proposal 📜

• Add the BUILD_TESTS option to CMake
• Guard the calls to find_package(GTest ...) with checks on BUILD_TESTS
• Only declare/enable tests if BUILD_TESTS is defined.

Add a "fail-fast" stage with shell linting, clang-format, clang-tidy.

TODOS:

• Add shellcheck, clang-format & clang-tidy to the docker image. 🐧
• Add pipeline shell linting with shellcheck. #️⃣
• Add pipeline clang-format job. 👔
• #42

Remove Doxygen tar from the container after build!

The file doxygen-1.9.6.src.tar.gz is present in ~ after building. This saves about 5Mb.

Simply use a rm command after the doxy build in the Dockerfile 🗑️

Add tests to cover untested lines

As we included coverage with #55, we can see that there are a couple of lines that are not covered in the unit tests.

Some of these lines may not ever be reachable because of the logic, but this one certainly can be by adding either a + or a / in the tests.

Add build generated files to .gitignore

Plenty of things are not being ignored by git 📑

1. conan directory
2. compile_commands.json
3. CMakeUserPresets.json

There may be more, but these are the ones I can spot right now.

Add more round-trip tests

Make sure we have more round-trip unit tests.

📑 A round trip test is defined here as input -> encoded input -> decode base64 -> check decoded with input

Here are some possible tests, make sure to remove the padding off some of them:

YW55IGNhcm5hbCBwbGVhcw==
bGVnYWwgcGFzcw==
dGVzdCBzdHJpbmc=
bGVnYWwgcHJvdmlkZXI=
ZW5vdWdoIHRoZSBzYW1lIG9mIHRoZSBwbGFjZQ==
YW5vdGhlciB0aGUgc3RyYWlnaHQ=
d2FzIG1lIGFkZHJlc3MgcHJvdmlkZXI=
YWJvdXQgdGhlIG1hc3RlciBvZiB0aGUgZGFtYWdl
ZW50aXJlIHRoYXQgYnJvdWdodCBvZiB0aGUgbW9uZXk=
bGVnYWwgc2VjdXJpdHk=
YmFzaWMgZ29vZCBvZiB0aGUgcGFkIHN0cmluZw==
ZGVsZXRlIHN0cmluZyBvZiB0aGUgc3RyYWlnaHQ=
YnJvdWdodCBvZiB0aGUgcGFkIGZvbGRlciBvZiB0aGUgZGFtYWdl
aW50ZXJmYWNlIHN0cmluZw==
Y29uc29sZS1tZS1jb21wYW55
aW5mb3JtYXRpb24tbWVkaWE=
c3RhdHVzLXNlY3VyZQ==
Y3JlYXRlLWNvbXBhbnktc3RyaW5n
b3JkZXItbGVhZGVy
Y2F0YWxvZy1wcm9maWxl
dGVzdC1jb25zdWx0aW5n
YnJvdWdodC1sZWFkZXI=
YXNzaWduLW1lY2hhbmlzbQ==
bGVnYWwtY29udGFpbmVy
ZW1haWwtY29udGFpbmVy
aW5zdGFuY2UtY29udGFpbmVy
dGVzdC1jb21wYW55LWFuZC1wcm9maWxl
YmFzZTY0LWJhc2U=
cGFzc3dvcmQ=
Zm9vYmFy
Y29vbC1iYXNl
YmFzZTY0LXNlY3VyZQ==
aW50ZXJ2YWw=
dGhlLW1hc3Rlci1vZi10aGUtZGFtYWdl
c2FtZS1wbGFjZS1vZi10aGUtZGFtYWdl
aGFzaC1zb21ldGhpbmc=

Sort out the API input parameters signatures

The signatures of the API feels kind of wrong. For instance, we have base64pp::encode(std::span<std::uint8_t const> const input), even if there's nothing wrong copying the span object as it's just a fat pointer.

Same thing goes to decode and the string_view param.

Add Release Builds To Pipeline

Build in Release for both Ubuntu and Windows.

TODOS 👍

• Add release build for Ubuntu using the unix-rel preset.
• Add release build for Windows using the vs2022-rel preset.