chronoxor / cpplogging Goto Github PK

Ultra fast and low latency C++ Logging library with flexible configuration and high throughput in multithreaded environment

License: MIT License

CMake 2.23% C++ 95.00% C 0.78% Shell 0.03% Batchfile 0.03% Python 1.94%

logging-library logging low-latency performance speed

cpplogging's Introduction

CppLogging

C++ Logging Library provides functionality to log different events with a high throughput in multithreaded environment into different sinks (console, files, rolling files, syslog, etc.). Logging configuration is very flexible and gives functionality to build flexible logger hierarchy with combination of logging processors (sync, async), filters, layouts (binary, hash, text) and appenders.

CppLogging API reference

Features
Requirements
How to build?
Logging examples
Logging benchmarks
Tools

Features

Cross platform (Linux, MacOS, Windows)
Optimized for performance
Binary, hash & text layouts
Synchronous logging
Asynchronous logging
Flexible configuration and logger processing hierarchy
Appenders collection (null, memory, console, file, rolling file, ostream, syslog)
Logging levels (debug, info, warning, error, fatal)
Logging filters (by level, by logger name, by message pattern)
Format logging records using {fmt} library
Hash layout using 32-bit FNV-1a string hashing
Log files rolling policies (time-based, size-bases)
Log files Zip archivation

Requirements

Linux
MacOS
Windows
cmake
gcc
git
gil
python3

Optional:

How to build?

Linux: install required packages

sudo apt-get install -y binutils-dev uuid-dev

Install gil (git links) tool

pip3 install gil

Setup repository

git clone https://github.com/chronoxor/CppLogging.git
cd CppLogging
gil update

Linux

cd build
./unix.sh

MacOS

cd build
./unix.sh

Windows (Cygwin)

cd build
unix.bat

Windows (MSYS2)

cd build
unix.bat

Windows (MinGW)

cd build
mingw.bat

Windows (Visual Studio)

cd build
vs.bat

Logging examples

Example 1: Default logger

This is the simple example of using default logger. Just link it with CppLogging library and you'll get default logger functionality with text layout and console appender:

#include "logging/logger.h"

int main(int argc, char** argv)
{
    // Create default logger
    CppLogging::Logger logger;

    // Log some messages with different level
    logger.Debug("Debug message");
    logger.Info("Info message");
    logger.Warn("Warning message");
    logger.Error("Error message");
    logger.Fatal("Fatal message");

    return 0;
}

Example will create the following log in console:

Example 2: Format with logger

CppLogging library provides powerful logging format API based on the {fmt} library:

#include "logging/logger.h"

class Date
{
public:
    Date(int year, int month, int day) : _year(year), _month(month), _day(day) {}

    friend std::ostream& operator<<(std::ostream& os, const Date& date)
    { return os << date._year << '-' << date._month << '-' << date._day; }

    friend CppLogging::Record& operator<<(CppLogging::Record& record, const Date& date)
    { return record.StoreCustomFormat("{}-{}-{}", date._year, date._month, date._day); }

private:
    int _year, _month, _day;
};

class DateTime
{
public:
    DateTime(Date date, int hours, int minutes, int seconds) : _date(date), _hours(hours), _minutes(minutes), _seconds(seconds) {}

    friend std::ostream& operator<<(std::ostream& os, const DateTime& datetime)
    { return os << datetime._date << " " << datetime._hours << ':' << datetime._minutes << ':' << datetime._seconds; }

    friend CppLogging::Record& operator<<(CppLogging::Record& record, const DateTime& datetime)
    {
        const size_t begin = record.StoreListBegin();
        record.StoreList(datetime._date);
        record.StoreList(' ');
        record.StoreList(datetime._hours);
        record.StoreList(':');
        record.StoreList(datetime._minutes);
        record.StoreList(':');
        record.StoreList(datetime._seconds);
        return record.StoreListEnd(begin);
    }

private:
    Date _date;
    int _hours, _minutes, _seconds;
};

int main(int argc, char** argv)
{
    // Create default logger
    CppLogging::Logger logger;

    // Log some messages with format
    logger.Info("argc: {}, argv: {}", argc, (void*)argv);
    logger.Info("no arguments");
    logger.Info("{0}, {1}, {2}", -1, 0, 1);
    logger.Info("{0}, {1}, {2}", 'a', 'b', 'c');
    logger.Info("{}, {}, {}", 'a', 'b', 'c');
    logger.Info("{2}, {1}, {0}", 'a', 'b', 'c');
    logger.Info("{0}{1}{0}", "abra", "cad");
    logger.Info("{:<30}", "left aligned");
    logger.Info("{:>30}", "right aligned");
    logger.Info("{:^30}", "centered");
    logger.Info("{:*^30}", "centered");
    logger.Info("{:+f}; {:+f}", 3.14, -3.14);
    logger.Info("{: f}; {: f}", 3.14, -3.14);
    logger.Info("{:-f}; {:-f}", 3.14, -3.14);
    logger.Info("int: {0:d};  hex: {0:x};  oct: {0:o}; bin: {0:b}", 42);
    logger.Info("int: {0:d};  hex: {0:#x};  oct: {0:#o};  bin: {0:#b}", 42);
    logger.Info("The date is {}", Date(2012, 12, 9));
    logger.Info("The datetime is {}", DateTime(Date(2012, 12, 9), 13, 15, 57));
    logger.Info("Elapsed time: {s:.2f} seconds", "s"_a = 1.23);
    logger.Info("The answer is {}"_format(42));

    return 0;
}

Example will create the following log:

2019-03-27T12:04:19.881Z [0x0000FFB8] INFO   - argc: 1, argv: 0x1ff83563210
2019-03-27T12:04:19.882Z [0x0000FFB8] INFO   - no arguments
2019-03-27T12:04:19.883Z [0x0000FFB8] INFO   - -1, 0, 1
2019-03-27T12:04:19.884Z [0x0000FFB8] INFO   - a, b, c
2019-03-27T12:04:19.884Z [0x0000FFB8] INFO   - a, b, c
2019-03-27T12:04:19.884Z [0x0000FFB8] INFO   - c, b, a
2019-03-27T12:04:19.884Z [0x0000FFB8] INFO   - abracadabra
2019-03-27T12:04:19.885Z [0x0000FFB8] INFO   - left aligned
2019-03-27T12:04:19.885Z [0x0000FFB8] INFO   -                  right aligned
2019-03-27T12:04:19.885Z [0x0000FFB8] INFO   -            centered
2019-03-27T12:04:19.886Z [0x0000FFB8] INFO   - ***********centered***********
2019-03-27T12:04:19.886Z [0x0000FFB8] INFO   - +3.140000; -3.140000
2019-03-27T12:04:19.887Z [0x0000FFB8] INFO   -  3.140000; -3.140000
2019-03-27T12:04:19.887Z [0x0000FFB8] INFO   - 3.140000; -3.140000
2019-03-27T12:04:19.887Z [0x0000FFB8] INFO   - int: 42;  hex: 2a;  oct: 52; bin: 101010
2019-03-27T12:04:19.888Z [0x0000FFB8] INFO   - int: 42;  hex: 0x2a;  oct: 052;  bin: 0b101010
2019-03-27T12:04:19.888Z [0x0000FFB8] INFO   - The date is 2012-12-9
2019-03-27T12:04:19.888Z [0x0000FFB8] INFO   - The datetime is 2012-12-9 13:15:57
2019-03-27T12:04:19.889Z [0x0000FFB8] INFO   - Elapsed time: 1.23 seconds
2019-03-27T12:04:19.889Z [0x0000FFB8] INFO   - The answer is 42

Example 3: Configure custom logger with text layout and console appender

This example shows how to configure a custom logger with a given name to perform logging with a text layout and console appender sink:

#include "logging/config.h"
#include "logging/logger.h"

void ConfigureLogger()
{
    // Create default logging sink processor with a text layout
    auto sink = std::make_shared<CppLogging::Processor>(std::make_shared<CppLogging::TextLayout>());
    // Add console appender
    sink->appenders().push_back(std::make_shared<CppLogging::ConsoleAppender>());

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    // Create example logger
    CppLogging::Logger logger("example");

    // Log some messages with different level
    logger.Debug("Debug message");
    logger.Info("Info message");
    logger.Warn("Warning message");
    logger.Error("Error message");
    logger.Fatal("Fatal message");

    return 0;
}

Example 4: Configure custom logger with text layout and syslog appender

Syslog appender is available only in Unix platforms and does nothing in Windows!

This example shows how to configure a custom logger with a given name to perform logging with a text layout and syslog appender sink:

#include "logging/config.h"
#include "logging/logger.h"

void ConfigureLogger()
{
    // Create default logging sink processor with a text layout
    auto sink = std::make_shared<CppLogging::Processor>(std::make_shared<CppLogging::TextLayout>());
    // Add syslog appender
    sink->appenders().push_back(std::make_shared<CppLogging::SyslogAppender>());

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    // Create example logger
    CppLogging::Logger logger("example");

    // Log some messages with different level
    logger.Debug("Debug message");
    logger.Info("Info message");
    logger.Warn("Warning message");
    logger.Error("Error message");
    logger.Fatal("Fatal message");

    return 0;
}

Example 5: Configure custom logger with binary layout and file appender

This example shows how to configure a custom logger with a given name to perform logging with a binary layout and file appender sink:

#include "logging/config.h"
#include "logging/logger.h"

void ConfigureLogger()
{
    // Create default logging sink processor with a binary layout
    auto sink = std::make_shared<CppLogging::Processor>(std::make_shared<CppLogging::BinaryLayout>());
    // Add file appender
    sink->appenders().push_back(std::make_shared<CppLogging::FileAppender>("file.bin.log"));

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    // Create example logger
    CppLogging::Logger logger("example");

    // Log some messages with different level
    logger.Debug("Debug message {}", 1);
    logger.Info("Info message {}", 2);
    logger.Warn("Warning message {}", 3);
    logger.Error("Error message {}", 4);
    logger.Fatal("Fatal message {}", 5);

    return 0;
}

Example 6: Configure custom logger with hash layout and file appender

This example shows how to configure a custom logger with a given name to perform logging with a hash layout and file appender sink:

#include "logging/config.h"
#include "logging/logger.h"

void ConfigureLogger()
{
    // Create default logging sink processor with a hash layout
    auto sink = std::make_shared<CppLogging::Processor>(std::make_shared<CppLogging::HashLayout>());
    // Add file appender
    sink->appenders().push_back(std::make_shared<CppLogging::FileAppender>("file.hash.log"));

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);

    // Startup the logging infrastructure
    CppLogging::Config::Startup();
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    // Create example logger
    CppLogging::Logger logger("example");

    // Log some messages with different level
    logger.Debug("Debug message {}", 1);
    logger.Info("Info message {}", 2);
    logger.Warn("Warning message {}", 3);
    logger.Error("Error message {}", 4);
    logger.Fatal("Fatal message {}", 5);

    return 0;
}

Example 7: Configure logger with custom text layout pattern

Text layout message is flexible to customize with layout pattern. Text layout pattern is a string with a special placeholders provided inside curly brackets ("{}").

Supported placeholders:

{UtcDateTime} / {LocalDateTime} - converted to the UTC/local date & time (e.g. "1997-07-16T19:20:30.123Z" / "1997-07-16T19:20:30.123+01:00")
{UtcDate} / {LocalDate} - converted to the UTC/local date (e.g. "1997-07-16")
{Time} / {LocalTime} - converted to the UTC/local time (e.g. "19:20:30.123Z" / "19:20:30.123+01:00")
{UtcYear} / {LocalYear} - converted to the UTC/local four-digits year (e.g. "1997")
{UtcMonth} / {LocalMonth} - converted to the UTC/local two-digits month (e.g. "07")
{UtcDay} / {LocalDay} - converted to the UTC/local two-digits day (e.g. "16")
{UtcHour} / {LocalHour} - converted to the UTC/local two-digits hour (e.g. "19")
{UtcMinute} / {LocalMinute} - converted to the UTC/local two-digits minute (e.g. "20")
{UtcSecond} / {LocalSecond} - converted to the UTC/local two-digits second (e.g. "30")
{UtcTimezone} / {LocalTimezone} - converted to the UTC/local timezone suffix (e.g. "Z" / "+01:00")
{Millisecond} - converted to the three-digits millisecond (e.g. "123")
{Microsecond} - converted to the three-digits microsecond (e.g. "123")
{Nanosecond} - converted to the three-digits nanosecond (e.g. "789")
{Thread} - converted to the thread Id (e.g. "0x0028F3D8")
{Level} - converted to the logging level
{Logger} - converted to the logger name
{Message} - converted to the log message
{EndLine} - converted to the end line suffix (e.g. Unix "\n" or Windows "\r\n")

#include "logging/config.h"
#include "logging/logger.h"

void ConfigureLogger()
{
    // Create a custom text layout pattern
    const std::string pattern = "{UtcYear}-{UtcMonth}-{UtcDay}T{UtcHour}:{UtcMinute}:{UtcSecond}.{Millisecond}{UtcTimezone} - {Microsecond}.{Nanosecond} - [{Thread}] - {Level} - {Logger} - {Message} - {EndLine}";

    // Create default logging sink processor with a text layout
    auto sink = std::make_shared<CppLogging::Processor>(std::make_shared<CppLogging::TextLayout>(pattern));
    // Add console appender
    sink->appenders().push_back(std::make_shared<CppLogging::ConsoleAppender>());

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    // Create example logger
    CppLogging::Logger logger("example");

    // Log some messages with different level
    logger.Debug("Debug message");
    logger.Info("Info message");
    logger.Warn("Warning message");
    logger.Error("Error message");
    logger.Fatal("Fatal message");

    return 0;
}

Example 8: Configure rolling file appender with time-based policy

Time-based rolling policy will create a new logging file to write into using a special pattern (contains date & time placeholders).

Time-based policy composes logging filename from the given pattern using the following placeholders:

{UtcDateTime} / {LocalDateTime} - converted to the UTC/local date & time (e.g. "1997-07-16T192030Z" / "1997-07-16T192030+0100")
{UtcDate} / {LocalDate} - converted to the UTC/local date (e.g. "1997-07-16")
{Time} / {LocalTime} - converted to the UTC/local time (e.g. "192030Z" / "192030+0100")
{UtcYear} / {LocalYear} - converted to the UTC/local four-digits year (e.g. "1997")
{UtcMonth} / {LocalMonth} - converted to the UTC/local two-digits month (e.g. "07")
{UtcDay} / {LocalDay} - converted to the UTC/local two-digits day (e.g. "16")
{UtcHour} / {LocalHour} - converted to the UTC/local two-digits hour (e.g. "19")
{UtcMinute} / {LocalMinute} - converted to the UTC/local two-digits minute (e.g. "20")
{UtcSecond} / {LocalSecond} - converted to the UTC/local two-digits second (e.g. "30")
{UtcTimezone} / {LocalTimezone} - converted to the UTC/local timezone suffix (e.g. "Z" / "+0100")

#include "logging/config.h"
#include "logging/logger.h"

void ConfigureLogger()
{
    // Create default logging sink processor with a text layout
    auto sink = std::make_shared<CppLogging::Processor>(std::make_shared<CppLogging::TextLayout>());
    // Add rolling file appender which rolls each second and create log file with a pattern "{UtcDateTime}.log"
    sink->appenders().push_back(std::make_shared<CppLogging::RollingFileAppender>(".", CppLogging::TimeRollingPolicy::SECOND, "{UtcDateTime}.log", true));

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    // Create example logger
    CppLogging::Logger logger("example");

    // Log some messages with different level
    logger.Debug("Debug message");
    logger.Info("Info message");
    logger.Warn("Warning message");
    logger.Error("Error message");
    logger.Fatal("Fatal message");

    return 0;
}

Example 9: Configure rolling file appender with size-based policy

Size-based rolling policy will create a new logging file to write when the current file size exceeded size limit. Logging backups are indexed and its count could be limited as well.

Size-based policy for 5 backups works in a following way:

example.log   -> example.1.log
example.1.log -> example.2.log
example.2.log -> example.3.log
example.3.log -> example.4.log
example.4.log -> example.5.log
example.5.log -> remove

#include "logging/config.h"
#include "logging/logger.h"

void ConfigureLogger()
{
    // Create default logging sink processor with a binary layout
    auto sink = std::make_shared<CppLogging::Processor>(std::make_shared<CppLogging::BinaryLayout>());
    // Add rolling file appender which rolls after append 4kb of logs and will keep only 5 recent archives
    sink->appenders().push_back(std::make_shared<CppLogging::RollingFileAppender>(".", "file", "bin.log", 4096, 5, true));

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    // Create example logger
    CppLogging::Logger logger("example");

    // Log some messages with different level
    logger.Debug("Debug message");
    logger.Info("Info message");
    logger.Warn("Warning message");
    logger.Error("Error message");
    logger.Fatal("Fatal message");

    return 0;
}

Example 10: Multithreaded logging with synchronous processor

Synchronous processor uses critical-section lock to avoid multiple threads from logging at the same time (logging threads are waiting until critical-section is released).

This example shows how to configure a custom logger with a given name to use synchronous processor in multithreaded environment:

#include "logging/config.h"
#include "logging/logger.h"

#include <atomic>
#include <iostream>
#include <thread>
#include <vector>

void ConfigureLogger()
{
    // Create default logging sink processor with a binary layout
    auto sink = std::make_shared<CppLogging::SyncProcessor>(std::make_shared<CppLogging::BinaryLayout>());
    // Add file appender with size-based rolling policy and archivation
    sink->appenders().push_back(std::make_shared<CppLogging::RollingFileAppender>(".", "rolling", "bin.log", 4096, 9, true));

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    std::cout << "Press Enter to stop..." << std::endl;

    int concurrency = 4;

    // Start some threads
    std::atomic<bool> stop(false);
    std::vector<std::thread> threads;
    for (int thread = 0; thread < concurrency; ++thread)
    {
        threads.push_back(std::thread([&stop]()
        {
            // Create example logger
            CppLogging::Logger logger("example");

            int index = 0;

            while (!stop)
            {
                ++index;

                // Log some messages with different level
                logger.Debug("Debug message {}", index);
                logger.Info("Info message {}", index);
                logger.Warn("Warning message {}", index);
                logger.Error("Error message {}", index);
                logger.Fatal("Fatal message {}", index);

                // Yield for a while...
                CppCommon::Thread::Yield();
            }
        }));
    }

    // Wait for input
    std::cin.get();

    // Stop threads
    stop = true;

    // Wait for all threads
    for (auto& thread : threads)
        thread.join();

    return 0;
}

Example 11: Multithreaded logging with asynchronous processor

Asynchronous processor uses lock-free queue to collect logging records from multiple threads at the same time.

This example shows much better performance with less threads contentions in comparison with the previous one for lots of threads:

#include "logging/config.h"
#include "logging/logger.h"

#include <atomic>
#include <iostream>
#include <thread>
#include <vector>

void ConfigureLogger()
{
    // Create default logging sink processor with a text layout
    auto sink = std::make_shared<CppLogging::AsyncWaitProcessor>(std::make_shared<CppLogging::TextLayout>());
    // Add file appender with time-based rolling policy and archivation
    sink->appenders().push_back(std::make_shared<CppLogging::RollingFileAppender>(".", CppLogging::TimeRollingPolicy::SECOND, "{UtcDateTime}.log", true));

    // Configure example logger
    CppLogging::Config::ConfigLogger("example", sink);
}

int main(int argc, char** argv)
{
    // Configure logger
    ConfigureLogger();

    std::cout << "Press Enter to stop..." << std::endl;

    int concurrency = 4;

    // Start some threads
    std::atomic<bool> stop(false);
    std::vector<std::thread> threads;
    for (int thread = 0; thread < concurrency; ++thread)
    {
        threads.push_back(std::thread([&stop]()
        {
            // Create example logger
            CppLogging::Logger logger("example");

            while (!stop)
            {
                // Log some messages with different level
                logger.Debug("Debug message");
                logger.Info("Info message");
                logger.Warn("Warning message");
                logger.Error("Error message");
                logger.Fatal("Fatal message");

                // Yield for a while...
                CppCommon::Thread::Yield();
            }
        }));
    }

    // Wait for input
    std::cin.get();

    // Stop threads
    stop = true;

    // Wait for all threads
    for (auto& thread : threads)
        thread.join();

    return 0;
}

Logging benchmarks