michaeltyson / tpcircularbuffer

A simple, fast circular buffer implementation

A simple, fast circular buffer implementation for audio processing

A simple C implementation for a circular (ring) buffer. Thread-safe with a single producer and a single consumer, using OSAtomic.h primitives, and avoids any need for buffer wrapping logic by using a virtual memory map technique to place a virtual copy of the buffer straight after the end of the real buffer.

Usage

Initialisation and cleanup: TPCircularBufferInit and TPCircularBufferCleanup to allocate and free resources.

Producing: Use TPCircularBufferHead to get a pointer to write to the buffer, followed by TPCircularBufferProduce to submit the written data. TPCircularBufferProduceBytes is a convenience routine for writing data straight to the buffer.

Consuming: Use TPCircularBufferTail to get a pointer to the next data to read, followed by TPCircularBufferConsume to free up the space once processed.

TPCircularBuffer+AudioBufferList.(c,h) contain helper functions to queue and dequeue AudioBufferList structures. These will automatically adjust the mData fields of each buffer to point to 16-byte aligned regions within the circular buffer.

Thread safety

As long as you restrict multithreaded access to just one producer, and just one consumer, this utility should be thread safe.

Only one shared variable is used (the buffer fill count), and OSAtomic primitives are used to write to this value to ensure atomicity.

License

Copyright (C) 2012-2013 A Tasty Pixel

This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software.

Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions:

1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.

2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.

3. This notice may not be removed or altered from any source distribution.

---

Virtual memory technique originally proposed by Philip Howard, and adapted to Darwin by Kurt Revis

See more info at atastypixel.com

tpcircularbuffer's Issues

using in c++ code and 'atomic_fetch_add'

had compile error: "No matching function for call to 'atomic_fetch_add'" when using inside obj-c++ and c++ classes.

changed
typedef std::atomic_int atomicInt;
to
typedef std::atomic<uint32_t> atomicInt;

and it worked. Is this change valid?

Xcode 14.3 beta not compiling

Any ideas how to resolve this issue?

ld: file not found: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/libarclite_iphoneos.a clang: error: linker command failed with exit code 1 (use -v to see invocation)

Podfile:

source 'https://github.com/CocoaPods/Specs.git'
platform :ios, '15.0'
use_frameworks!

target '<AppName>' do
  pod 'TPCircularBuffer', '1.6.1'
end

Example to use the audio circular buffer

Hello,

Could you please tell us how to use your library for audio ring buffers? Thanks. Could you provide an example code.

Redefinition of module TPCircularBuffer

Issue:

I have a problem with a submodule using TPCIrucloarBuffer, when I try to build the project every time I got the same error: Redefinition of module TPCircularBuffer

Project structure:

• Cocoapods
• Main project
  • Sub module with TPCircularBuffer

Tooling:

• Xcode 9.4
• Cocoapods
• Swift
• MacOS

Path in the binary

Hello,

In TPCircularBuffer.c this directive

#define reportResult(result,operation) (_reportResult((result),(operation),strrchr(FILE, '/')+1,LINE))

cause the path (FILE) to be written into the binary file (lib or executable)

reportResult must be use only in DEBUG mode

#if DEBUG
reportResult(result, "Buffer allocation");
#endif

I fixed it :

TPCircular.zip

New Tag for Swift Package Manager Support

Hello, and thanks again for this code. I must have been using this for 10 years by now!

I think the repository could do with a new tag to get the SPM support working. Currently when you add 1.6.1 to a package manifest, the following warning is seen:

.package(url: "https://github.com/michaeltyson/TPCircularBuffer", from: "1.6.1") // => the package manifest at '/Package.swift' cannot be accessed (/Package.swift doesn't exist in file system)

When using branch: "master" it works. All that’s needed is a new 1.6.2 tag.

No need for the atomic operations.

Just a hint: you can get rid of fillCount and atomic operations if you keep head and tail as they are and reduce them to the buffer size only when returning pointers.

cocoapod repo has out of date podspec?

I'm running into problems loading this pod in osx:

[!] The platform of the target OneLight (OS X 10.8) is not compatible with TPCircularBuffer (1.3), which does not support osx.

The podspec in this github repo seems correct (supports osx 10.8+), the podspec on cocoapods seems to be ios only.

Podspec update for osx

hey! thanks for this great lib 👍

btw, I've found a problem with very easy solution. I tried to use your lib w/ cocoapods but w/ osx target, but it failed telling that incompatible target message. i've just updated my mac and installed cocoapods, so i guess is the latest version. it turns out that the podspec file needs to specify both platforms in this way:

s.ios.platform = :ios, "4.3"
s.osx.platform = :osx, "10.10"

instead of:

s.platform = :ios, '4.3', :osx, '10.0'

i made a simple local podspec and it works!

How to change buffer size?

Any way to change buffer size (existing buffer)

thanks

TPCircularBuffer `fillCount` Swift access

This might be a stupid question but I am currently creating an app that requires the use of TPCircularBuffer but I can't see to get access to the fillCount - could you guide me in the right direction.

Thanks in advance

Add data to the tail of the buffer.

Hello would be possible to have a function that add bytes to the tail of the buffer instead of in the head? there are some situations where I might want to add data to the tail.

Thanks

TPCircularBuffer cause noise

I am playing raw PCM data over Bluetooth, and I do a lost of tests, and finally found that when TPCircularBuffer produce bytes, the noise happen, here is the two part of my code (One get PCM data from Bluetooth device, one use Portaudio to render the PCM).

Produce bytes ->:

- (void)a2dpSink:(A2DPSink *)sink channel:(IOBluetoothL2CAPChannel *)channel rawRTPdataReceived:(NSData *)data
{
    NSData *mediaPayload = getMediaPayloadInRTP(data);
    getSBCFramesInMediaPayload(mediaPayload);
    recordCount++;

    if (recordCount == 50) {
        long index = 0;
        uint16 i;
        long size = 0;
        for (i = 0; i < [sbcFrames count]; i++) {
            size = [[sbcFrames objectAtIndex:i] length];
            memcpy((unsigned char*)buf + index, (unsigned char*)[[sbcFrames objectAtIndex:i] bytes], size);
            index += size;
        }

        [sbcFrames removeAllObjects];

        // Start decode
        int pcmBytes = 0;
        decodeSBCFramesBuffer(buf, (int)index, pcm, &pcmBytes);
        
        TPCircularBufferProduceBytes(&paData.buffer, pcm, pcmBytes);
      
        if (!played) {
            framesPerBuffer = pcmBytes / 4;
            playPortAudio();
            played = YES;
        }

        if (!Pa_IsStreamActive(stream)) {
            e = Pa_StartStream(stream);
            if( e != paNoError ) {
                NSLog(@"start stream error!");
            } else {
                NSLog(@"start stream OK!");
            }
        }
        recordCount = 0;
    }
}

Consume bytes ->

static int patestCallback( const void *inputBuffer, void *outputBuffer,
                          unsigned long framesPerBuffer,
                          const PaStreamCallbackTimeInfo* timeInfo,
                          PaStreamCallbackFlags statusFlags,
                          void *userData )
{
    paTestData *data = (paTestData*)userData;
    short *out = (short*)outputBuffer;
    unsigned long i, j;

    (void) timeInfo; /* Prevent unused variable warnings. */
    (void) statusFlags;
    (void) inputBuffer;

    uint32_t availableBytes = 0;
    unsigned char *b = TPCircularBufferTail(&data->buffer, &availableBytes);

    uint32_t frames = availableBytes / 2 / 2;
    if (b == NULL) return paContinue;
    uint32_t availableFrames = (uint32_t) MIN(frames, framesPerBuffer);
    uint32_t index = 0;
    for(i = 0; i < availableFrames; i++){
        for( j = 0; j < CHANNEL_COUNT; ++j ){
            unsigned char high = b[index];
            unsigned char low =  b[index+1];
            short s = low + (high << 8);
            *out++ = s;
            index += 2;
        }
    }
    TPCircularBufferConsume(&data->buffer, (uint32_t)index);
    return paContinue;
}

If I increase the recordCount to like 200

 if (recordCount == 50)

then the noise occur in a long period.

So I think the noise is just happen when TPCircularBufferProduceBytes(&paData.buffer, pcm, pcmBytes); is called, any idea? I am testing this issue for a few days.

how to add without error in my project?

I want to use TPCircularBuffer for play NSData PCM raw file in objective c.

my big challeng in below link:
https://stackoverflow.com/questions/48841382/how-to-play-back-audio-after-receive-from-web-socket

Android equivalent

does anybody know if there is an Android equivalent out there? this buffer implementation is just awesome, i cant find anything like that for android

thread sanitizer warnings on macOS

Hi Michael,

I'm receiving the following thread sanitizer warning (and couple more) when running my app in debug mode on macOS High Sierra and XCode 9.

WARNING: ThreadSanitizer: data race (pid=10371)
  Read of size 8 at 0x00010c0b5058 by thread T27 (mutexes: write M471887713214197208, write M133836300544511056):
  * #0 TPCircularBufferDequeueBufferListFrames <null> (XXXXXX:x86_64+0x100284355)
    #1 myRenderCallback <null> (XXXXXX:x86_64+0x10014b08b)
    #2 AUMultiChannelMixerInputElement::PullMixerInput(unsigned int, unsigned int&, AudioTimeStamp const&, unsigned int, AudioBufferList*) <null> (CoreAudio:x86_64+0x112bb0)

  Previous write of size 8 at 0x00010c0b5058 by main thread:
  * #0 TPCircularBufferPrepareEmptyAudioBufferList <null> (XXXXXX:x86_64+0x10027d92e)
    #1 TPCircularBufferCopyAudioBufferList <null> (XXXXXX:x86_64+0x10027fb84)

  Issue is caused by frames marked with "*".

I'm not sure if I'm using TPCircularBuffer correctly and it seems that the same code is not an issue on the iOS.

The producer will use TPCircularBufferCopyAudioBufferList to fill data into the buffer, the consumer (render callback) uses TPCircularBufferDequeueBufferListFrames to satisfy the render callback requirement...

Am I doing this right? Are these lockless and thread safe on the macOS?

thanks,
Martin

Possible patch

Hi there,

We're using TPCircularBuffer (older revision from February I believe ~ adc2299) in OpenEmu and have a possible patch you may be interested in: OpenEmu/OpenEmu@655ea5e

New release including the use of `stdatomic` over `OSAtomic`

Should a 1.5 tag be made including the new changes in master from b1e15ca and 7364065? 😇

I was making these changes locally on top of 1.4 and then realized you had already made them.

Buffer does not use any circular features when calling TPCircularBufferProduceBytes

Calls to TPCircularBufferProduceBytes will just return when the buffer does not have enough space left. So basically the buffer fills up and then when you call TPCircularBufferProduceBytes, the method just returns with no changes and the buffer stays exactly as it was. Offending line is: if ( space < len ) return false;

Instead it should be a circular buffer and if it is out of space, overwrite the older data.

My colleague found this issue a couple of years ago and I ran into it again when getting the code from source instead of using his fixed version (new method added). I will see if he wants to open a pull request with the changes. But basically if no space is left, call a Consume operation to free up that space, get a fresh pointer to the head, then write the data.

Issues with signedness when compiling with aggressive settings on XCode 9

Hi Micheal,

I have turned on some aggressive compiler checking on my projects, along with the new XCode 9 undefined behavior, runtime behavior, and address sanitizer and uncovered couple of compiler warnings around converting int to uint and others in TPCircularBuffer code.

I wonder whether I should spend some time trying to fix them (read: whether you are interested in pulling these into master).

Attached is an example screenshot from xcode pointing towards buffer length, head, and tail being int32_t which can be negative and (from my limited understanding of your code) they would better be unsigned somethings...

feel free to close as inappropriate or +1 if you want me to investigate this further...

thanks,
Martin

No example? No readme?

No example? No readme?

Are stdatomic changes ready? Could you make a new release?

I see a commit which replaces OSAtomic with stdatomic functions. But it was done after latest release v1.4 so I couldn't get the changes via CocoaPods.

There's no hurry. Thanks for your great work.

Compiler Error: TPCircularBuffer.h:159:9: No matching function for call to 'atomic_fetch_add'

I just integrated TPCircularBuffer into my Swift project via pod 'TPCircularBuffer', '~> 1.5', and I'm getting the following compiler error at line 159 of TPCircularBuffer.h.

I see the following StackOverflow answer, which is helpful, but I'm wondering if support for Swift projects could be implemented as part of this library.

Thanks!