Frame access and manipulation about audio

[udoprog]

I'm working on getting everything into shape for the stable release of GATs in about 6 weeks in #4.

I'll be opening an issue to gather feedback about what kind of APIs people want to see added for an initial 0.2 release of these audio abstractions. The first on those will be frame iteration API as was initially proposed in this issue.

Thank you!

[udoprog]

We have added some preliminary support for frame-oriented iteration in git now. It would be great if you could take a look and see if it corresponds to what you're looking for. No mutable iteration (yet) but that should be simple enough to add.

Even better if you could point me to a project that we can try and port ;)

[Be-ing]

Welp, guess it's going to be a while longer before GATs are stabilized :/ rust-lang/rust#96709

[udoprog]

What's been added so far are the following:

Implementations for:

• audio::buf::Interleaved
• audio::buf::Sequential
• audio::wrap::Interleaved
• audio::wrap::Sequential

/// A buffer which has a unifom channel size.
pub trait UniformBuf: Buf {
    /// The type the channel assumes when coerced into a reference.
    type Frame<'this>: Frame<Sample = Self::Sample>
    where
        Self: 'this;

    /// A borrowing iterator over the channel.
    type FramesIter<'this>: Iterator<Item = Self::Frame<'this, Sample = Self::Sample>>
    where
        Self: 'this;

    /// Get a single frame at the given offset.
    ///
    /// # Examples
    ///
    /// ```
    /// use audio::{Frame, UniformBuf};
    ///
    /// fn test<B>(buf: B)
    /// where
    ///     B: UniformBuf<Sample = u32>,
    /// {
    ///     let frame = buf.get_frame(0).unwrap();
    ///     assert_eq!(frame.get(1), Some(5));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [1, 5]);
    ///
    ///     let frame = buf.get_frame(2).unwrap();
    ///     assert_eq!(frame.get(1), Some(7));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [3, 7]);
    ///
    ///     assert!(buf.get_frame(4).is_none());
    /// }
    ///
    /// test(audio::sequential![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::sequential([1, 2, 3, 4, 5, 6, 7, 8], 2));
    ///
    /// test(audio::interleaved![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::interleaved([1, 5, 2, 6, 3, 7, 4, 8], 2));
    /// ```
    fn get_frame(&self, frame: usize) -> Option<Self::Frame<'_>>;

    /// Construct an iterator over all the frames in the audio buffer.
    ///
    /// # Examples
    ///
    /// ```
    /// use audio::{Frame, UniformBuf};
    ///
    /// fn test<B>(buf: B)
    /// where
    ///     B: UniformBuf<Sample = u32>,
    /// {
    ///     let mut it = buf.iter_frames();
    ///
    ///     let frame = it.next().unwrap();
    ///     assert_eq!(frame.get(1), Some(5));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [1, 5]);
    ///
    ///     assert!(it.next().is_some());
    ///
    ///     let frame = it.next().unwrap();
    ///     assert_eq!(frame.get(1), Some(7));
    ///     assert_eq!(frame.iter().collect::<Vec<_>>(), [3, 7]);
    ///
    ///     assert!(it.next().is_some());
    ///     assert!(it.next().is_none());
    /// }
    ///
    /// test(audio::sequential![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::interleaved([1, 5, 2, 6, 3, 7, 4, 8], 2));
    ///
    /// test(audio::interleaved![[1, 2, 3, 4], [5, 6, 7, 8]]);
    /// test(audio::wrap::sequential([1, 2, 3, 4, 5, 6, 7, 8], 2));
    /// ```
    fn iter_frames(&self) -> Self::FramesIter<'_>;
}

/// The buffer of a single frame.
pub trait Frame {
    /// The sample of a channel.
    type Sample: Copy;

    /// The type the frame assumes when coerced into a reference.
    type Frame<'this>: Frame<Sample = Self::Sample>
    where
        Self: 'this;

    /// A borrowing iterator over the channel.
    type Iter<'this>: Iterator<Item = Self::Sample>
    where
        Self: 'this;

    /// Reborrow the current frame as a reference.
    fn as_frame(&self) -> Self::Frame<'_>;

    /// Get the length which indicates number of samples in the current frame.
    fn len(&self) -> usize;

    /// Test if the current frame is empty.
    fn is_empty(&self) -> bool {
        self.len() == 0
    }

    /// Get the sample at the given channel in the frame.
    fn get(&self, channel: usize) -> Option<Self::Sample>;

    /// Construct an iterator over the frame.
    fn iter(&self) -> Self::Iter<'_>;
}

Some of the documentation needs to be fixed up since in quite a few places I believe we refer to sample as a frame. The terminology also needs to be better documented overall (but that is on my TODO).

[udoprog]

Yeah. My goal is to author a set of traits and data structures that can be used across audio libraries in the ecosystem. I'm currently holding off on GATs landing, since that's needed to provide proper abstractions without incurring a runtime overhead. GATs been making a lot of progress, but is still probably going to take a while. I never intended to propose converging on a solution until that's in place.

I'd probably be happy to incorporate more Frame-esque APIs here, and I'd even be open to provide this crate as a namespace as long as the API is reasonable and is something that audio crates actually want to use.

I think it would be good to start by replacing this crate's Sample trait with dasp::Sample.

I'm not so sure. dasp::Sample has more things than you need in an audio abstraction and puts an overly high burden on what can and cannot be a sample. It has conversions and manipulation functions which are better suited for traits exclusive to dasp. In my view each crate provide their own Sample trait to provide the functions that they need (e.g. rubato) and I believe the audio abstraction should only require the minimum necessary it needs for it to work since that will make it easier to stabilize.

But I'm open to debate this! I never intended to do this until audio was in a somewhat complete state though.

[Be-ing]

Looks like GATs will like stabilize for Rust 1.61 or 1.62.

[udoprog]

I am definitely looking forward to it!

[Be-ing]

The GAT stabilization PR has been merged! 🚀

[Be-ing]

Rust 1.65 has been released with GATs on stable!! 🚀