flucoma / flucoma-sc Goto Github PK

Situation: Running example/dataset/learning/12 with its 1400 slices crashes the server sometimes.

How to reproduce: running the example after the server has been running for a certain time crashes it. Sometimes, the first time round. It is hard to catch but a Mac user had similar issues I never had. Linux might be a better place to catch it.

Situation: In both Mac and Win, it works as expected, but on Linux, its use does not free the synth that is waiting to free itself in KR mode. It is true for all instances.

How to reproduce: trying the FluidBufThreadDemo KR example in the helpfile shows right away the avalanche of errors we get in that context.

I'd like to write a SynthDef collecting features of an audio stream in a buffer:

SynthDef(\rtAnalysis, {|out = 0, buf = 0, featureBuf = 1|
	var sig, mfccs, isPlaying;
	
	sig = PlayBuf.ar(1, buf, doneAction: 2);
	
	mfccs = FluidMFCC.kr(
		sig, q.numFccCoefs, 
		startCoeff: 1, 
		maxNumCoeffs: q.numFccCoefs
	); 

	FluidKrToBuf.kr(mfccs, featureBuf);
	
	sig!2

}).add;

my (naive) approach was to provide a buffer number here but soon i found out that a Buffer object is required:

this implementation leads to

^^ ERROR: Message 'numFrames' not understood.
RECEIVER: an OutputProxy

I'd propose to base the iteration in FluidKrToBuf on the dimensionality of the kr signal instead:

FluidKrToBuf {
	*kr {
		arg krStream, buffer;
		^krStream.collect{
			arg sig, i;
			BufWr.kr(sig, buffer, i);
		}
	}
}

A definition for FluidBufToKr is a little more involved but I could imagine that providing (optional) numFrame and startFrame arguments would increase usability:

FluidBufToKr {
	*kr {
		arg buffer, numFrames, startFrame=0;


		if(buffer.isKindOf(Buffer), {
			numFrames = numFrames ?? {buffer.numFrames - startFrame};
		}, {
			numFrames = numFrames ? 1;
		});

		if(numFrames > 1,{
			^numFrames.collect{
				arg i;
				BufRd.kr(1,buffer,i,0,0);
			}
		},{
			^BufRd.kr(1,buffer,0,0,0);
		});
	}
}

After implementing this, I realised you already fixed this in the dev branch:

However, I could imagine that my startFrame argument might be a good addition ;)
Also, I'd recommend initialising numFrames with nil instead of -1.

Good work, anyhow!

Perhaps I'm misremembering. I thought that setting numDimesions in PCA to 0 caused it to return all the PCs? In this dummy example (extracted and simplified from other code) it's returning a DataSet with 1 dimension.

setting PCA's numDimensions to 1 also returns a DataSet of 1 dimension
setting PCA's numDimensions to 2 returns a DataSet of 2 dimensions

~ds = FluidDataSet(s);
~pca = FluidPCA(s,0);

(
~cols = 10;
~data = Dictionary();
100.do{
	arg i;
	~data["point-%".format(i)] = Array.fill(~cols,{rrand(0.0,1.0)});
};
~dict = Dictionary.newFrom([
	"cols",~cols,
	"data",~data
]);
~ds.load(~dict);
)

~pca.fitTransform(~ds,~ds);

~ds.print;

Situation: Dataset learning example 13 works as expected on Mac and Windows, but under Linux the loop doing the massive parallelization of MelBand analysis does not finish. I am not even certain it starts at all since the synth count stays low.

How to reproduce: run example 13 one item at a time. Everything is fine until the MelBands analysis batch processing. Observer the Console: it starts but never ends.

Greetings,

The interface is inconsistent between the Max and SC FluidWaveform objects. Should we resolve this by making them consistent across CCEs? The different argument names are:

(Max vs. SC)

• buffer vs. audioBuffer (audio that should be displayed as a waveform)
• image vs. rasterBuffer (for plotting a raster of spectrogram-type data, or MFCCs or Chroma)
• layer vs. featureBuffer (for plotting a squiggly line of other time-based data, like pitch)
• markers vs. indicesBuffer (for plotting vertical lines at particular indices)

Also I'm realizing that just passing in a buffer to create as a raster might cause some problems with scaling. For example, just linearly scaling STFT magnitudes to between 0 and 255 (for color lookup) doesn't make for a pretty or informative raster (see below).

So perhaps we need a way to let the user specify what type of scaling to use when creating the raster?

Trying the helpfile, one gets nothing back. I presume it is due to the new optional parameter.

also, maybe we should consider adding the same optional parameter to knearestdistances ?

FluidPlotter needs to be extended to match the functionality in Max: flucoma/flucoma-max#158

Hello flucoma people!

Really excited to try out the new beta! I've heard so many good things about it.

I just have a humble request: I maintain the Arch Linux package for flucoma-sc and just wanted to ask if it was possible for you to keep the release version naming scheme consistent (using for example https://semver.org/). This makes it easier to update and release the package on Arch (and probably other distros as well). I know the current latest release is a beta but we archers love betas and cutting edge software (and flucoma!).

Thanks and keep up the good work

This doesn't work:

(
SynthDef(\test,{
	arg passed_buffer;
	var stream = FluidBufToKr.kr(passed_buffer);
}).add;
)

But this does:

(
Routine{
	var created_buffer = Buffer.alloc(s,3);
	
	s.sync;
	
	SynthDef(\test,{
		var stream = FluidBufToKr.kr(created_buffer);
	}).add;
}.play;
)

If the buffer is intended to be passed as an argument to the SynthDef, FluidBufToKr will throw an error saying that it doesn't know how many frames the buffer is. To be more specific, the SynthDef has no idea that it is supposed to be a buffer. The error says it can't call .numFrames on an OutputProxy.

This is correct behavior because FluidBufToKr needs to know how many frames it will be outputting (as channels in the kr stream), but maybe there should be a way to specify the number of frames manually so that one can swap out different buffers on the fly.

and should be switched to "id" or identifier

I think it wouldn't be much work to make a small extension to the Buffer object to allow the Fluid objects to stash some useful info in them. I imagine it looking something like this (this is quick and dirty, the syntax and interface could be much more thoughtful).

It could allow for quick visual inspection of what is in a Buffer to help remind the user what data they have. It could also be useful for a more stylized posting of buffer contents and easier/nicer gui making.

this is tiny investigation in that direction: https://github.com/flucoma/flucoma-sc/tree/FluidBuffer

s.boot;

~buf = Buffer.read(s,FluidFilesPath("Nicol-LoopE-M.wav"));
~specbuf = FluidBuffer(s);
~statsbuf = FluidBuffer(s);

FluidBufSpectralShape.processBlocking(s,~buf,features:~specbuf,action:{"done".postln;});

~specbuf.frames; // posts: fftFrames (we can calc the # of them)
~specbuf.chans; // [ spectral centroid, spectral spread, normalized spectral skewness, normalized spectral kurtosis, spectral rolloff, spectral flatness, spectral crest ]

FluidBufStats.processBlocking(s,~specbuf,numChans:7,stats:~statsbuf,action:{"done".postln;});

~statsbuf.frames // [ mean, std dev, skewness, kurtosis, min, median, max ]
~statsbuf.chans // [ spectral centroid, spectral spread, normalized spectral skewness, normalized spectral kurtosis, spectral rolloff, spectral flatness, spectral crest ]

~statsbuf.getFrameChan(0,5); // "median spectral centroid"
~statsbuf.findLocation("median","spectral centroid"); // [0,5]

~statsbuf.postShape
/*
(but better:)
mean	std dev	skewness	kurtosis	min	median	max	
-------------------------
spectral centroid	|	*	*	*	*	*	*	*
spectral spread	|	*	*	*	*	*	*	*
normalized spectral skewness	|	*	*	*	*	*	*	*
normalized spectral kurtosis	|	*	*	*	*	*	*	*
spectral rolloff	|	*	*	*	*	*	*	*
spectral flatness	|	*	*	*	*	*	*	*
spectral crest	|	*	*	*	*	*	*	*
*/

I know we're getting to this in the coming weeks, just flagging it here for me while it's on my mind. Just want to make sure this method call doesn't slip through the cracks.

Hello FluCoMa devs,

I just wanted to share an experience building flucoma-sc on arm64 macOS 12.2 / Xcode 13 (Apple M1 hardware).

TL;DR it works!

There's one issue I encountered: flucoma-core / eigen had an issue building with Xcode 13, so I commented out one line as suggested in flucoma/flucoma-core#69 (comment)

Moreover, I had to explicitly specify the architecture:

cmake -DSC_PATH=/paht/to/supercollider -DCMAKE_OSX_ARCHITECTURES=arm64 ..

I might be wrong, but I don't think the architecture needs to be hardcoded in CMakeLists.txt, I believe it should be picked up automatically.

It seems to work well so far.

Thanks so much for a great library! I hope you find this report useful.

Hi ! Just testing out compiling and installing this from master branch GitHub source on a Pi4 running arch Linux arm and I have run into a problem. It smells like an x86 exclusive dependency ? Thanks !! Keep up the nice work !

[ 8%] Building CXX object _deps/flucoma-core-build/CMakeFiles/HISSTools_FFT.dir//hisstools-src/HISSTools_FFT/HISSTools_FFT.cpp.o In file included from /home/alarm/.cache/yay/supercollider-flucoma-git/src/supercollider-flucoma-git-r1179.5956e97/build/_deps/hisstools-src/HISSTools_FFT/HISSTools_FFT.cpp:3: /home/alarm/.cache/yay/supercollider-flucoma-git/src/supercollider-flucoma-git-r1179.5956e97/build/_deps/hisstools-src/HISSTools_FFT/HISSTools_FFT_Core.h:16:10: fatal error: emmintrin.h: No such file or directory 16 | #include <emmintrin.h> | ^~~~~~~~~~~~~ compilation terminated. make[2]: *** [_deps/flucoma-core-build/CMakeFiles/HISSTools_FFT.dir/build.make:76: _deps/flucoma-core-build/CMakeFiles/HISSTools_FFT.dir//hisstools-src/HISSTools_FFT/HISSTools_FFT.cpp.o] Error 1 make[1]: *** [CMakeFiles/Makefile2:1071: _deps/flucoma-core-build/CMakeFiles/HISSTools_FFT.dir/all] Error 2 make: *** [Makefile:136: all] Error 2 ==> ERROR: A failure occurred in build(). Aborting...

I would put them in the 'describe audio'

there is also the duplicate more verbose and move inviting 'fluid decomposition toolbox' guide still existing and probably needing to be either augmented by these objects, or else?

Hello,

My steps to compile this for my machine (ARM m1) go as follows:

1. clone this repo
2. recurse clone supercollider
3. mkdir -p build && cd build
4. cmake -DSC_PATH=../supercollider/ -DFLUID_ARCH=-mcpu=native ..
5. make install

Which produces this error.

/Users/james/dev/software/flucoma-sc/include/SCBufferAdaptor.hpp:13:10: fatal error: 'boost/align/aligned_alloc.hpp' file not found

I have boost installed on my system with brew so not too sure how to remedy this situation.

@james do I remember correctly that this was happening in Max too? Was there a decision made about how to address this? It's actually not really a problem in SC because the action function gets passed a variable that can indicate if it's a mouse up, so maybe it doesn't need to be fixed?

Greetings FluCoMa,

I just noticed that objects FluidBufSpectralShape, FluidBufPitch, FluidBufMFCC, and FluidBufMelBands all have a hopSize default of -1 (which I understand to be windowSize / 2), however FluidBufLoudness has a hopSize default of 512, which is the default windowSize of 1024 / 2, but is inconsistent with the other objects.

Thanks!

some files in the release-packaging folder are generated, some are editable. If we could keep a constent beaviour at least per folder that would make a lot more sense and make maintenance and upgrades easier to do.

Hi, this is an FYI relevant for users on Arch Linux, but this project has been packaged for the AUR and can now be installed via the Arch User Repository:
yay -S supercollider-flucoma

More info here: https://aur.archlinux.org/packages/supercollider-flucoma/

I think the current working theory is that FluidModelObject's setParams message is messing with OSC message processing order on server.

The issue is that this doesn't work:

(
b = Buffer.alloc(s,2); 
b.loadCollection([1,2]); 
a = FluidDataSet(s);
c = FluidDataSet(s); 
a.setPoint(\blah,b);
a.cols(); 
d = FluidNormalize(s); 
d.fitTransform(a,c); 
)

but this does:

(
fork{
	b = Buffer.alloc(s,2); 
	b.loadCollection([1,2]); 
	a = FluidDataSet(s);
	c = FluidDataSet(s); 
	a.setPoint(\blah,b);
	a.cols(); 
	d = FluidNormalize(s); 
	s.sync;
	d.fitTransform(a,c); 
}
)

Having the first one working is the desired functionality.

Short:
When running FluidDataSet(s); the server disconnects with this error

-> FluidDataSet(0)
Server 'localhost' exited with exit code 0.
server 'localhost' disconnected shared memory interface

However running the examples in the documentation for FluidAudioTransport everything works fine.

OS: linux manjaro - ryzen 5600X
supercollider version: scsynth -v 3.12.1 (Built from branch 'not_a_git_checkout' [na])
present when installing through the following methods: download and drop in ~/.local/share/Supercollider/Exentsion, building from source, and installing for the AUR.
output of FluidDataSet.version: Fluid Corpus Manipulation Toolkit: version 1.0.0-TB2.beta4
I've also recompiled the class library.

Hi,
I have an issue when using certain crucial classes, I was going to ask over here but the account needs manually approving before I can post, but not here! Basically I can browse the documentation and use the Transform Audio UGens, but nothing else without the server disconnecting requiring me to kill it before I can reboot it. I was trying to get more information about the crash but I can't find any, so if you have any suggestion let me know!

Best,
Jordan

When I fit the FluidKDTree, my FluidDataSet identifiers were all strings, so I expected the FluidKDTree to also return strings when I call kNearest. However it returns a Symbol. I want to take the nearest it returns and compare it to my set of identifiers, but == won't work because they're different classes.

Can we make FluidKDTree return the same class that was used to set the points in the FluidDataSet that it was trained on?

Hi

This is to keep track of a bug discovered today during workshop. The buffer based UGens no longer work on Arch Linux while the other ones seem to work fine.

A theory that was developed was that libppthread was to blame as it is a run time dependency that is mentioned in the core dump. Arch linux' libpthread is currently at 2.33 and GLIBC is at the same version.

@weefuzzy @salkin-mada

Hi,

Is there a way (hopefully without having to modify the core lib) to get the FluidMLPClassifier to return a confidence vector when doing prediction? I mean smething like scikit MLPClassifier's predict_proba. This is for doing multiclass prediction. In TensorFlow I would do this with a softmax output layer and train with categorical_crossentropy.

Thanks

When I am running a script I am receiving this traceback error:
ERROR: FluidPCA - Invalid JSON format.
I cannot understand where is the exception code that triggered and give us this exception. Could anyone help me?
Thank you in advance!

Hello

So we should ignore that Resource folder like we used to do for AudioFiles and Plugins - this is to allow us to work on a live version with the files in there but not actually being copied during install...

now it is quite potentially fraught with duplication.

proposals:

we manually put Resources/AudioFiles and Resources/Data in the .gitignore and we check that we remove them from the cmake (which at the moment copies the full content of the folder)

or

we move the CCE-unique resource in another folder at the top (sc-resources) and we git ignore the full of resources and copy sc-resource in resources at the same time we copy audiofiles and data.

I think I prefer the 2nd option.

Same as flucoma/flucoma-cli#8

-DFLUID_ARCH=-mnative is not a flag and should be -DFLUID_ARCH=-mcpu=native

This helped me compile SC for ARM :)

Ciao,

The FluidLoudness helpfile says that it returns:

A 2-channel KR signal with the [pitch, confidence] descriptors. The latency is windowSize.

which seems to be a copy paste error from FluidPitch.

Thanks!

T

Similar to:

• flucoma/flucoma-docs#62
• flucoma/flucoma-docs#59

FluidNMFFilter has a different order of arguments between the c++ and the SC class definition. (Just as a reminder that it throws ugly warnings in the post window every time this help file is opened.)

Reordering these seems a little more problematic than the others that we looked at. But may be the way to go nonetheless.

The arguments are all there though, so maybe we just let these warnings get thrown until the eventual removal of species and @weefuzzy upgrade[^tm]

Greetings,

When I...

1. use fitPredict on FluidKMeans,
2. dump the FluidLabelSet that it puts the predictions in, and then
3. look into that Dictionary that it dumps, the cluster id integers are strings, as in...

dict.at("data").at(key).class returns String

This was unexpected to me as they are integers.

Thanks!

FluidNormalizeQuery posts when .kr is initialized.
It's actually triggered by the user when FluidNormalize's .kr method is called.

FluidStats has a .postln in its .init method.

I can comment these out no problem. @weefuzzy, would you like me to take care of it? Branch and PR? Commit straight to main? 😳 I don't know the protocol yet.

This is the case even if the user uses strings to add points to the FluidDataSet that it is trained on. This is for important UDP communication and efficiency reasons, so it will not change.

Let's just be sure to make it very clear in the documentation that this is what the user should expect.

But the docs seem to think that it should, so I'm guessing that the C++ would like to be able to receive this kind of message, but there's currently no way to send it from sclang.

Platform: Arch 5.16.14-arch1-1 #1 SMP PREEMPT Linux x86_64
SC ver: 3.12.2
Flucoma-sc ver: 1.0.0-TB2.beta5 I have also tested with building from source 10 mins ago, same crash

The following happens when evaluating FluidDataSet(s)
Using: sudo gdb scsynth -p $(ps -C scsynth -o pid h)

Reading symbols from scsynth...
(No debugging symbols found in scsynth)
Attaching to program: /usr/bin/scsynth, process 67445
[New LWP 67448]
[New LWP 67451]
[New LWP 67452]
[New LWP 67453]
[New LWP 67454]
[New LWP 67455]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007effa4b4715a in __futex_abstimed_wait_common () from /usr/lib/libc.so.6
(gdb) n
Single stepping until exit from function __futex_abstimed_wait_common,
which has no line number information.

Thread 4 "scsynth" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7eff709fe640 (LWP 67452)]
0x000055e96cbb8d08 in ?? ()
(gdb) n
Cannot find bounds of current function
(gdb) bt
#0  0x000055e96cbb8d08 in ?? ()
#1  0x00007eff8d2e19b6 in ?? () from /usr/share/SuperCollider/Extensions/FluidCorpusManipulation/plugins/FluidManipulation.so
#2  0x00007eff8d423eac in ?? () from /usr/share/SuperCollider/Extensions/FluidCorpusManipulation/plugins/FluidManipulation.so
#3  0x00007effa4f56e3f in ?? () from /usr/lib/libscsynth.so.1
#4  0x00007effa4f3e1eb in ?? () from /usr/lib/libscsynth.so.1
#5  0x00007effa4db74d4 in std::execute_native_thread_routine (__p=0x55e96bc17b10) at /usr/src/debug/gcc/libstdc++-v3/src/c++11/thread.cc:82
#6  0x00007effa4b4a5c2 in start_thread () from /usr/lib/libc.so.6
#7  0x00007effa4bcf584 in clone () from /usr/lib/libc.so.6

Similarly this happens with several classes, ex. FluidBufOnsetSlice.process(s, ...)

Since flucoma/flucoma-core#87 this test's expected results will no longer hold, and require @tremblap's special ears to devise some new settings

These are

NMFFilterClient	maxComponents
NMFMatchClient	maxComponents
HPSSClient	maxHarmFilterSize
NoveltyFeatureClient	maxKernelSize
NoveltySliceClient	maxKernelSize
MelBandsClient	maxNumBands
ChromaClient	maxNumChroma
MFCCClient	maxNumCoeffs
HPSSClient	maxPercFilterSize
AmpGateClient	maxSize

Affects

• FluidAudioTransport
• FluidBufAudioTransport
• FluidBufSTFT
• FluidChroma
• FluidBufChroma
• FluidHPSS
• FluidBufHPSS
• FluidMFCC
• FluidBufMFCC
• FluidMelBands
• FluidBufMelBands
• FluidNMFFilter
• FluidNMFMatch
• FluidNMFMorph
• FluidNoveltyFeature
• FluidBufNoveltyFeature
• FluidNoveltySlice
• FluidBufNoveltySlice
• FluidOnsetFeature
• FluidBufOnsetFeature
• FluidOnsetSlice
• FluidBufOnsetSlice
• FluidPitch
• FluidBufPitch
• FluidSines
• FluidBufSines
• FluidSpectralShape
• FluidBufSpectralShape
• FluidNoveltyFeature
• FluidBufNoveltyFeature
• FluidNoveltySlice
• FluidBufNoveltySlice

Hi all

Thanks for your amazing work!

I noticed a problem lately that I think should be addressed somehow.
When instantiating Flucoma's UGens in SuperCollider, xruns are caused on Linux (and perhaps other things happen on other systems on instantiation, I dunno). Even with low cpu usage and basically nothing else going on. This is seen quite clearly when using Flucoma-UGens in a live coding context in NodeProxy land. As an example, an Ndef that has fadeTime at 1. Every time the Ndef is reevaluated, it will create a copy of itself and slowly fade from the past to the future version of the synth. When using Flucoma's Ugens this is where the spikes/dropouts are clearly heard/seen:

Ndef(\yo, {  FluidHPSS.ar(WhiteNoise.ar)}).fadeTime_(1);

Try reevaluating that a couple of times.

My suspicion (which I haven't been able to fully prove yet, sorry) is that this is caused by allocation in the C++ code. Looking through the source code for the UGens I don't see any use of the recommended RTAlloc method for reserving memory from the realtime memory pool and so I suspect this is being done with STL containers or something, which may be the cause of the problem. As I said, this is only a suspicion by now but perhaps this can be a pointer to help you bug hunt this.

This SuperCollider help file has more info on avoiding STL containers.

Thanks!

via https://discourse.flucoma.org/t/fluidbufstft-windowsizes-above-1024-dont-work/953

Buffer#loadToFloatArray fails to work with window sizes greater than 1024. The tmp file doesn't seem to get written, so perhaps there's an upper channel limit that's failing in scsynth.

getToFloatArray works, so the process itself seems to be ok (although streaming caps the size).

I investigated the first example of the helpfile after all UTs using MFCC failed.

1- bufmelbands works (I thought the new interface of MaxCoeff was the culprit)
2- the rt version of MFCC work
3- the order of parameters match the order of the 'client' in core

so I reckon it has to do with the new maxRegime and that this class def has a bug but I can't find it.

I think this would be easy to implement and a little more elegant for accessing the demo audio files in all the help files. I know this is not the right implementation because there is no FluCoMa class, but something like:

FluCoMa {
	*audioFilesFolder {
		^(File.realpath(FluidBufPitch.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/");
	}
}

This suggestion came from an attendee at the CIRMMT workshop and I thought it was a good one. Thoughts?

This makes sense since it doesn't know the shape of its input and output layers until training begins.

I think this should be made explicit in the documentation, or even better have it throw an error saying that it needs to be trained first. Right now, .write doesn't throw any error and a file is written to disk, but it only contains the word null.

When using inverse=1 and passing a buffer to resynth the resulting sample rate of that buffer seems to be the sampleRate of the magnitude buffer * the hopSize, which makes sense when the sampleRate of the magnitude buffer is ~86 Hz because it is frames.

However, if I pack my own buffer of magnitudes (as below) and then as to do resynthesis with this buffer the sample rate of the resynth buffer is 22579200 Hz. (This is 44100 * 512).

I tried setting the sampleRate of my "hand rolled" buffer to ~86, but this doesn't seem to have worked, maybe a lang vs. server thing. I even post the buffers to be sure that both magnitude and phase buffers have the appropriate sampleRate before doing the inverse STFT (~86 Hz), and it still is giving me a sample rate of 22579200 Hz).

(note that this doesn't happen if I pass the original magnitude buffer to the inverse STFT).

Code that reproduces:

s.boot;

~src = Buffer.read(s,FluidFilesPath("Olencki-TenTromboneLongTones-M.wav"));

~src.play;

// see the spectrogram of magnitudes
(
~mags = Buffer(s);
~phases = Buffer(s);
FluidBufSTFT.processBlocking(s,~src,magnitude:~mags,phase:~phases);
)

// transform the magnitudes in the frequency domain for the spectral adjustments
(
~mags.loadToFloatArray(action:{
	arg mags;
	fork{
		var new_mags = Buffer.alloc(s,~mags.numFrames,~mags.numChannels);
		
		s.sync;
		
		"mags: %".format(mags).postln;
		
		mags = mags.clump(~mags.numChannels);
		mags = mags.collect{
			arg frame, idx;
			"frame %:\t%".format(idx,frame).postln;
			frame.collect{
				arg mag, bin;
				mag = mag * bin.linlin(0,frame.size-1,0,pi/2).sin.abs;
			};
		};
		
		new_mags.loadCollection(mags.flat,action:{
			fork{
				// new_mags.sampleRate_(~mags.sampleRate);
				
				// s.sync;
				
				~resynth = Buffer(s);
				
				"magnitude: %".format(new_mags).postln;
				"phase:     %".format(~phases).postln;
				FluidBufSTFT.processBlocking(s,~src,magnitude:new_mags,phase:~phases,resynth:~resynth,inverse:1,action:{
					
					"playing resynth".postln;
					~resynth.postln;
					defer{FluidWaveform(~resynth)};
					~resynth.play;
					
				});
			}
			
		});
		
	}
});
)

Situation: Max uses the FluidStandardize object as a discrete scaler, feeding it coefficients via the json interface. The SC example is not as elegant and simple.

It seems Flucoma currently doesn't produce Supernova builds. Would be awesome to have support for that.

Normally one produces supernova builds by adding -DSUPERNOVA=On to the cmake command.

See the sc_add_server_plugin function that comes with the sc cmake scripts for inspiration:
https://github.com/madskjeldgaard/portedplugins/blob/c05ae18797d4c500c4dc4099af87702f488dbfd9/cmake_modules/SuperColliderServerPlugin.cmake#L57

Here's an example of linking a library to a supernova object:
https://github.com/madskjeldgaard/portedplugins/blob/main/CMakeLists.txt#L94

thanks!

FluidAudioTransport class definition has the two arguments in and in2. Because it is a species: transformer, it is receiving the in argument from the transformer template, however it is not getting the in2 argument from anywhere, so when the schelp gets made it isn't in there (see below). in2 is in the YAML though (see below). Also I noticed that argument in2 is not actually in the JSON. Is it being excluded because it is an audio input to a UGen?

I see two potential fixes:

1. Create yet another species (hey, these are gonna get nuked in the future anyways) and template for a transformer with 2 audio inputs "transformer2" (implemented here: flucoma/flucoma-docs#72)
2. Make sure the argument in2 is put into the JSON from the c++ (I looked in the C++ [see below] to see if I could get at that, but didn't want to go rogue)

We've had a couple of requests for Supernova support (see #82 but also elsewhere, by email etc., and a mention here supercollider/supercollider#5347 (comment)). I was asked by @dyfer to actually write this up properly and publicly (some weeks ago, sorry).

The short version is that we can't reasonably do it without some additions to the SC PlugIn API: nasty hacks that we currently do for scsynth are untenable for Supernova. These are:

1. Being able to SendReply from the NRT command thread, including sending return data
2. (by extension) a way of copying and cleaning up void* pointers to ReplyAddr (or Supernova equivalent) without needing to concern ourselves with what they are
3. a counterpart to fBufAlloc to also free SndBuffer data
   • There should, ideally, be an API function to copy a SndBuffer struct, because what works in scsynth (*a = *b). doesn't work in supernova, and frankly, I don't think plugin authors should be burdened with that knowledge)

At the 'and a pony too' end of the list, I'd also like there to be better support for Plugins to invoke and test for the existence of PluginCmds, but that's more to do with what I'd like to be able to do in the future

As it stands, the brittle workaround for these lacunae in scsynth is to commit a code crime and just link to a bunch of internals that I have no business knowing or caring about. This then breaks if our plugins are built against a different SC (or Boost) to what's running. Not ideal but (I stress), not a decision made lightly.

The SendReply stuff

The core issue is that a lot of FluCoMa isn't real-time UGen plugins, which are the typical use for the API.

• Our FluidBuf* objects behave like BufGens
• FluidDataSet and FluidLabelSet (and friends) behave more like Buffer

This means that a lot of our time is spent in the NRT command thread, via the AsyncPluginCmd thread, and rely on being able to respond to messages from the client with actual data (rather than just \done). To further complicate matters, many of our processors offer the option of delegating long-running computation to a worker thread so that the command FIFO doesn't get choked and unresponsive.

I guess a first question someone might ask is why not? I.e. why not just try and make these things 'normal' RT plugins and use UnitCmds to build a message interface? This was our original approach, but it ended up unworkable:

• it wasn't possible to get behaviour in the client that felt natural for bunches of commands because the timing of responses gets messy when they have to be dispatched from the RT thread (i.e. s.sync basically didn't work as expected)
• we need things on the server that can survive cmd-period. This proved to be basically impossible to do reliably when we had RT plugins acting as the core interface. The resulting code in both C++ and sclang was complex and unreliable
• There was (is?) also a bug that allows a UnitCmd to be invoked before the plugin constructor had run. That was bad, and crashy.

This underpins the motivation for wanting a NRT SendReply available in the API (and, by extension, the ability to copy and destroy the inReply pointers that get passed to PluginCmds). With this in place we are able to provide an interface that is robust and works as expected in the language. To make this happen in the current version, the Bad Things (i.e. stuff that pulls in ReplyAddr) are sandboxed away in a static library that provides the functions needed.

void* copyReplyAddress(void* inreply);
void deleteReplyAddress(void* inreply);
void SendReply(void* inReplyAddr, char* inBuf, int inSize);

This happens to work for scsynth, but I don't think can be made to work for supernova: the classes involved in supernova are (very) different, and much more buried in the details. The end result seemed to be that it would involve building the whole of supernova as a static library, and even then felt decidedly more iffy (from a pretty iffy starting point).

FWIW, I think the absence of these from the InterfaceTable is more an oversight than anything else. As far as I can tell, no-one has needed to push the AsyncPluginCmd stuff so hard before, although I understand that the SC VST project would also benefit from this. I think the approach taken there is to instead use Buffer as a hacky way to 'transmit' return data back to clients. I guess, if we knew that the above changes were never going to happen, then we could try and fall back to this, although I can't see how it works without imposing at least one additional round-trip between client and server to get the buffer data.

Buffer freeing and copying

This is much simpler! The InterfaceTable has a fBufAlloc but no corresponding free. Again, this feels like something missing because nobody needed it: if you use the BufGen facilities, then this takes care of freeing old buffers for you. However, this is limited to Buffer processing that involves a single output buffer, whereas we have a number of cases that return multiple buffers. As such, we need to manage their cleanup ourselves. The allocations involved are actually aligned, and use Boost to handle cross-platform horribleness. So, again, we do a code crime, and pull in the corresponding aligned free to ensure that stuff behaves.

Again, it works very well, code crime notwithstanding.

The issue with copying SndBuf instances in Supernova is down to how locks are added to the struct. Basically, this changes it from being POD-ish to being a non-copyable class. As such, the only way to 'copy' one in supernova is to go through field-by-field. Given that supernova itself has code to do this, then a more robust alternative to duplicating this for plugin code would be for there to be a function that worked for both scsynth and supernova, relieving plugin authors of needing to know or care about the internal details of SndBuf.

Now, unfortunately, one can't just wave one's paw and have the SC API magically extended. There is already active discussion over at SC on whether / if API changes could be enacted at all, or at least in a way that minimizes ABI-breakage chaos. I get the distinct impression that some community members would be uneasy with the prospect of adding new stuff to the API, on the sound basis of resisting function creep (although I think all of these are oversights that complete existing functionality rather than introducing new stuff).

Those discussions on the SC seem quite on and off, as people have bigger fish to fry. If they do decide to proceed, however, we could certainly offer to help out, whilst I'm still being paid to work on this.