Regex RE2 toolkit about administration

[ddebrunner]

I would consider having the toolkit have a generic name, like Regex, rather than HighPerf, and then have a re2 namespace within it (e.g. com.ibm.streamsx.regex.re2).

Then any additional regex related functionality can be added to the toolkit in the future.

[leongor]

It's a good idea.
For now the toolkit name is com.ibm.streams.regex (also the namespace), and only the operator name is HighPerfRegex.
Actually, I can call the operator similarly to SPL function - RegexMatchRE2 and leave the namespace com.ibm.streams.regex.

[ddebrunner]

+1 to the toolkit, though the pattern in IBMStreams is to use namespaces starting with 'com.ibm.streamsx' (note the x). The namepace prefix com.ibm.streams is reserved for the product.

[leongor]

Ok, com.ibm.streamsx.regex indeed.
Thanks.

[ddebrunner]

Yeah, the RE2 name is better than HighPerf, as what happens when someone comes out with a faster one. :-)

[rrea]

Since we have regex capability in Text Toolkit (aka System T) can you differentiate the two and why we need both?

[hildrum]

@leongor I think I've seen a presentation on this toolkit, is there any way you can link to it from here? The toolkit you're proposing is different than SystemT, and as I recall, in addition to better performance, it also has some features that make it different than just using regexMatch and regexMatchPerlin the SPL standard toolkit (eg, reading the regexes from a file?).

[leongor]

@rrea This toolkit is performance oriented and based on one of the fastest regex libraries called RE2. Addtionally, it's designed to be stateful allowing compiling regex once and running it multiple times boosting performance even more.
@hildrum You're right, we used the toolkit to handle a very long regex read from a blacklist file, but we did it by using FileSource.
Actually, adding optional parameter to read a regex from a file sounds as very nice idea!
There is an internal IBM link about our toolkits I have added in Ngrams toolkit discussion - once we have all the approvals we'll publish it on external site.

[mikespicer]

+1 for this toolkit

[petenicholls]

+1 on streamsx.regex

[leongor]

I've uploaded the initial version to streamsx.regex repository and opened issues there.