1.0 Changes about midas

[liurui39660]

Hi, the v1.0.0 is more like a re-implementation of the MIDAS algorithm plus a new FilteringCore for our new publication.

Since the new implementation (v1.0.0) is header-only, therefore one of the core concepts is to reduce the number of #include, so I use C style arrays instead of STL vector.

As for the input, I found even for the old implementation, the midas function takes int as input. So I'm not sure where the first point comes from.

For all the floating numbers, they will use float instead of double now, simply because we don't need the additional precision from double.

Note that this new implementation also includes some optimizations, please consider including them when upgrading the ruby code.

[ankane]

Hey @liurui39660, thanks for the response. The updates are great.

For input, was trying to say that input was an int previously, whereas now it is a float. I used an STL vector in the example to make it easy to swap into existing code (but realize it's not required).

Also, I'm seeing different results with this example and relations. Not sure if that's intentional.

Previously, first 10 scores (with 0.6 factor) were: 0.0, 0.0, 0.693147, 1.098612, 0.693147, 1.098612, 0.693147, 0.693147, 0.693147, 0.693147 (midas -i example.csv -o scores.txt -a 0.6 && head scores.txt)
Now, the first 10 scores are: 0, 0, 1, 2, 1, 2, 1, 1, 1, 1

[liurui39660]

Sorry I'm still not very clear about the float in the input. the operator() for all the three cores only accept int parameters; for constructors, the numRow and numColumn are also int, though factor and threshold are float. Can I know which float parameter you are referring to?

For the result, yes, as some optimizations in this new implementation changed the underlying procedure, e.g., now all the CMSs (NodeHash or EdgeHash) for the same entity (source, destination, edge) share the same layout. But the score formula of NormalCore and RelationalCore remain the same.

I also performed a quick test on my machine, indeed the first few results are integers, but later the floating numbers will come in.

[ankane]

My bad, not sure what I was looking at - source, destination, and timestamp are still int. Will strike that out above.

Thanks for all the explanations! It think it could be helpful to include upgrade instructions in the readme or a changelog, but at the very least, users can find them here.

[liurui39660]

I agree a changelog can help some, but since it's a re-implementation, maybe an "unchanglog" can summarize them better, that's partially why I didn't include it in the readme.

Basically, the high-level idea of the algorithm (core and CMS) and score formula for NormalCore and RelationalCore remain the same. Other things are more or less different from the old version.

For users, one of the major differences is the function call style. The old version provides midas and midasR which takes all the records as input, while now the three cores only accept individual records, which enables streaming processing. To upgrade, users may need to write a wrapper resembles the midas and midasR. MIDAS/example/Demo.cpp can be a reference.

[ankane]

Since people might be using the original implementation, I think it'd be nice to provide instructions for how they can upgrade to the new one. In general, Demo.cpp is helpful, but I think it'd be nice to document how to use the C++ APIs in the readme as well. Thanks again for the info.

[liurui39660]

The usage is already in the section Customization -> Custom Dataset + Custom Runner.

I'll consider later on writing an upgrade guide.

Thanks.