Move NAACCR*Prep calculations into algorithms about algorithms

[depryf]

Thanks Will.

These new computations need to be in place for the 2019 submission, so we have time.

I would love to use some kind of encryption of the census instead of re-adding that really big module to the library. I am sure we will discuss more later.

[depryf]

@howew how do you want to proceed with this? I would prefer to have individual issues per algorithm that needs to be added. We also need to make sure those algorithms are discussed and agreed with the entire group.

[howew]

I agree with having separate issues per algorithm.

I spoke with NAACCR and year of dx + 1 is no longer needed - so we can cross that one off the list.

ACS/ABSM (the big module) supposedly is going away. We will need to get confirmation on that. Hopefully it does.

County at DX Analysis needs to go in.
ICCC Site Recode Major Category needs to go in.

I will set a reminder to create issues for these soon.

[depryf]

Sounds good.

And "ICCC Site Recode Major Category" is different than the current "ICCC" algorithm?

[howew]

Yes, we basically take the current code, which is 3 digits and collapse it even further to a 2 digit code representing the major category (lung, liver, brain, breast, etc). There are only like 12 or so values.

[depryf]

OK. We could do this as a second variable returned by the ICCC algorithm, or a new algorithm. Either way.

I will keep this issue opened for now, but at some point it should be closed, and individual issues should be created.

[howew]

While we are on the topic....

The ABSM stuff... the two HUGE tables. If (and that's a BIG if) we were to code it "properly" and have individual lookups for each of the hundreds of fields... would memory consumption in algorithms still be an issue? So instead of 2 giant tables we would have hundreds of smaller tables.

[depryf]

The issue was not memory consumption (well, there was some of that too); the main issue was the size of the raw data. That almost made the distribution of some other software balloon. Having many small file vs a few really big would probably not help that much. I am not sure.

If an algorithm has a negative impact (memory or physical space) for other software, then I would prefer to not add it to this library.

That's why the new framework allows you to register an algorithm yourself (all you have to do is implement the interface and register it). I know it's not what you prefer, but it's a plan B for things that are only needed in NAACCR*Prep and might cause issues for other applications.

Hopefully that one won't be needed and we won't have to use that trick.

[howew]

OK, sounds good.

[howew]

Closed - creating separate issues for ICCC Major Category and County at DX analysis calculations.