Sub-100% Data Validity about sdv

[prupireddy]

Since the true data type was supposed to be numerical, I followed your first suggestion and just hardcoded it to numerical (as opposed to having it get detected which gave categorical). This resolved the issue.

[srinify]

Hi there @prupireddy it looks like the reason the score isn't 100% is because the PARSynthesizer model isn't adhering to the min and max values in your original dataset column.

I have 2 questions:

1. When you crafted your SingleTableMetadata object, what sdtype was detected or did you assign? You can run print(your_single_table_metdata_object) on your machine to look this up and just tell me the one one for this Days_Supplied column. I'm also curious what pandas DataFrame dtype this column is (int or float)?

2. Did you set the enforce_min_max_values parameter when defining your PARSynthesizer object? Or did you skip this and just left the model use the default value?

[prupireddy]

1. Categorical was detected (I didn't assign). The pandas dtype is float64.
2. No I did not; it used the default.

Thank you

[npatki]

Hi @prupireddy and @srinify there is currently a known issue that PARSynthesizer specifically has when the column is categorical but it is represented in a float format. I wonder if this is the root cause? #1910.

I would start by confirming whether this column (Days_Supplied) was correctly detected in the metadata, as the detection is not guaranteed to be 100% accurate. Does this column truly represent discrete categories or is it numerical? To help you decide, see this sdtypes reference.

1. If it's supposed to be numerical, please update your metadata and try with the updated version. There are currently no known bugs in PARSynthesizer for numerical data.
2. If it's supposed to categorical, then you can try the workaround I've listed in #1910.

[npatki]

Hi @prupireddy I noticed you closed the issue. Does that mean you were able to come up with a resolution?

For our knowledge (and perhaps to help others running into the same problem), you could clarify what the issue was?

[npatki]

Great. Appreciate the confirmation!