michaelgao8 / veil Goto Github PK
View Code? Open in Web Editor NEW:bride_with_veil: Python utility for de-identification of csv files in accordance with 45 CFR § 164.514
:bride_with_veil: Python utility for de-identification of csv files in accordance with 45 CFR § 164.514
Users need to be able to specify all of the files they want to deidentify with which maps and then have a script that loops over all of the files and applies the correct mappings
When the configuration is autogenerated, it can have artifacts due to .csv escaping behavior, i.e.
file1:
id:
'"column1"'
This should be handled in the generation stage or afterwards. Probably in the generation stage.
In the current iteration of this utility, the user must know about the order with which to deidentify columns. Let's take a simple example:
If dataframe
has two columns, id
and timestamp
, the two associated maps would be:
id_map: id -> new_id
time_map:id -> new_timestamp
OR
id_map: id -> new_id
time_map: new_id -> new_timestamp
The difference between these two makes a large difference to their usage.
In case 1, the user must shift the time columns BEFORE shifting the id (or else the time map will not see the associated IDs).
In case 2, the user must shift the id first BEFORE shifting the times.
This is an issue that needs to be resolved in order for ease of use for the end-user.
the two major paths this could take is to create an entity that has both the logic for the time AND the id in the same object, or to somehow make it clearer which order must be used.
Need to implement a way for the offset map to update when the underlying time offset method is moving to the start year. This may get confusing, as if this means if the currently DataFrame being processed is not of the same nature of timestamp as the original one used to create the offset_map, unintended behavior may occur.
As they are currently implemented, aliases have a particular edge case which should be handled, but are not.
When there is a column, for example id
, which exists in multiple tables but do not refer to the same id
and a user wants one instance of id
to be in an alias (and ignore others), this is not currently supported.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.