deepsparkchaker / data-evaluation

Data-Evaluation

Previous evaluation steps dealt with factors such as the accuracy and generality of the model. This step assesses the degree to which the model meets the business objectives and seeks to determine if there is some business reason why this model is deficient. Another option is to test the model(s) on test applications in the real application, if time and budget constraints permit. Moreover, evaluation also assesses other data mining results generated. Data mining results involve models that are necessarily related to the original business objectives and all other findings that are not necessarily related to the original business objectives, but might also unveil additional challenges, information, or hints for future directions.

Output

Assessment of data mining results

– summarise assessment results in terms of business success criteria, including a final statement regarding whether the project already meets the initial business objectives.

Approved models

– after assessing models with respect to business success criteria, the generated models that meet the selected criteria become the approved models.

Tasks

1. Evaluate results

• Understand data mining result. Check impact for data mining goal.
• Check result against knowledge base to see if it is novel and useful.Evaluate and assess result with respect to business success criteria
• Rank results according to business success criteria. Check result impact on initial application goal.
• Are there new business objectives? (address later in project or new project?)
• State conclusions for future data mining projects.

2. Review of process

• Summarize the process review (activities that missed or should be repeated).
• Overview data mining process. Is there any overlooked factor or task? (did we correctly build the model? Did we only use attributes that we are allowed to use and that are available for future analyses?)
• Identify failures, misleading steps, possible alternative actions, unexpected paths
• Review data mining results with respect to business success

3. Determine next steps

• Analyze potential for deployment of each result. Estimate potential for improvement of current process.
• Check remaining resources to determine if they allow additional process iterations (or whether additional resources can be made available).
• Recommend alternative continuations. Refine process plan.

4. Decision

• According to the results and process review, it is decided how to proceed to the next stage (remaining resources and budget)
• Rank the possible actions. Select one of the possible actions.
• Document reasons for the choice.