Validation form is out of date for the demo assignment 3 about mlcourse.ai

[Yorko]

I don't see the point. Questions 3.6 and 3.7 are in total correspondence with the assignment.

[fralik]

The point is that anyone who tries to do demo assignments now will get their answers marked incorrect. Despite the answers being correct. Take me: I did the assignment, checked my answers and then boom! Answers I gave to 3.6 and 3.7 were incorrect. But the questions are relatively straightforward. I triple-checked my code. I was still getting the same values, which were incorrect according to the web form.

I then went on and checked out a previous version of the assignment. Run my code (exactly the same code). Hey! I got the correct answers.

Now to wrap up. Possible answers to question 3.6 from the web form are: 0.895; 0.856; 0.788; 0.845.

Code with the answer to question 3.6 (corresponds to section 3.1 in the notebook)

tree = DecisionTreeClassifier(max_depth=3, random_state=17)
tree.fit(X_train, y_train)
tree_predictions = tree.predict(X_test)
accuracy_score(y_test, tree_predictions)

If one runs it in ed28f5f (current master), the result is 0.790.
If one runs it in 152a534 (previous version), the result is 0.845.

Please note how the answer from the previous version matches web form expectations.

So I am saying that although this is a demo assignment, the validation form doesn't validate answers of the current version of the assignment.

[Yorko]

fixed that in 4130b09

I've found the reason. Maybe it's a bug in sklearn implementation: accuracy depends on the order of columns. First categorical, then numeric features (1) or vice versa (2) - accuracy differs very much in (1) and (2). Now checked with docker, shall be all right.

Thanks!