minerva-ml / open-solution-avito-demand-prediction Goto Github PK

• check if we can translate using Google translate or Bing.
• translate all text and features into English.
• make sure that you saved DataFrame with translated text.

Explore/add timestamp features:

• day
• day of the week
• month (two values only)

Then explore/add group-by features with times.

UPDATE (from @Leoniak713):

• features are extracted
• group-by accepts these features

https://github.com/titu1994/neural-image-assessment

Add stacking transformer abstraction (pipeline block)

Add tfidf vectorizer likely followed by svd

Build model that is trained:

• train on image and categories
• use a feature based on top_1/top_5 predictins or probability for true label

add simple image features based on intensity statistics and colours

train_active.csv contains info about other user offers at the time. Could be useful

Based on: https://www.kaggle.com/shivamb/ideas-for-image-features-and-image-quality/code

Where can I find necessary requirements for the project?

Add count vectorizer likely followed by svd

How to handle missing values?
Currently replace with 0 +add new binary nan/not nan is used

In general, encode NaN as a feature. For example missing photo correlated with deal probability.
Add this as Step.

Train model on images, with targets (multi-output model):

• category name
• parent category name

Features are two vectors:

• probability distribution over category name (softmax)
• probability distribution over parent category name (softmax)

• I have column with text
• execute LabelEncoder on this
• load it to LGBM or other estimator

Is there any logic by repeting the same code twice? Why you put is_missing under train_mode flag? You can declare it out of train_mode and then in case you need return it...

    if train_mode:
        is_missing = Step(name='is_missing',
                          transformer=fe.IsMissing(**config.is_missing),
                          input_data=['input'],
                          adapter={'X': ([('input', 'X')])},
                          cache_dirpath=config.env.cache_dirpath, **kwargs)

        is_missing_valid = Step(name='is_missing_valid',
                                transformer=is_missing,
                                input_data=['input'],
                                adapter={'X': ([('input', 'X_valid')])},
                                cache_dirpath=config.env.cache_dirpath, **kwargs)

        return is_missing, is_missing_valid

    else:
        is_missing = Step(name='is_missing',
                          transformer=fe.IsMissing(**config.is_missing),
                          input_data=['input'],
                          adapter={'X': ([('input', 'X')])},
                          cache_dirpath=config.env.cache_dirpath, **kwargs)

        return is_missing

Explora best/worst results and most important features from solution_1 in the notebooks/devbook.ipynb

• Check features from Toxic competition and add them to this challenge.

Initialize your participation in the challenge:

• make submit,
• run code from branch master,
• make submit calculated from this code,
• review discussion, add some issues.

periods_train.csv contains information about item_id being on auction. Could be useful

• images
• text
• csv
• check discussion