Competition: AAIA'17 Data Mining Challenge: Helping AI to Play Hearthstone

Machine learning challenge. The task for participants is to construct a prediction model that can learn how to accurately evaluate particular intra-game states.

AAIA'17 Data Mining Challenge is the fourth data mining competition organized within the framework of International Symposium Advances in Artificial Intelligence and Applications (https://fedcsis.org/2017/aaia).

The competition attracted 296 teams from 28 different countries. We got 8th place - top 2.7% result!

The task is to predict the probability of winning the game for evaluation quality moves of the AI player.

Data volume:

  • Train – 2 000 000 + 1 250 000 (deprecated) objects (game stats)
  • Test – 750 000 objects (game stats)
  • Evaluation metric - AUC

public leaderboard – 5%, private leaderboard – 95%. Results link.

results table

Used algorithms:

  1. Xgboost (Extreme Gradient Boosting)
  2. sklearn.linear_model.LogisticRegression

Data:

  • Hearthstone - computer game using thematic cards.
  • Basic data - current move, the number of crystals, player health, id Heroes, the number of cards.
  • Data on the cards on the table of the player and the opponent (only minions) - the id of the cards, Attack, health, other abilities.
  • Data on cards in the player's hand - id cards, card types (minions, spell, weapon), Attack, health, etc.

Xgboost features importance:

features importance

Final result is algorithms results blending using weighted arithmetic average:

0.3 * xgboostResult + 0.7 * logreResult

Also tried: KNN, GradientBoostingClassifier with worse results.