Ranking list of top participants.

SMHP Challenge
Ranking Team Name MAE Spearmanr's Rho
1 hmjmf 1.7837 0.6516
2 Southern Taiwan Club 1.9089 0.4631
3 Kun 1.9797 0.4559
4 GAN 2.0023 0.4316
5 midas 4.0376 0.0738


The ranking for the competition is based on the results from challenge task, respectively. Specifically, a rank list of teams is produced by sorting their scores on evaluation metrics, respectively.


Task Metrics
SMHP Challenge MAE, Spearmanr's Rho and HR

The evaluation provided here can be used to obtain performances on the testing set of SMP. It contains multiple common metrics, including Spearman Ranking Correlation (Spearmanr's Rho), Mean Absolute Error (MAE), Mean Squared Error (MSE), and Hit Ratio (HR).
By quantitative evaluation, we measure the systems submitted to this challenge on a testing set. Our evaluation protocol is applied on the following criteria:
Prediction Correlation: whether the predicted popularity more correlates with the actual value of the popularity?
Prediction Error: to judge the error of the score prediction
Ranking Relevance: to measure the ranking relevance between top-k groundtruths and predicted items.