http://www.dialog-21.ru/media/3410/loukachevitchnvrubtsovayv.pdf
Russian sentiment analysis evaluation SentiRuEval-2016 devoted to reputation monitoring of banks and telecom companies in Twitter. We describe the task, data, the procedure of data preparation, and participants’ results. At the previous evaluation SentiRuEval-2015, it was noticed that the presented machine-learning approaches significantly depended on the training collection, which was not enough for qualitative classification of the test collection because of data sparsity and time gap. The current results of the participants at SentiRuEval-2016 showed that they have made successful steps to overcome the above-mentioned problems by combining machine-learning approaches and additional manual and automatically generated lexical resources.
There are two domains datasets in Russian: banks and telecom companies.
-
bank_train_2016.xml
-
banks_test_2016.xml
-
banks_test_etalon.xml
-
tkk_test_2016.xml
-
tkk_test_etalon.xml
-
tkk_train_2016.xml
You can find official results of sentiment analysis evaluation SentiRuEval-2016 here: Results_SentiRueval_2016.xlsx
And also evaluation script is in Eval folder. Be aware to find out formulas and idea if F-measure calculation.
Loukachevitch, N. V., & Rubtsova, Y. V. (2016). SentiRuEval-2016: overcoming time gap and data sparsity in tweet sentiment analysis. In Computational Linguistics and Intellectual Technologies (pp. 416-426). http://www.dialog-21.ru/media/3410/loukachevitchnvrubtsovayv.pdf