For the book and glory

[mkrakovian]

Added a spacy classifier.
The data is evaluated and tested on workers via udf functions' calls, and is only collected on driver for model training.

Multiclass classification (41 unique labels).
Model trained on 40148 records, for 10 iterations in 27min.
Model tested on 160705 records, distributed over 4 cores in 5min.

Results:
160705 total in test, 93318 predicted correctly, 0.581
Prediction by guessing categories as uniformly distributed: 0.024

[nikolaypavlov]

Well done, @mkrakovyan. Can you send me your email to me ~~at~~ nikolaypavlov dot com?

[mkrakovian]

Done.
Thanks for the interesting workshop!!