Finally built a text classifier - Part 2

-
After applying the Naive Bayes, I found that the accuracy was only 0.7738980350504514. Now This is the part where I was stuck for a long time. It took a lot of hitting the old notes and random blogs to find out how can I use a Support Vector Machine based model to get a better accuracy.

The advantages of support vector machines are:
  1. Effective in high dimensional spaces.
  2. Still effective in cases where a number of dimensions are greater than the number of samples.
  3. Uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.
  4. Versatile: different Kernel functions can be specified for the decision function. Common kernels are provided, but it is also possible to specify custom kernels.


 The disadvantages of support vector machines include:
  1. If the number of features is much greater than the number of samples, avoid over-fitting in choosing Kernel functions and regularization term is crucial.
  2. SVMs do not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation (see Scores and probabilities, below).
These are some of the reasons which made me decide to go the svm route. 

The accuracy did jump to 82%, but it still wasn't that great. I was clearly missing something. That's when I came across GridSearch. Almost all the classifiers will have various parameters which can be tuned to obtain optimal performance. Scikit gives an extremely useful tool ‘GridSearchCV’. 

I started by creating a list of parameters for which we would like to do performance tuning. All the parameters name start with the classifier name (remember the arbitrary name we gave). E.g. vect__ngram_range; here we are telling to use unigram and bigrams and choose the one which is optimal.

Next, I create an instance of the grid search by passing the classifier, parameters and n_jobs=-1 which tells to use multiple cores from user machine. 

****Right about now would be a good time to learn how to embed jupyter code into a blog from github, but I am going to save that for later! You can follow it on the github code link mentioned later.******

I was able to increase the accuracy for the Naive Bayes Classifier to 90.6% and the svm to an 89%. Ah! the sweet taste of success!

github: https://github.com/jayeetaroy/Text-analytics-Python

Comments

Post a Comment

Popular posts from this blog

Finally built a text classifier - Part 1

-
Image
A text classifier primarily uses machine learning principles to classify text into different categories. These supervised learning based classifiers are used in a variety of different places such as spam filters, user comment categorization, etc.  Text categorization is the grouping of documents into a fixed number of predefined classes. Each document can be in multiple, exactly one, or no category at all. Using machine learning, the objective is to classifiers from examples which perform the category tasks automatically. This is a supervised learning problem. Since categories may overlap, each category is treated as a separate group. Supervised Learning for Text Classification. PART-I : Training 1) During training, a feature extractor is used to transform each input value to a feature set. 2) These feature sets, which capture the basic information about each input that should be used to categorize it. 3) Pairs of feature sets and labels are
Read more

Linear Regression Theory

-
Image
The term "linearity" in algebra refers to a linear relationship between two or more variables. If we draw this relationship in a two-dimensional space (between two variables, in this case), we get a straight line. Linear regression assumes a linear or straight line relationship between the input variables (X) and the single output variable (y). More specifically, that output (y) can be calculated from a linear combination of the input variables (X). When there is a single input variable, the method is referred to as a simple linear regression. In simple linear regression, we can use statistics on the training data to estimate the coefficients required by the model to make predictions on new data. The line for a simple linear regression model can be written as: y = b0 + b1 * x where b0 and b1 are the coefficients we must estimate from the training data. This reminds me of my high school geometry lessons. It's funny how this points out that when we write the e
Read more