Getting started with NLTK

-
Natural Language Processing or NLP caught my interest because the ability to extract useful information and actionable insights from heaps of unstructured and raw textual data is very fascinating. It has applications in every industry, starting from e-commerce to property management. I've spent the past year often dabbling with one or the other text analytics package. In the past, I have primarily worked with SpaCy and for this module, I wanted to work with NLTK. These both very heavily used in the industry and have a plethora of functionalities built into them. They also have a very detailed documentation and a wide variety of tutorials available online. My target in this module is again going to be divided into two ways, one in explaining what am I working on and the approach I am taking and the other one on writing the code itself.

For NLP preprocessing, there are many open source tools available. Some of them are developed by organizations to build their own NLP applications, while some of them are open-sourced. Here is a
small list of available NLP tools:
• GATE
• SpaCy
• Open NLP
• UIMA
• Stanford toolkit
• Genism
• Natural Language Tool Kit (NLTK)

However, when it comes to the ease of use and explanation of the concepts, I felt that NLTK scores really high. NLTK is also a very good learning kit because the learning curve of Python is very fast. NLTK has incorporated most of the NLP tasks, it's very elegant and easy to work with. For all these reasons, NLTK has become one of the most popular libraries in the NLP community.

Getting started wasn't much of a challenge since I already had python installed for my previous module. I just had to install the package I was ready to get started.
For my first tutorial, I will be analyzing a Twitter dataset. Let's see how that goes.

Comments

Post a Comment

Popular posts from this blog

Finally built a text classifier - Part 1

-
Image
A text classifier primarily uses machine learning principles to classify text into different categories. These supervised learning based classifiers are used in a variety of different places such as spam filters, user comment categorization, etc.  Text categorization is the grouping of documents into a fixed number of predefined classes. Each document can be in multiple, exactly one, or no category at all. Using machine learning, the objective is to classifiers from examples which perform the category tasks automatically. This is a supervised learning problem. Since categories may overlap, each category is treated as a separate group. Supervised Learning for Text Classification. PART-I : Training 1) During training, a feature extractor is used to transform each input value to a feature set. 2) These feature sets, which capture the basic information about each input that should be used to categorize it. 3) Pairs of feature sets and labels are
Read more

Linear Regression Theory

-
Image
The term "linearity" in algebra refers to a linear relationship between two or more variables. If we draw this relationship in a two-dimensional space (between two variables, in this case), we get a straight line. Linear regression assumes a linear or straight line relationship between the input variables (X) and the single output variable (y). More specifically, that output (y) can be calculated from a linear combination of the input variables (X). When there is a single input variable, the method is referred to as a simple linear regression. In simple linear regression, we can use statistics on the training data to estimate the coefficients required by the model to make predictions on new data. The line for a simple linear regression model can be written as: y = b0 + b1 * x where b0 and b1 are the coefficients we must estimate from the training data. This reminds me of my high school geometry lessons. It's funny how this points out that when we write the e
Read more