Why is NLP used for Text Processing?

Data Science, AI and ML
, ,
#1

NLP uses.

#2

The usecases of NLP encompass almost anything you can do with Language in relation to a problem.

  1. Sentiment Analysis - Finding if the text is leaning towards a positive or negative sentiment.

The process of computationally identifying and categorizing opinions expressed in a piece of text, especially in order to determine whether the writer’s attitude towards a particular topic, product, etc. is positive, negative, or neutral is called Sentiment Analysis. The information present over the Internet is constantly growing resulting in a large number of texts expressing opinions in review sites, forums, blogs and different social media forums. Sentiment analysis is therefore a topic of great interest and development since it has many practical applications. It is immensely useful in figuring the overall sentiment of products (Amazon), movies (Netflix), food (Yelp),etc. Its applications include Market Research, Social Monitoring, Customer Support and Product Analytics.

  1. Text Classification - Categorizing text to various categories

Text classifiers can be used to organize, structure, and categorize almost any text data we have. For e.g. New articles can be organized by topics, chat conversations can be organized by language, support tickets can be organized by urgency etc. Other examples of text classification include:

  • Directing customer queries to the right vertical
  • Detection of spam and non-spam emails,
  • Auto tagging of customer queries
  1. Document Summarization - Compressing a paragraph/document into few words or sentences

Text summarization is the method of compressing a text document, in order to create a summary of the major points of the document. The idea of summarization is to find a subset of data which contains the information of the entire set. It’s applications include News summary(Inshorts app), Novel Summary, Book Summary (Blinkist) etc. With the overall attention span declining, the need to provide information in the shortest possible words has risen - and summarization helps solve this problem.

  1. Parts of Speech Tagging - Figuring out the various nouns, adverbs, verbs etc in the text

Identifying part of speech tags is much more complicated than it looks. This is because over time in the development of language, a single word can have different parts of speech tag in different sentences based on different contexts. This makes it impossible to have a generic mapping for POS tags. Few of its applications include:

  • Text to speech conversion
  • Word Sense Disambiguation (Teach machine to know the difference of the meaning of word ‘bears’ in “I saw a couple of bears” and “Hard work always bears fruit”)
  1. Machine translation - Translate text from one language to another

Machine Translation is the task of automatically translating one natural language into another while retaining the meaning of the original text. Translation from one language to another is complex because some of the words in the original language could have multiple meanings and these words could have different forms in the output language. Its most popular application is Google Translate and it is employed in devices like Google Home as well. Machine translation allows business transactions between partners in different countries without the need of a human interpreter.

  1. Named Entity Recognition - Identify the entities present in text

Named Entity Recognition deals with named entity mentions in text and categorizes these entities into person, organization, datetime reference etc. This is used a lot in the field of bioinformatics, molecular biology and other medical NLP applications. It also plays an important role in the overall field of Information Extraction where we try to extract knowledge from unstructured text.

  1. Conversational AI - Chat with a machine in natural language and get queries resolved

Conversational AI deals with creating an interface between machines and humans to converse in natural language. Such interfaces are known as chatbots. A user can interact in natural language with natural language, the same way he usually communicates with a human. For organizations to truly scale in terms of customer support, chatbots are increasingly adopted as the first point of contact for customer query resolution across all organizations.

So for enabling all the NLP usecases, the first challenge is to convert the text into a form that the machine can understand. For that, we need to arrive at a fundamental component of text known as tokens .

#3

Text mining identifies facts, relationships and assertions that would otherwise remain buried in the mass of textual big data. Once extracted, this information is converted into a structured form that can be further analyzed, or presented directly using clustered HTML tables, mind maps, charts, etc. Text mining employs a variety of methodologies to process the text, one of the most important of these being Natural Language Processing (NLP).

The structured data created by text mining can be integrated into databases, data warehouses or business intelligence dashboards and used for descriptive, prescriptive or predictive analytics.