Introduction
Natural language processing has evolved dramatically over the past few years owing to a rapid rise in R&D and uses of deep learning techniques. NLP consists of processes and techniques which can be used to extract meaningful information out of text data.
For decades now, NLP has been used to solve language translation challenges. It's not just the translation of texts from one language to another; it's also involved in the transformation of text from one context into another by picking up on the same underlying patterns.
The field of natural language processing (NLP) has evolved a lot in the past 50 years. We have now better model trained to represent the meaning of a sentence, better algorithms for parsing sentences and much better tools for text generation.
So let’s deep dive into evolution of NLP and what are some of the applications where NLP techniques used widely.
Table of Contents:
- What is a Natural Language Processing?
- History of NLP(from 1950 to 2022)
- How to apply NLP techniques?
- What are the applications of NLP?
- Conclusion
- What is a Natural language processing(NLP)?
A branch of linguistics, computer science, and artificial intelligence called "natural language processing" (NLP) studies how computers and human language interact, with a focus on how to train computers to process and analyze massive volumes of natural language data. The ultimate goal is to create a machine that is able to "understand" the contents of documents, including the subtle subtleties of language used in different contexts. Once the information and insights are accurately extracted from the documents, the technology can classify and arrange the documents themselves.
- History of NLP (from 1950 to 2022)
In a paper published in 1950, Alan Turing described a test for a "thinking" machine. He asserted that a machine may be said to be capable of thinking if it could participate in a conversation using a teleprinter and imitated a person so perfectly that there are no discernible distinctions. Soon later, in 1952, the Hodgkin-Huxley model demonstrated how the brain creates an electrical network by using neurons. These incidents contributed to the development of computers, natural language processing (NLP), and artificial intelligence (AI).
In the beginning, NLP used relatively basic models like the Bag-of-Words model, which only counts the number of times each word appears in a document. Nevertheless, real-world applications have corpora of millions of documents, some of which may contain millions of distinct words.
This prompted the development of another method known as TF-IDF, which got rid of the usual words or "stop words." These sophisticated algorithms did not take into account the semantic connections between words, hence the Co-occurrence Matrix was developed. After this more advanced versions of NLP evolved. One of the earliest prediction-based modelling approaches in NLP was Word2Vec. It contained strategies like Skip Gram and Continuous Bag of Words that are still widely utilized today.
After this. ELMo came into existence with a key idea of solving the problem of homonyms in word representation. Take these sentences for example, #1, “I like to play cricket” and #2, “I am watching the Julius Cesar play”. The word ‘play’ has different meanings.
Encoders and decoders for the Transformer Models were introduced alongside these advancements. It is a model that outperformed the Google Neural Machine Translation model in some tasks and makes use of attention to speed up training.
By 2018, BERT was created and published. BERT (Bidirectional Encoder Representations from Transformers) uses encoder representations of the transformer network and has ushered in a new era of natural language processing by setting numerous records for handling language-based jobs. In 2019 there was an announcement they have started leveraging BERT for their search engine and by late 2020 BERT was used in almost every English-language query.
After BERT, In 2019 An architecture called the XLNet, based on the Attention Network, was created by Google and Carnegie Mellon researchers. It claims to perform better than BERT in 20 distinct tasks
To continue the trend of Large language models, a capacity of 175 billion machine learning parameters was introduced by mid 2020. This development in natural language processing (NLP) systems known as GPT-3, which was unveiled in May 2020 and was still in beta testing as of July 2020.
In very recent development of large language models, Meta’s released Open pre-trained transformer (OPT) which was in training for three months from October 2021 to December 2022. It was officially announced on June 23, 2022 by meta for open source release of the model.
Meta also announced another language model named ‘Atlas’ on august 9, 2022 which is powerful on question-answering and fact-checking tasks.
- How to apply NLP techniques?
We briefly reviewed the development of the models used to carry out NLP jobs starting from 1950 to 2022, but what processes are required for every NLP activity, exactly? In any case, we can't merely give the machines text data. Before converting to vector format, it must first be processed, which is where text pre-processing is helpful.
Here is process to process text data into vectors that machine understands:
- Noise removal(Data cleaning)
- Normalization
- Tokenization
- Vocabulary
- Vectorization
After completing the pre-processing procedures, you are prepared to start creating the NLP models.
4) What are the applications of NLP?
Let’s find out some of real-world NLP applications that uses NLP models
- Sentiment analysis
- Spam detection
- Contextual advertisements
- Question answering
- Search engine querying
- Chatbot
5) Conclusion
In this article, we saw a brief history of NLP starting from 1950 to present 2022, we saw how NLP techniques evolved from count vectorization to word-embeddings to large language models like BERT, XLNet, and GPT-3.
References:
[1] https://www.freshgravity.com/evolution-of-natural-language-processing/
[2] https://www.analyticsvidhya.com/blog/2022/07/the-evolution-of-nlp-from-1950-to-2022/
[5] https://en.wikipedia.org/wiki/Natural_language_processing
