Natural Language Processing (NLP) Using Deep Learning

Recent Developments in NLP

In the past couple of years, the amount of research and breakthroughs in NLP has skyrocketed. Now every month we get to see state-of-the-art NLP models and architectures. 

This incredible rise of NLP can be traced back to the paper – “Attention is All You Need” by Google AI in June 2017. This breakthrough has spawned so many new and exciting NLP models and some key developments are listed below:

• Transformer Architecture - It achieved state-of-the-art status on sequence modeling tasks. It can handle long-range dependencies in text with ease

• BERT - This architecture by Google took the Transformer a notch higher and achieved state-of-the-art performance in Language Modeling

• GPT and GPT 2 - OpenAI’s GPT was state-of-the-art language model till BERT toppled it from the top spot and soon they launched GPT 2 a colossal language model which was trained on many GB’s of text scraped from the web

• Transformers Library - This amazing library by Huggingface has democratized the use of Transformers. It has a large number of Transformer based pre-trained models that can also be fine-tuned.

In the modules given below, all the projects will be implemented using PyTorch.

Language Modeling

In this module, you will understand a very important sequence modeling task that is known as language modeling. This is the main ingredient in many modern NLP tasks such as machine translation, text summarization, speech recognition, etc.

Project: Building an Autocomplete System

In this project, you will train a Language Model from scratch and use it to automatically give next word suggestions to an input text. You will use PyTorch to implement this project.

Sequence-to-Sequence Modeling

Once you are familiar with the language model, you will learn about the sequence-to-sequence models wherein both the input and the output are sequences of different lengths. You will also get a thorough understanding of the architecture along with the attention mechanism.

Project: Neural Machine Translation

In this project you will get to apply sequence-to-sequence modeling to build a deep learning model to translate one language to another automatically. 

Transformers and Transfer Learning in NLP

The next step in your journey is the Transformer. In this module, you will understand the Transformers models in detail. You will also learn how Transformers are used to perform transfer learning in NLP. You will get to use the Transformers library by Huggingface.

Project: Text Classification using Transformers

You will fine-tune a pre-trained Transformer model to perform text classification on a new dataset. The domain of the new dataset will be different from that of the dataset on which the Transformer was trained.

Project: Build a Dialog System using RASA

Conversational agents or chatbots have been instrumental in across several industries. In this project you will build a chatbot using RASA, an open source. You will also learn to deploy the chatbot in slack.

Project: Build a Voice Command System

In this project you will not only use text data but audio data as well. The objective of this project is to build a deep learning system to convert a speech input to text. It can be used for tasks such as controlling home automation systems by just a few voice commands.