Natural Language Processing (NLP) speech to text (Technical)

Natural Language Processing (NLP) speech to text is a profound application of Deep Learning which allows the machines to understand human language and read it with a motive to act and react, as usual, humans do. The basic idea behind NLP is to feed the human language as in the form of data for intelligent tts system to consider and then utilize in various domains.

Natural Language processing has made it possible to mimic another important human trait i.e comprehension of language and has made it possible to bring about all the transformational technologies 1. The basic examples of such are Alexa and Siri on a more commercial scale and autonomous call center agents on a more operational scale.

NLP is usually deployed for two of the primary tasks namely Speech Recognition and Language Translation. Google translator is one of the most common examples of Natural Language Processing 2.  Using the deep learning algorithm for text to speech and in specific the Neural Networks, the NLP can do a lot with the unstructured text data by finding patterns of sentiments, major phrases used for specific situations, and specific text slates within a block of text.

Speech Recognition

Going a little deeper and taking one thing at a time in our impression, NLP primarily acts as a means for a very important aspect called “Speech Recognition”, in which the systems analyze the data in the forms of words either written or spoken 3. Helping us out with the text-to-speech and speech-to-text systems. For our view, we will focus on Speech-to-text which will allow us to use audio as a primary source of data and then train our model through deep learning 4.

But first and the foremost important thing is to understand the term “Speech Recognition” and how this amazing trait of human cognition was mimicked and what it helps us in achieving. You must have interacted with Alexa and Siri, how do you think it all works and in real-time, how can they understand your wish and then react accordingly 5. I once asked Siri about going on a date and it was flattering, “That’s very generous of you Hanan but”.

Can we spot some emotions within this response, how did Siri conclude that I am being generous? Why did it conclude that I am being polite as well, because if politely asked the response amounts to generosity? The answers lay within the recognize the speech technology. With a huge database of several commands on the back, the system improves itself and the more I interact with it, the better it gets. Typical of deep learning neural networks.

Speech-to-Text

Of course, one of the major perks of Natural Language Processing is converting speech into text. The real-time words that we speak or as we speak, the NLP through Deep Learning can help us with the text to speech conversion of the words we utter (in short, the sounds we make) Into the words we read (the text block we get on our computer screen or maybe a piece of paper) 6.

Let us delve into another perspective, think about this! The words you utter are subjective in nature but on an objective level, “mere sounds” 7. Special sounds that we make in a specific tone, voice, through our movement of lips and then tongue too. We have five senses and no sense mentions a word recognition faculty 8. “We can only hear sounds” so our primary sense that comes into play is, “Hearing” of course. Then this “audio data” is mined and made sense of this calling for a reaction.

The NLP works almost on the same profile, there are models based on algorithms that get the “audio data” (which of course is gibberish to them in the beginning) and then try to identify patterns and then come up with a conclusion that is text 9.

Preparing for NLP Speech-to-Text

We have already got enough of the idea of what Natural Language Processing is and how does it work. Now let us look at the technical side of it as a process as if we wish to deploy it. There are certain prerequisites to any of such project both basic and specific.

Like programming in a specific language which in our case will be Python 3 because it is one of the most reliable and productive languages given its utility and convenience, it offers to the programmers. In Machine Learning and other processes like Deep Learning and Natural Language Processing, Python offers a range of front-end solutions that help a lot.

As there is a huge range of libraries in Python that help programmers to write too little a code instead of other languages which need a lot of lines of code for the same output. However, on the other end when it comes to the execution of the codes, Python is slower but it is compensated as the coding saves a lot of time.

The Coding Environment – Convert natural language to code

For Python, we can use the “Project Jupyter” which is open-source software that facilitates the Python environment and for anyone having a knack for programming and who wants to learn it conveniently. One more and my personal preference is “google colaboratory” because of its suggestive features while writing codes. In “google colaboratory” the most convenient of its features is its suggestions as a pop-up while we are writing codes to call a “Library” or a specific function of any “library”.

There are various other platforms where one can polish their coding skills including Kaggle, HackerEarth, and they’re like. One more and the most convenient is downloading the Python on your machine itself. Although you won’t need the internet support much, to download the “libraries” which are usually built-in in all the online platforms mentioned earlier.

Putting Things into Perspective

So, keeping it simple, the main process of the speech-to-text system includes the following (steps in order from 1 – 4).

1. Uploading the audio file or the real-time voice from the microphone or a recording (audio data).
2. Second, comes the process of converting the sound into electrical signals (feature engineering).
3. Using an analog-to-digital converter for conversion of the signal into digital data (input).
4. Using the specific model to transcribe the audio(data) into text (output).

We see that speech-to-text using Python doesn’t include many complications at all and all one needs is the basic proficiency with the Python environment. Let us see how exactly all the 4 steps are deployed through a python program.

Speech to text converter in python: Kicking-off with it

Now that we have all the prior resources ready on hand, its time we try and put our skills to the test and see how things work. One most important thing while writing any program is the “pseudocode”. Simply put, an English narration of every action or step that we take by writing codes. For Example, If I am to call the “pandas library”, the code and the pseudocode will go something like this

#Now we will call the pandas library to bring in the data and start cleaning it (Pseudocode)

Import pandas as pd (Actual Code that will place the program into action)

Now let us see what libraries we will need.

1. Speech_recognition (to identify words & phrases in the input audio file and later convert them into text for human comprehension and reading)
2. In case if the code doesn’t work we need to install the speech_recognition package for which we will use the code as “conda install -c conda-forge speechrecognition” and then proceed with step one. Usually, it takes some 30 seconds to download but it may vary depending upon your internet speed.
3. Next up is “Recognizer Class”, a package of speech_recognition to for recognition fo speech and its conversion into text. There are several methods for reading a range a range of audio input sources but we will, for now, use recognize_google() API. Thus we must create an instance and an argument aud_data.
4. Now since we will be using the microphone as our source of speech, thus we need to install “PyAudio” modules through the command “conda install -c conda-forge PyAudio”.
5. We can check the available microphone options by calling the “list_microphone_names()” .
6. Then we need to set up for the conversion of spoken words to test through the Google Recognizer APi (speech recognition apis) by calling the “recognize_google()” function and further, we will pass the “aud_data” to it.
7. Finally putting the whole thing together, we can very conveniently get things done.

Packages to Install:

1. Speech Recognition
2. PyAudio

Libraries to be called:

1. Speech_recognition
2. PyAudio

Functions to call:

1. Speech_recognition
2. Recognize_google

Note: click here to download python 3.8.2

For Libraries: Once in Python, you will need to write the install commands detailed in “red”.

Concluding the Pathways

In this article, we have gone through the practical side of Artificial Neural Networks and specifically to solve a major problem that is speech-to-text. Readers can run the codes on their own and if you wish to share your insight or a problem. Feel free to share the details in the comments section, I would love to interact with you. Programming and especially the AI-related Python programming is a skill polished only if shared and discussed. I will soon be back with another such go-to article for you to not only get the gist of the major aspects of Artificial Intelligence in practice but also explore further endeavors too.

About Post Author

Muhammad Hanan Iftekhar

See author's posts