Digital Signal Processing Basics and Nyquist Sampling Theorem

By: Columbia Gorge Community College

>> Good day. This is Jim Pytel from Columbia Gorge Community College. This is Digital Electronics.

This lecture is entitled Digital Signal Processing Basics and the Nyquist Sampling Theorem. A tiny disclaimer before we begin here. This particular unit of the Digital Electronic Series is at most a brief outline of digital signal processing. Entire courses, entire textbooks and entire careers are made upon this technology and my feeble attempt at putting 45 total minutes of garbage on the internet is in no way meant to represent the summation of human knowledge about DSP. What I'm trying to do with this particular unit is present a basic overview of the whole DSP process and the various methods that are used to convert analog signals to digital and vice-versa, and digital signals back to analog.

What I'm hoping to accomplish with this particular unit is for you to obtain at least a technician level understanding of DSP, being made aware of its role in modern industrial and communication systems. I'm skipping all the crazy math. I'm making broad sweeping generalizations, waving my hands at other things, and most definitely telling you to ignore the man behind the curtain about others. So let's begin with DSP, and how I like to begin these lectures, let's start this off with a common character that pretty much if you're breathing probably should be able to recognize. What I want you to start off with is just taking the upper right-hand corner of your notebook and draw Pacman. And most of you guys have probably drawn a character that is semicircular in nature with a little wedge out of it, sometimes he's got an eye. And the deal is that's not Pacman, that's your idealized representation of Pacman. Over here in my upper right-hand corner of the screen is the actual Pacman as displayed by your 1980s era Atari or whatever game system.

Notice how he is pixelated and blocky. However, it's close enough to round, your brain perceives him as such. And that's the point of digital signal processing. It's to represent analog data as an approximated digital version or vice-versa to represent digital data as an approximate representation of an analog signal that is close enough. And the reason why we're doing this, there are certain advantages and they kind of go back to those beginning advantages I talked about digital in the first place, okay? So the advantages of converting some naturally occurring analog signal to digital form you could potentially remove unaudited noise, we could potentially increase or reduce the amplitude of certain signals, can encode that digital data for secure transmission. Say, for example, cell phone, you can detect and correct errors, you could perhaps store that digital data a little bit better and, like I said, you're going right back to the very most basic lecture that I talked about in digital electronics in the first place - why are we using digital in the first place, because of these basic advantages, it can be processed, stored, transmitted and reproduced more accurately and reliably and easily, that's the reason why we're taking this analog data and converting it to digital in the first place. There is a certain basic block diagram to understand DSP, which I've got written right here.

Digital Signal Processing Basics and Nyquist Sampling Theorem

I've got a certain analog signal coming in in the left and it's being put into an input filter. So, say for example, this is a naturally occurring analog signal that has a spurious high frequency noise you do not want. Say, for example, you would want to just go ahead and put in a low pass filter and get rid of that high frequency noise. So there's no sense including it in your dataset if it's something you don't want to begin with. So there's this input filter getting rid of something that you may not potentially want to include. This next block here is what's known as a sample and hold circuit. Okay, the sample and hold circuit is basically taking that smoothly varying analog signal, which I've represented right here on the left-hand side with this red and basically the sample and hold circuit at a certain frequency it is sampling, i.e., looking at that analog data. And in between the times it samples, let's say it looks at the data right there, right there, right there, right there, so on and so forth, basically it is sampling at that point and it is holding it at that value until the next sample occurs because I know these things are smoothly changing over the period of time, however, there is a certain frequency at which we are sampling, okay? So it's sampling that data, holding it at the value it was sampled at until the next sample occurs, at which point it goes on to this thing called an ADC, this is really kind of the heart or the first steps of digital signal processing.

Okay, what is an ADC? It's an analog to digital converter, so ADC, analog to digital converter. And it's using that sampled positions, and what it's doing it's quantizing it. So what does quantization mean? Basically, it's assigning binary value to those analog inputs. So, again, quantization is basically assigning a binary value to those analog examples.

And what I've drawn here on the right-hand side is a quantized representation of that same analog signal, so where we have given these are supposed to represent binary values, okay? These are supposed to represent binary values here. in our particular case what we've got is the [inaudible] quantization of that analog wave form, let's say for example this is a digital multi-meter, you know, it's sampling voltage levels smoothly varying at certain period of time and it's assigning a binary value to each one of those, at which point this binary data is now being fed into this DSP. What is the DSP, it's the digital signal processor, very similar to a microprocessor except it's handling some pretty advanced math at an incredibly, incredibly fast rate. When you think about a varying voltage signal, every 16.67 milliseconds it's gone through its entire sequence from high positive, back to zero, to low negative, back to zero. You've got to sample that at an incredibly fast rate. Think about speech, think about music, all these things are occurring in real time, that digital signal processor is having to keep up with that. And what is that digital signal processor doing? Well, it's processing and it's storing it, transmitting and, or reproducing that analog data, and removing, basically all those advantages, that's what the digital signal processor is doing.

It's going ahead and changing that binary representation of the analog data to a better form that might be used a little bit later. What is the next block in this diagram here? What's the DAC, D-A-C? If ADC stands for analog to digital converter, what does DAD stand for? Obviously, it stands for digital to analog converter, digital to analog converter, and that's basically what your iPod or your iPhone is doing to that digital representation of music that's stored on it, it's taking the digital representation, it's making an analog wave form, which is being sent to the speakers, however, there might be an output filter because notice the stair step-like approximation, our digital data looks like there? Perhaps we want to go ahead and smooth that output so smoothly varying analog output which could perhaps represent music or speech or some other quantity. So this very basic block diagram is going to be used over and over when we discuss DSP. We are going to go into in-depth ADC and the DAC and some of the methods used to go ahead to do that, but your understanding of this basic step-by-step block diagram is pretty essential to your understanding of DSP. So I've only briefly mentioned this concept of sampling and frequencies, and I've also told you that, okay, I'm going to go ahead and forego the crazy math. All right, we do have to actually discuss some of the crazy math and how we're going to do this is actually in a pretty easy method to discuss the sampling here. So we're going to go ahead and discuss some pretty crazy complicated math in the form of what's called the Nyquist Sampling Theorem. We're going to have Lassie, the world's most famous dog, present it to you.

So to set up a joke here what we've got is Lassie laying by the road asleep, and Farmer Brown revs up his big old pickup truck and takes off for work and as he makes the turn going into town he looks at Lassie and sees that she's asleep. As soon the dust from Farmer's Brown pickup truck disappears over the hill, Timmy falls in a well. So what does Lassie do? Lassie responds to his cries, cries for help. Goes ahead, saves him, pulls him out of the well, gets him back inside the house. She lays back down and tries to get a nap. Two seconds go by, Timmy is in the medicine cabinet, he's ingested a bunch of codeine.

So Lassie has got to run inside, go ahead and get the codeine out of Timmy and perhaps administer first aid, CPR, perhaps even using an AED, and now Timmy is doing fine. She puts him on the couch. She goes back out, lays down and tries to take a nap. Pretty soon Timmy gets into the gun cabinet and he is shooting wildly in all different directions. Lassie goes ahead, calls the SWAT Team, gets the SWAT Team in there. They subdue Timmy and then, finally, Timmy is a little bit tired from today's activity, falls asleep.

And then Lassie goes back out to the driveway and falls asleep again, at which point Farmer Brown comes back from work. And as he's making a turn into the driveway he sees Lassie asleep by the driveway. He look at her and he says, lazy dog. The point of that joke is Farmer Brown did not sample Lassie at the correct frequency. He only looked at her two times during that day, and she happened to be in the same position.

So what this Nyquist Sampling Theorem states is you've got to go ahead and sample something at least twice the frequency at which it is naturally occurring. And had Farmer Brown been there more than two times during the day he would have realized that Lassie had a pretty incredibly difficult day. And that's point of the Nyquist Sampling Theorem, you're going to go ahead and have to sample a naturally occurring analog piece of data at twice the frequency that you wish to capture to go ahead and reliably reproduce that data. A simple illustration of the Nyquist Sampling Theorem gone terribly wrong, let's use the Farmer Brown method.

Here what we've got is an analog signal of a certain frequency. Let's say it is a United States distribution wave, so it's 60 hertz, and let's sample it at 60 hertz, okay? And let's say our first sample occurs here, the next sample occurs here, here, here. And because we are inappropriately sampling this at an inappropriate frequency very clear, you're like, oh, it's obviously 170 volts DC signal.

Do you get the point here? It is not a DC signal at 170 volts constantly because it is taking a value from zero to positive 170, back to zero, down to negative 170 with an RMS value of 120 volts. So that is a misapplication of the Nyquist Sampling Theorem, very clearly demonstrating that it is being aliased, you know, think about an alias? It is a disguise, okay, it is not a DC so you know, in fact, it's an AC signal and this misapplication, this Farmer Brown method of just looking at it not with the correct frequency you're going to get all sorts of bad value. So just say, for example, our first sample is right there, let's do that one in blue, our first sample is right there, right there, right there. If you're occurring at that 60 hertz frequency you're going to say, okay, it is obviously a certain DC value and it's not, okay? So you're incorrectly sampling this thing. The Nyquist Sampling Theorem is we're going to have to sample this way and in this particular example it's 60 hertz, we're going to have to sample it at 120 hertz to accurately represent this wave. So the naturally occurring period of this is right there for a single cycle.

So let's say we sample there and there, what we're getting is a positive maximum and a negative maximum. It is not exactly the prettiest rendition of a wave, but if we know that we are sampling a sign wave to begin with we can accurately recreate that sign wave knowing that it peaks out at a certain value and it bellies out at another negative value. There are advantages perhaps of sampling at an increased frequency because you could potentially gain better representation of it, but we're not necessarily always sampling just a pure sign wave. And this is where the crazy, crazy, crazy math comes in. And I do want to illustrate a little bit about some of this crazy math here, and I'm going to start off with this basic statement, which may surprise you. Everything is sign waves. Every piece of information that can be - any signal is a summation of sign waves, including digital pulses, okay? And this is crazy and even to the point of reality, you know, because one of the most famous quotes from Nichola Tessla [Assumed Spelling] is if you want to find the secrets of the universe think in terms of energy frequency and vibration, reality, as we get down to it we are finding that as we get smaller and smaller we're realizing nothing is there, but vibrations and frequency, okay? So that is crazy, crazy math. What I want to talk to you here is just how you can possibly create some nations of sign wave.

And one of the simplest ways to do this is take your TI89 graphing calculator and add sign waves up of different frequencies, what you get is wave forms that are different analog values, but they are composed of sign waves. And one of the ways I want to do this is a pretty crazy example, is a digital pulse. How can I represent a digital pulse as a summation of sign waves? Well, let me go ahead and just draw some diagrams here. So look at this digital pulse here, it's not a sign wave. How can I use sign waves of different frequencies to go ahead and represent this digital pulse? Well, first off, what I'm going to have to do is just think about a sign wave, it's zero centered. This is not zero centered.

I'm going to have to raise it up by a certain amount. Okay, so there's a DC component in this, so this idealized representation of this digital - assuming these summated sign waves representation of this digital pulse, there is a DC component plus some sign waves, and those sign waves have a magnitude, a certain value and the amplitude and frequency. The more sign waves you put into this the more you can accurately represent any data, so I've chosen the digital pulse. This is one of the hardest things for you to do, to realize that, okay, sign waves can be used to create anything. If we have got a bias, we've got a DC bias here, let's just say if I took a sign wave of the fundamental frequency, let's say again it's 60 hertz, that kind of looks like a digital pulse. It's not, but it's a little bit closer.

And that is a certain amplitude, which I'm going to call one, one sign of the fundamental frequency. What if I added to that? So I've got my DC component, I've got amplitude one of the fundamental frequency, what if I added the following - one-third of sign 3F? What is one-third sign 3F, three times the fundamental frequency, but a third of the value. Well, think about it, if this is the full period with the amplitude of one, what is one-third of that? Well, it's obviously going to be maxing out and minimizing out one-third of it, but it's going to go three full cycles in that same period. And when you add one-third sign three times the frequency to one sign, one frequency, believe it or not what you're getting is kind of something like this. Does that look a little bit more pulse-like to you? And I know I'm messing it up, like I said, waving my hands at certain things here. And now continue to add components, one-fifth sign of five frequency, that summation of that DC component of fundamental frequency, three times the frequency, five times the frequency, what you're getting is a little bit closer to that pulse. Add seven times, you're still going to get a little bit of an overshoot and ringing there.

You're going to get closer and closer. This is the point, what I'm trying to say is everything is sign waves and now when I say that the Nyquist Sampling Theorem you have to do two times the highest frequency which you wish to capture. How accurately do you wish to capture this digital pulse data? Because this thing goes on forever, and this is what I'm saying about there's some crazy math here. Don't worry about the crazy math, but realize that everything is a summation of sign waves. I would have to go on forever to accurately and truly represent clear digital pulse. However, can I go twice the frequency forever? No, the answer is no, you can't.

There's a practical limitation to these. So you're going to get an idealized representation. Your Pacman can never be the truly analog version of Pacman, but it can get pretty darn close.

Now I've done an incredibly complicated example, you should realize that everything can be presented as sign waves. Think about the audible range for human speech, there's a certain frequency low and a frequency high that can occur. Why would you ever sample something out of that range if it's not going to be heard to begin with? We've got this range right here, and do not quote these things, these things I'm just doing these off the top of my heard here. If I remember right like the human hearing response is around 20 hertz if I remember right, maybe even lower, six or eight hertz, up to 20 kilohertz, that's hearing response range. And this 20 kilohertz maximum here is really kind of representative of your age. Those individuals of younger ages, I do this experiment in my Electronics I class, I get something going at 22 kilohertz and I slowly drop it down and I see which people turn their heads to hear the noise first and it's always the younger students that do so, okay? And then pretty soon I'm lowering it down, lowering it down, lowering it down, finally some of the older students say, hey, I can finally hear that, okay? So that upper range might be around 21, 22 kilohertz. Most people 16 kilohertz you're going to hear that. So the point is the human hearing range goes from that 20 to 20 kilohertz for our purposes of this example, however, human speech kind of tops out at 10 kilohertz.

So if you are potentially just talking, you're not playing a musical instrument, you're not doing anything like that, you know, what is the minimum frequency which you'd be sampling, which you should be sampling human speech at? The answer is twice its maximum, so it's 20 kilohertz. So if I'm going to do a very, very accurate representation of, well, reasonably accurate representation of human speech I want to sample it at 20 kilohertz. That wave form, that it's obviously not a pulsed wave form, but it is a wave form of human speech. And like I said previously is I know that's not a pure sign wave, but it can be represented as a summation of sign waves. The highest frequency of which is around 10 kilohertz for human speech.

I don't mean to delve into the crazy esoteric math too much here, but it is important for you to understand pretty much everything can be composed of sign waves and for you to accurately represent it there is a certain frequency which you want to and you're going to go ahead and somehow neglect certain amounts of data, which may not be necessary for your accurate representation later on. This concludes the lecture of Digital Signal Processing Basics and Nyquist Sampling Theorem. We're to go into converting analog signals to digital, basically, the analog to digital converter.

Views: 77 955 Likes: 498 Dislikes: 13
98% Likes
2% Dislikes
Quantum Computers Explained – Limits of Human Technology

Quantum Computers Explained – Limits of Human Technology For most of our history, human technology consisted of our brains, fire, and sharp sticks. While fire and sharp sticks became…

Views: 6 403 387 By: Kurzgesagt – In a Nutshell

- You know, it's always important to remember the good times (laughs). Happy Tuesday! Clicky. Clicky. Double clicky. Hey guys, this week on right now, we are going to be talking about…

Views: 213 515 By: MyHarto
College Application Essay Tips - 8-Point Essay Checklist Intro

Hi I'm Carol Barash, the founder of Story to College, and I'm here to introduce our eight-point college checklist. There's no formula for a great college essay, it's…

Views: 292 By: Story2
Graduate Program in Occupational Therapy

Hi everybody welcome to the webinar for the Occupational Therapy Program at Hofstra University. Thank you for joining us today and if you have any questions please feel free to type…

Views: 62 By: Hofstra University