In this lecture, we're going to talk a little bit about logistic regression. These are binary outcome models. So let's get started. So first of all, we have to discuss what is a logit model. And a logit model is a binary model, binary meaning two, which is used in marketing, and it uses a logistic function to model a binary dependent variable. So what is a binary dependent variable? The dependent variable y, that's our dependent variable, is a binary choice. It's either going to be a 0 or a 1. So you might want to think of some examples of what's a 0 or 1 variable. Whether or not a customer purchases a widget, you can code that as a 1 if they buy, a 0 if they don't buy. Gender typically is described as a binary variable, male-female, 0, 1. Republican, Democrat political party affiliation, something like that. So here is the formula for the logit function. And in many ways, studying logistic regression, it's all about understanding this logit function. The right-hand side of the equation is essentially the same as linear regression. So it's about being able to convert these Y equaling 1, Y equaling 0, and their probabilities into this logit function. And then, also being able to back it out for interpretation purposes. So let's start here. Here's the logit function. The logit function is the probability that Y equals 1. And we take the probability that Y is equal to 1 in the numerator here. And then, we take the probability that Y is not equal to 1 in the denominator. So let's study that first part there a little bit. There we go. So if we have the probability of Y equaling 1, equaling 1 is say 80%, or 0.8. Then the probability that Y is not equal to 1, probability that Y is not equal to 1, is equal to 1 minus 0.8, okay? And that's what's going on here, 1 minus the probability that Y is equal to 1. Recall from Kolmogorov's axioms that the sum of the probabilities have to equal 1. In other words, something happens in your probability space. Okay, so and then, you take the log of this number. So there's a lot of things going on. So let's first look at the probability of odds. And what does that mean? So the odds, as I mention, is the probability of an event occurring over 1 minus the probability of the event the event occurring. And if you want to go in the other direction, here's the formula. If you want to calculate the probability, you take the odds over 1 plus the odds. So let's take an example of the probability of some event happening, say it's 0.8. That means the probability of the event not happening is 0.2. And so, the odds are equal to 4, okay? And then, if you wanted to go from 4 back to probability, it's 4 over 5. And that, if you do the math, it'll come out to 0.8, all right? So I have here in this slide, this first column, I just have probabilities ranging from 0 to 1. And I just incremented by 10%. So 0% probability, 10% probability, 20, 30, 40, all the way up to a 100% probability. And then, I calculated the odds for each of these values. So 0 over 1 minus 0 is 0. This is the one I just did on the previous slide. Probability of 0.8 over 0.2 is odds of 4. And then, this is just all the calculations. And we can see the relationship between the probability and odds. So probabilities must range between 0 and 1 inclusive. And we can see the relationship that odds have against probability. On this slide, I have odds here. And in the other column, I calculated the log of the odds. And that's just the natural logarithm. Notice that that's undefined here, as well as here. But you can see as odds go in this direction, the log odds, is a concave curve. Next, we get to the logistic function, which is log odds here, and here are the probabilities. So on the y-axis, I have probabilities. And notice, log odds here can go from infinity in the right-hand side, and minus infinity. So it can range the full number line. But on the y-axis, you'll see the probabilities range from 0 to 1. And that's essentially what we're trying to figure out with binary variables. Are they going to buy, not by? And what's the probability that they buy? And the probability they buy is somewhere between 0 and 1, okay? So now that we have bound the range the probabilities of our outcome variable, and we can we can use this log odds that goes from 0 to minus infinity to plus infinity, we can now run some sort of regression. Before I show you how to run a logistic regression, I wanted to show you how I created those graphs. And hopefully, that'll help you get a better feel for what's going on. So probabilities. I just took a sequence of numbers from 0 to 1, and incremented by 0.1. So that's 0, 0.1, 0.2, 0.3. So that's what this command does here. And if I look at this, you can see the actual values down below. 0.0, 0.1, 0.2, etc., etc. And then, I have odds. And it's exactly as I described, is taking the probability over 1 minus the probability. And if we look at that variable, odds, there they are. And then, taking the log of the odds is just essentially taking exactly that. To calculate the natural logarithm of a number in R, you use this log function, log. There we go. And then, these next three three commands, I'm just creating little tables that you saw in the left-hand side of those PowerPoint slides. So C by means Columbine. Takes the column of the probabilities and the column of the odds and put them together into a little data frame. So let's do that. And I'll do them all so that you can see. And I'll just show you one, and there they are. They're essentially how I got the little tables in the PowerPoint slides. But now, I want to show you how to create those plots. So here, I did that same sequence for probabilities from 0 to 1. This time, instead of going increment by 0.1, I'm using length of 100. That means give me 100 numbers between 0 and 1. And the reason I do that is just for the graphing functions. It gives me a little more granularity. But you can see there, goes from 0 to 1, and there are 100 numbers of those. Again, odds is calculated in the same manner. Now, I'm going to plot probability against odds. And this is the graph that I showed you on the PowerPoint slides. I'm going to create the log odds number again, and plot that. Here we go. There we go. And then, here's the log odds against probability. And notice that this is the graph I showed you before. And this is another way to create that logistic function with that S curve, and that just looks a little nicer. And again, the key points to keep in mind is that x can range from minus infinity to positive infinity, while the y-axis ranges from 0 to 1. Now, I'm going to talk about the logistic regression. Here's our logit function. And on the right-hand side of the your your regression model, you have the same same betas and x's that you see. So here, it's beta 0 plus beta 1x, strictly speaking, 1. And because the Y is now a logit function, the interpretation of the betas becomes a little trickier. So beta 0 is the log odds of the event when x is 0. So given whatever your x is, if x is 0, then you're just left with that beta 0 amount there. And then, if your x1 is binary, beta 0 plus beta 1 is the log odds of the event when x is equal to 1. If x is some sort of continuous variable, then this really boils down to beta 0 plus beta 1 times x. So beta 1 is the difference in the log odds of the event when x is 1 compared to x is 0 if x is a binary variable. The odds, again, is the ratio of the probability that an event happens against the probability that doesn't happen. We talked about that earlier. And this is the example I gave you earlier. That's exactly the same thing. The odds ratio is the ratio of odds of an event occurring in one group compared to the odds of it occurring in another group. So P1 over 1 minus P1. So that's the odds of happening in group 1 versus the odds of it happening in group 2. And that's denoted in the denominator. The thing to sort of keep in mind is that if the odds ratio is equal to 1, that means the event is equally likely to happen in group 1 or group 2. It's equally likely. If the odds ratio is greater than 1, that means there's a higher degree of probability that it will occur in the first group. And if it's less than 1, it'll less likely to occur in the first group, and more likely to occur in the second group.