In this lesson, we're going to be a little more conceptual and look at some of the types of analyses data scientists employ to answer questions in data science. There are, broadly speaking, six categories in which data analysis fall. In the approximate order of difficulty, they are, descriptive, exploratory, inferential, predictive, causal, and mechanistic. Let's explore the goals of each of these types and look at some examples of each analysis. To start, let's look at descriptive data analysis. The goal of descriptive analysis is to describe or summarize a set of data. Whenever you get a new data set to examine, this is usually the first kind of analysis you will perform. Descriptive analysis will generate simple summaries about the samples and their measurements. You may be familiar with common descriptive statistics, including measures of central tendency e.g, mean, median, mode. Or measures of variability e.g, range, standard deviations, or variance. This type of analysis is aimed at summarizing your sample, not for generalizing the results of the analysis to a larger population, or trying to make conclusions. Description of data is separated from making interpretations. Generalizations and interpretations require additional statistical steps. Some examples of purely descriptive analysis can be seen in censuses. Here the government collects a series of measurements on all of the country's citizens, which can then be summarized. Here you are being shown the age distribution in the US, stratified by sex. The goal of this is just to describe the distribution. There is no inferences about what this means or predictions on how the data might trend in the future. It is just to show you a summary of the data collected. The goal of exploratory analysis is to examine or explore the data and find relationships that weren't previously known. Exploratory analyzes explore how different measures might be related to each other but do not confirm that relationship is causative. You've probably heard the phrase correlation does not imply causation, and exploratory analysis lie at the root of this saying. Just because you observed a relationship between two variables during exploratory analysis, it does not mean that one necessarily causes the other. Because of this, exploratory analysis, while useful for discovering new connections, should not be the final say in answering a question. It can allow you to formulate hypotheses and drive the design of future studies and data collection. But exploratory analysis alone should never be used as the final say on why or how data might be related to each other. Going back to the census example from above, rather than just summarizing the data points within a single variable, we can look at how two or more variables might be related to each other. In this plot, we can see the percent of the work force that is made up of women in various sectors, and how that has changed between 2000 and 2016. Exploring this data, we can see quite a few relationships. Looking just at the top row of the data, we can see that women make up a vast majority of nurses, and that it has slightly decreased in 16 years. While these are interesting relationships to note, the causes of these relationship is no apparent from this analysis. All exploratory analysis can tell us is that a relationship exist, not the cause. The goal of inferential analysis is to use a relatively small sample of data to or infer say something about the population at large. Inferential analysis is commonly the goal of statistical modelling. Where you have a small amount of information to extrapolate and generalise that information to a larger group. inferential analysis typically involves using the data you have to estimate that value in the population, and then give a measure of uncertainty about your estimate. Since you are moving from a small amount of data and trying to generalize to a larger population, your ability to accurately infer information about the larger population depends heavily on your sampling scheme. If the data you collect is not from a representative sample of the population, the generalizations you infer won't be accurate for the population. Unlike in our previous examples, we shouldn't be using census data in inferential analysis. A census already collects information on functionally the entire population, there is nobody left to infer to. And inferring data from the US census to another country would not be a good idea, because they US isn't necessarily representative of another country that we are trying to infer knowledge about. Instead, a better example of inferential analysis is a study in which a subset of the US population wasn't safe, for their life expectancy given the level of air pollution they experienced. This study uses the data they collected from a sample of the US population, to infer how air pollution might be impacting life expectancy in the entire US. The goal of predictive analysis is to use current data to make predictions about future data. Essentially, you are using current and historical data to find patterns, and predict the likelihood of future outcomes. Like in inferential analysis, your accuracy and predictions is dependent on measuring the right variables. If you aren't measuring the right variables to predict an outcome, your predictions aren't going to be accurate. Additionally, there are many ways to build up prediction models with some being better or worse for specific cases. But in general, having more data and a simple model, generally performs well at predicting future outcomes. All this been said, much like an exploratory analysis, just because one variable one variable may predict another, it does not mean that one causes the other. You are just capitalizing on this observed relationship to predict this second variable. A common saying is that prediction is hard, especially about the future. There aren't easy ways to gauge how well you are going to predict an event until that event has come to pass. So evaluating different approaches or models is a challenge. We spend a lot of time trying to predict things. The upcoming weather. The outcomes of sports events. And in the example we'll explore here, the outcomes of elections. We've previously mentioned Nate Silver of FiveThirtyEight, where they try and predict the outcomes of US elections, and sports matches too. Using historical polling data and trends in current polling, FiveThirtyEight builds models to predict the outcomes in the next US presidential vote, and has been fairly accurate at doing so. FiveThirtyEight's models accurately predicted the 2008 and 2012 elections, and was widely considered an outlier in the 2016 US elections, as it was one of the few models to suggest Donald Trump at having a chance of winning. The caveat to a lot of the analyses we've looked at so far is we can only see correlations and can't get at the cause of the relationships we observe. Causal analysis fills that gap. The goal of causal analysis is to see what happens to one variable when we manipulate another variable, looking at the cause and effect of the relationship. Generally, causal analysis are fairly complicated to do with observed data alone. There will always be questions as to whether are these correlation driving your conclusions, or that the assumptions underlying your analysis are valid. More often, causal analysis are applied to the results of randomized studies that were designed to identify causation. Causal analysis is often considered the gold standard in data analysis, and is seen frequently in scientific studies where scientists are trying to identify the cause of a phenomenon. But often getting appropriate data for doing a causal analysis is a challenge. One thing to note about causal analysis is that the data is usually analyzed in aggregate and observed relationships are usually average effects. So, while on average, giving a certain population a drug may alleviate the symptoms of a disease, this causal relationship may not hold true for every single affected individual. As we've said, many scientific studies allow for causal analysis. Randomized controlled trials for drugs are a prime example of this. For example, one randomized control trial examine the effect of a new drug on a treating infants with spinal muscular atrophy. Comparing a sample of infants receiving the drug versus a sample receiving a mock control. They measure various clinical outcomes in the babies and look at how the drug affects the outcomes. Mechanistic analysis are not nearly as commonly used as the previous analysis. The goal of mechanistic analysis is to understand the exact changes in variables that lead to exact changes in other variables. These analyses are exceedingly hard to use to infer much, except in simple situations or in those that are nicely modeled by deterministic equations. Given this description, it might be clear to see how mechanistic analyses are most commonly applied to physical or engineering sciences, biological sciences. For example, are far too noisy of datasets to use mechanistic analysis. Often, when these analyses are applied, the only noise in the data is measurement error, which can be accounted for. You can generally find examples of mechanistic analysis in material science experiments. Here, we have a study on biocomposites, essentially making biodegradable plastics that was examining how biocarbon particle size, functional polymer type, and concentration affected mechanical properties of the resulting plastic. They are able to do mechanistic analysis through a careful balance of controlling and manipulating variables with very accurate measures of both those variables and the desired outcome. In this lesson, we've covered the various types of data analysis, their goals. And looked at a few examples of each to demonstrate what each analysis is capable of, and importantly, what it is not.
