WEEK 3 HOMEWORK – SAMPLE SOLUTIONS
IMPORTANT NOTE
These homework solutions show multiple approaches and some optional extensions for most of the
questions in the assignment. You don’t need to submit all this in your
...
WEEK 3 HOMEWORK – SAMPLE SOLUTIONS
IMPORTANT NOTE
These homework solutions show multiple approaches and some optional extensions for most of the
questions in the assignment. You don’t need to submit all this in your assignments; they’re included here
just to help you learn more – because remember, the main goal of the homework assignments, and of
the entire course, is to help you learn as much as you can, and develop your analytics skills as much as
possible!
Question 1
Using crime data from http://www.statsci.org/data/general/uscrime.txt (description at
http://www.statsci.org/data/general/uscrime.html), test to see whether there is an outlier in the last
column (number of crimes per 100,000 people). Is the lowest-crime city an outlier? Is the highest-crime
city an outlier? Use the grubbs.test function in the outliers package in R.
Here’s one possible solution. Please note that a good solution doesn’t have to try all of the possibilities in
the code; they’re shown to help you learn, but they’re not necessary.
The file HW3-Q1-fall.R contains R code and some explanation for the following approach.
First, because the Grubbs test assumes normality, we start by running a normality test that you’ll
probably remember from basic statistics: the Shapiro-Wilk test. The test actually suggests that the data
is not normally distributed (p=0.001882) – but looking at the Q-Q plot below, it seems that the reason
for the non-normality is the tails, which might imply that the test is affected by potential outliers. The
middle of the distribution looks normal, so we’ll go ahead with the Grubbs test.
Figure 1. Q-Q plot of the Crime column.
Note here that this is really a judgment call. On the one hand, it could be that the Shapiro-Wilk test is
identifying that the tails, especially on the upper end, are really not normally-distributed, enough so that
the extreme values aren’t really outliers, they’re just part of the distribution. On the other hand, it
could be that the distribution really is close enough to normal, and the reason it fails the Shapiro-Wilk
test is that there’s outlying data. The Grubbs test’s validity depends on which of these is closer to true.
In this case, let’s go on with the Grubbs test. At worst, it’ll either show that there aren’t outliers, or it’ll
identify potential outliers – then we would (if this was more than a homework assignment) investigate
those data points more carefully to see what’s going on, to determine whether they seem like a real part
of the distribution or whether they’re real outliers.
It turns out that the lowest-crime city is unlikely to be an outlier (p-value so close to 1 that it just comes
up as 1).
On the other hand, the highest-crime city might be an outlier (p=0.079), and if we remove it, the secondhighest-crime city also appears to be an outlier (p=0.028). The box-and-whisker plot below shows the
outliers more clearly.
[Show More]
.png)
