Many college courses conclude by giving students the opportunity to evaluate the course and the instructor anonymously. However, the use of these student evaluations as an indicator of course quality and teaching effectiveness is often criticized because these measures may reflect the influence of non-teaching related characteristics, such as the gender, race, or physical appearance of the instructor. The article titled, “Beauty in the classroom: instructors’ pulchritude and putative pedagogical productivity” (Hamermesh and Parker, 2005) found that instructors who are viewed to be better looking receive higher instructional ratings. (Daniel S. Hamermesh, Amy Parker, Beauty in the classroom: instructors pulchritude and putative pedagogical productivity, Economics of Education Review, Volume 24, Issue 4, August 2005, Pages 369-376, ISSN 0272-7757, 10.1016/j.econedurev.2004.07.013. http://www.sciencedirect.com/science/article/pii/S0272775704001165.)
In this lab we will analyze the data from this study in order to learn what goes into a positive professor evaluation. You will find your previous labs helpful here.
The data are contained in the evals dataframe in the
openintro package. The data were gathered from end of
semester student evaluations for a large sample of professors from the
University of Texas at Austin. In addition, six students rated the
professors’ physical appearance. (This is a slightly modified version of
the original data set that was released as part of the replication data
for Data Analysis Using Regression and Multilevel/Hierarchical
Models (Gelman and Hill, 2007).) The result is a data frame where
each row contains a different course and columns represent variables
about the courses and professors.
Chapter 24 in the online text R for Data Science, posted on our course webpage, may also be very helpful for working through this lab.
Your first activity is to go to the Help file for this dataset and become familiar with the variables. Also, go take a look at the original research paper and read the abstract. What were the authors trying to determine?
Describe the distribution of score. Is the
distribution skewed? What does that tell you about how students rate
courses? Is this what you expected to see? Why, or why not?
Excluding score, select two other variables and
describe their relationship using an appropriate visualization
(scatterplot, side-by-side boxplots, etc.).
The fundamental phenomenon suggested by the study is that better
looking teachers are evaluated more favorably. Create a scatterplot of
score by bty_avg to see if this appears to be
the case:
Before we draw conclusions about the trend, compare the number of observations in the data frame with the approximate number of points on the scatterplot. Is anything awry?
Replot the scatterplot, but this time use jitter the points (see R for Data Science). What was misleading about the initial scatterplot?
Let’s see if the apparent trend in the plot is something more
than natural variation. Fit a linear model called m_bty to
predict average professor score by average beauty rating (use the
lm function). Write out the equation for the linear model
and interpret the slope. Is average beauty score a statistically
significant predictor? Does it appear to be a practically significant
predictor?
Using the smooth geom, add the line of the
regression equation to your plot. (Use your previous lab, R for Data
Science or simply Google how to do it)
Use residual plots to evaluate whether the conditions of least squares regression are reasonable, and comment. Check for patterns in the residuals and normality. Use the code from the last lab, or see R for Data Science Section 24.2.1 in the online book.
The data set contains several variables on the beauty score of the professor: individual ratings from each of the six students who were asked to score the physical appearance of the professors and the average of these six scores. Let’s take a look at the relationship between one of these scores and the average beauty score.
ggplot(data = evals, aes(x = bty_f1lower, y = bty_avg)) +
geom_point()
evals %>%
summarise(cor(bty_avg, bty_f1lower))As expected the relationship is quite strong - after all, the average score is calculated using the individual scores. We can actually take a look at the relationships between all beauty variables (columns 13 through 19) using the following command:
ggpairs(evals, columns = 13:19) # in the GGally packageThese variables are collinear (correlated), and adding more than one of these variables to the model would not add much value to the model. In this application and with these highly-correlated predictors, it is reasonable to use the average beauty score as the single representative of these variables.
In order to see if beauty is still a significant predictor of professor score after we’ve accounted for the gender of the professor, add the gender term into the model. Run a summary and comment on the results. Also include the residual plots for a model check.
Is the above model better than the null model? Why or why not?
Does adding gender significantly improve the model?
Use the anova function to compare the two models.
Note that the estimate for gender is now called
gendermale. You’ll see this name change whenever you
introduce a categorical variable. The reason is that R recodes
gender from having the values of female and
male to being an indicator variable called
gendermale that takes a value of \(0\) for females and a value of \(1\) for males. (Such variables are often
referred to as “dummy” variables.)
As a result, for females, the parameter estimate is multiplied by zero, leaving the intercept and slope form familiar from simple regression.
\[ \begin{aligned} \widehat{score} &= \hat{\beta}_0 + \hat{\beta}_1 \times bty\_avg + \hat{\beta}_2 \times (0) \\ &= \hat{\beta}_0 + \hat{\beta}_1 \times bty\_avg\end{aligned} \]
The decision to call the indicator variable gendermale
instead ofgenderfemale has no deeper meaning. R simply
codes the category that comes first alphabetically as a \(0\). (You can change the reference level of
a categorical variable, which is the level that is coded as a 0, using
therelevel function. Use ?relevel to learn
more.)
The interpretation of the coefficients in multiple regression is
slightly different from that of simple regression. The estimate for
bty_avg reflects how much higher a group of professors is
expected to score if they have a beauty rating that is one point higher
while holding all other variables constant. In this case, that
translates into considering only professors of the same gender with
bty_avg scores that are one point apart.
We will start with a full model that predicts professor score based on rank, ethnicity, gender, language of the university where they got their degree, age, proportion of students that filled out evaluations, class size, course level, number of professors, number of credits, average beauty rating, outfit, and picture color.
Let’s run the model…
m_full <- lm(score ~ rank + ethnicity + gender + language + age + cls_perc_eval
+ cls_students + cls_level + cls_profs + cls_credits + bty_avg
+ pic_outfit + pic_color, data = evals)
summary(m_full)Check your suspicions from the previous exercise. Include the model output in your response.
Interpret the coefficient associated with the ethnicity variable.
Drop the variable with the highest p-value and re-fit the model. Did the coefficients and significance of the other explanatory variables change much? (One of the things that makes multiple regression interesting is that coefficient estimates depend on the other variables that are included in the model.) If not, what does this say about whether or not the dropped variable was collinear with the other explanatory variables?
Create a new model by simply removing all predictors with a p-value higher than 0.05 from the previous model (there are MUCH better ways to do variable selection, but we are not covering those ways right now). Compare the full model with this reduced model. Is there evidence that the full model is better?
Based on this model, describe the characteristics of a professor and course at University of Texas at Austin that would be associated with a high evaluation score.
When you are finished with the lab, go to the very top and change the
output from html_document to pdf_document. The
pdf document doesn’t look as nice, but it is easier to grade and upload
to schoology. Now turn in this PDF file to Schoology. Note the due date
and time. If Schoology says it’s late, it’s late. Make sure your
final Markdown document Knits properly and shows all your work. Look
through it to make sure everything looks organized and professional.
Also remember that if you needed output (graphs, numeric output, etc.)
to answer a question, the code to generate that output needs to be in
the lab report. Other code should not be included.