Lab 5: Exploratory Data Analysis
Mario Giacomazzo
February 24, 2024
Introduction
In this lab assignment, make sure you work in order of the code chunks, and knit after you complete each code chunk.
Consider the dataset Wages1 from the Ecdat
package.
## exper sex school wage
## 1 9 female 13 6.315296
## 2 12 female 12 5.479770
## 3 11 female 11 3.642170
## 4 9 female 14 4.593337
## 5 8 female 14 2.418157
## 6 9 female 14 2.094058This observational dataset records the years experienced, the years schooled, the sex, and the hourly wage for 3,294 workers in 1987. A Guide to Modern Econometrics by Marno Verbeek utilizes this data in a linear regression context. According to Marno Verbeek, this data is a subsample from the US National Longitudinal Study. The purpose of this tutorial is to practice the creative process in exploratory data analysis of asking questions and then investigating those questions using visuals and statistical summaries.
As a member of the birth class of 1988, I do not have any clue of
what the workforce looked like in 1987. It is your job to apply your
detective skills to the information hidden in this data. For future use,
utilize the modified datasetwage according to the R code
below:
wage=as.tibble(Wages1) %>%
rename(experience=exper) %>%
arrange(school)## Warning: `as.tibble()` was deprecated in tibble 2.0.0.
## ℹ Please use `as_tibble()` instead.
## ℹ The signature and semantics have changed, see `?as_tibble`.
## This warning is displayed once every 8 hours.
## Call `lifecycle::last_lifecycle_warnings()` to see where this warning was
## generated.head(wage)## # A tibble: 6 × 4
## experience sex school wage
## <int> <fct> <int> <dbl>
## 1 18 male 3 5.52
## 2 15 male 4 3.56
## 3 18 male 4 9.10
## 4 10 female 5 0.603
## 5 11 male 5 3.80
## 6 14 male 5 7.50Part 1: Questions About Variation
1.1: Use a figure to find what is the most common number of years of experience found in the data (0.5 Points).
First, use geom_bar() to investigate the distribution of
level of experience found in wage.
#1.2: Use a table to find what is the most common number of years of experience found in the data (1.5 Points).
Use group_by(experience) along with the pipe
%>% to output the most common amount of years of
experience along with the number of occurrences found in the data. The
most common value for years of experience is _____ and occurs _____
times. Fill in the blanks with the correct answers, and change
eval=FALSE to eval=TRUE and print out the
output that led you to your answer.
wage %>%
group_by(experience) %>%
COMPLETE1.3: Use a figure to find what is the maximum number for years of schooling found in the data. (0.5 Points)
First, use geom_bar() to visualize the overall
distribution of level of schooling found in the data.
#1.4: Use a table to find what is the maximum number for years of schooling found in the data. (1.5 Points)
Next, modify the code in Question 1.2 to display the maximum level of schooling and the number of workers in the data that had that number of schooling. The maximum number of years in school was ____ years which occurred _____ times in our sample. Fill in the blanks with the correct answers.
#Part 2: Questions about Covariation
2.1: Follow-up to Questions 1-2: Is there a relationship between level of schooling and level of experience? (1 Point)
Use geom_point() to display a scatter plot representing
the relationship between these two discrete numeric variables. Consider
using alpha=0.1 to indicate where the relationship is
represented the best.
The years of experience seem to _____ (increase/decrease) as the years of schooling increases. Is this what you expected to see? ____ (yes/no).
#Question: Practically, what reasons do you hypothesize for this observed relationship? Write your answer in complete sentences below:
2.2: How do hourly wages differ between males and females? (1 Point)
Use geom_freqpoly() to compare the distribution of wage
of females to the distribution of wage of males.
#Question: Where do these distributions look the same and/or where do they differ? Write your answer in complete sentences below:
2.3: How do hourly wages differ between males and females? (1.5 Points)
Use group_by() along with summarize to report the mean
wage, standard error of wage, and 95%
confidence interval for the unknown population mean hourly wage for the
various levels of sex. The standard error is equal to the
standard deviation divided by the square root of the sample size. The
95% confidence interval is approximated by obtaining the lower and upper
bound of an interval within 2 standard errors of the sample mean. Based
on the confidence limits, do we have statistical evidence to say that
the average hourly wage for men was different than the average hourly
wage for women in 1987? ______ (yes/no).
wage %>%
group_by(sex) %>%
summarize(n=n(),mean= COMPLETE,se=COMPLETE,
lb=COMPLETE,ub=COMPLETE)Question: How would you explain your answer in terms of the confidence intervals that are constructed below? Write your answer in complete sentences below:
wage %>%
group_by(sex) %>%
summarize(n=n(),mean=mean(wage),se=sd(wage)/sqrt(n),
lb=mean-2*se,ub=mean+2*se)## # A tibble: 2 × 6
## sex n mean se lb ub
## <fct> <int> <dbl> <dbl> <dbl> <dbl>
## 1 female 1569 5.15 0.0726 5.00 5.29
## 2 male 1725 6.31 0.0842 6.14 6.482.4: Does the relationship between hourly wage and years of experience differ between the sexes? (0.5 Points)
Use geom_point() along with the option
color=sex to overlay scatter plots. Does there seem to be a
clear distinction between female and male regarding this relationship?
______ (yes/no).
#2.5: Does the relationship between hourly wage and years of schooling differ between the sexes? (0.5 Points)
Repeat the graphic created in Question 4 replacing
x=experience with x=school. Does there seem to
be a clear distinction between female and male regarding this
relationship? ______ (yes/no).
#2.6: What is the relationship between hourly wage and the interaction between the years of experience and years of schooling? (0.5 Points)
The graphic below summarizes the average hourly wage for the
different combinations of schooling and experience level. The additional
facet_grid(~sex) makes comparing the relationship of the
three key numeric variables between the sexes quite easy.
wage %>%
group_by(experience,school,sex) %>%
summarize(n=n(),mean=mean(wage)) %>%
ungroup() %>%
ggplot() +
geom_tile(aes(x=experience,y=school,fill=mean)) +
scale_fill_gradientn(colors=c("black","lightskyblue","white"))+
facet_grid(~sex) + theme_dark()## `summarise()` has grouped output by 'experience', 'school'. You can override
## using the `.groups` argument.
Question: What are some differences between the sexes regarding this relationship that are apparent in this chart? Write your answer in complete sentences below:
2.7: What is the relationship between hourly wage and the interaction between the years of experience and years of schooling? (1 Point)
The next figure is similar to the previous one except that the tile color reflects the standard deviation of wage rather than the mean. Interactions of experience and school levels containing less than or equal to 10 instances are ignored in this image.
wage %>%
group_by(experience,school,sex) %>%
summarize(n=n(),sd=sd(wage)) %>%
ungroup() %>%
filter(n>10) %>%
ggplot() +
geom_tile(aes(x=experience,y=school,fill=sd)) +
scale_fill_gradientn(colors=c("black","lightskyblue","white"))+
facet_grid(~sex) + theme_dark()## `summarise()` has grouped output by 'experience', 'school'. You can override
## using the `.groups` argument.
Question: Which plot is generally darker and what does that imply? Write your answer in complete sentences below:
Question: Specifically for the scenario where a worker has 5 years of experience and 11 years of schooling, what does the extreme contrast between female and male cells imply for this figure? Write your answer in complete sentences below: