AskSia

Plus

This part is loosely based on Angrist, Lang, and Oreopoulos (2009).1 Numerous co...

May 9, 2024

This part is loosely based on Angrist, Lang, and Oreopoulos (2009).1 Numerous college students encounter difficulties and often require more time to complete their studies compared to the expected duration. Particularly, first-year students face heightened challenges, with some of struggling students experiencing academic probation. One primary factor contributing to this issue is inadequate preparation, notably in terms of study skills. Suppose an innovative university conducted a randomized controlled trial (RCT) involving first-year students. Within the RCT, a random half of students were offered enrollment in the Student Support Program (SSP), which provided access to peer advising and supplemental instruction services. Peer advisors, upper-class students in the same university, offered academic guidance and coping strategies through regular email communication and in-person meetings. Additionally, the supplemental instruction services aimed to enhance study habits through class-specific sessions. The remaining students formed the control group. The university then assessed the impact of the SSP by analyzing student academic performance, measured by the binary indicator variable for academic probation, indicating students with GPAs falling below a specific threshold and at risk of suspension. (1) If you are mainly interested in intention-to-treat (ITT) analysis, how would you analyze the data collected from the RCT? (2) Let Wi be a binary indicator variable that equals 1 if student i enrolls in SSP and 0 otherwise. Assume that you are interested in estimating the impacts of enrolls in SSP. How would you define never-takers, always-takers, compliers, and defiers in this context? (3) Suppose that no students in the control group enrolled in SSP. Would it be plausible to assume that there are no defiers in this study? (4) Additionally, would it be plausible to assume that there are no always-takers in this study? Explain briefly. (5) Suppose that 25% of students who received an offer to enroll in SSP accepted the offer and received the SSP services. Would it be plausible to assume that there are no never-takers in this study? (6) Continuing from the previous part, what is your estimate of the proportion of compliers? Justify your answer. (7) Suppose that the ITT estimate is −0.05. What is your estimate of the average treatment effect for compliers? Is this larger than, smaller than or equal to the ITT estimate? Explain briefly. (8) Can you point-identify the average treatment effect (ATE) for the entire population? Explain briefly. (9) Suppose that one critic comments that the impact estimate from the study design described above misses an important channel through which college students study. Many first-year college students live together and study in a group; hence, it is important how to understand the students form their study networks. Would you agree or disagree? If you agree, can you come up with a different experimental design and/or an alternative estimation method to mitigate the critic’s concern? If you do not agree, justify why it is not a concern.

1 Solution

ITT Analysis: To perform an intention-to-treat analysis, compare the academic performance of all students who were offered the SSP (regardless of whether they actually enrolled) to those who were not offered the program

Statistical Method: Use a two-sample t-test or a non-parametric equivalent to determine if there is a statistically significant difference in the proportion of students on academic probation between the two groups

1 Answer

Compare the academic performance of students offered SSP to those not offered, using appropriate statistical tests to determine significance.

Key Concept

Intention-to-treat analysis

Explanation

ITT analysis compares outcomes based on the initial treatment assignment, not on the treatment eventually received.

2 Solution

Never-takers: Students who were offered SSP but chose not to enroll

Always-takers: Students who would enroll in SSP regardless of whether they were offered it or not

Compliers: Students who enroll in SSP if offered and do not enroll if not offered

Defiers: Students who do the opposite of what they are offered; they enroll if not offered SSP and do not enroll if offered

2 Answer

Definitions of never-takers, always-takers, compliers, and defiers in the context of SSP enrollment.

Key Concept

Types of participants in RCTs

Explanation

Understanding these categories is crucial for analyzing the causal effects of interventions.

3 Solution

No Defiers Assumption: If no students in the control group enrolled in SSP, it suggests that there are no defiers, as defiers would enroll despite not being offered the program

3 Answer

It is plausible to assume there are no defiers if no control group students enrolled in SSP.

Key Concept

Defiers in RCTs

Explanation

Defiers act contrary to the assignment, and their absence simplifies causal inference.

4 Solution

No Always-takers Assumption: If the SSP was only available to those offered it, and assuming the offer was random, there should be no always-takers since their enrollment depends on the offer

4 Answer

It is plausible to assume there are no always-takers if SSP enrollment was strictly controlled.

Key Concept

Always-takers in RCTs

Explanation

Always-takers would enroll regardless of the offer, but strict control over enrollment can eliminate their presence.

5 Solution

No Never-takers Assumption: Since 25% of the students offered SSP enrolled, it implies that there are never-takers who chose not to enroll despite the offer

5 Answer

It is not plausible to assume there are no never-takers since some students offered SSP did not enroll.

Key Concept

Never-takers in RCTs

Explanation

Never-takers are individuals who do not take up the treatment despite being offered it.

6 Solution

Estimating Compliers: If 25% of those offered SSP enrolled, and assuming all of these are compliers, the proportion of compliers is 25%

6 Answer

The estimate of the proportion of compliers is 25%, assuming all who enrolled are compliers.

Key Concept

Proportion of compliers

Explanation

Compliers are those who follow the treatment assignment, and their proportion is estimated based on enrollment rates.

7 Solution

Calculating ATE for Compliers: Use the formula for the Local Average Treatment Effect (LATE), which is the ITT estimate divided by the proportion of compliers. In this case, the ATE for compliers is

-0.05 / 0.25 = -0.2

7 Answer

The average treatment effect for compliers is -0.2, which is larger in absolute value than the ITT estimate.

Key Concept

Average treatment effect for compliers

Explanation

The ATE for compliers is the ITT estimate adjusted for the proportion of compliers and reflects the effect on those who comply with the treatment assignment.

8 Solution

Point-Identifying ATE: Without additional assumptions or information, we cannot point-identify the ATE for the entire population because we only have the ITT estimate and the proportion of compliers

8 Answer

The average treatment effect for the entire population cannot be point-identified with the given information.

Key Concept

Point-identification of ATE

Explanation

Point-identification requires information on the entire distribution of treatment effects, not just the ITT estimate and complier proportion.

9 Solution

Addressing Study Network Concerns: Agree with the critic and suggest incorporating network effects into the experimental design, such as randomizing study groups or using instrumental variables to account for peer effects

9 Answer

Agree with the critic and propose an experimental design that accounts for study networks, such as randomizing study groups or using alternative estimation methods like instrumental variables.

Key Concept

Study networks in RCTs

Explanation

Study networks can influence treatment effects, and experimental designs should consider these peer interactions.