Steps to Develop a Model to Estimate School- and District-Level Postsecondary Success (ed.gov)

Skip to content
# Category: Survey Design

## Steps to Develop a Model to Estimate School- and District-Level Postsecondary Success

## Matching with or without replacement (Propensity Score Matching)

## Sample size calculation using Excel sheet

## Survey Sampling Design and Regression Analysis using SAS SURVEYREG

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3144483/

Quote:

"I now discuss different methods for forming matched pairs of treated and untreated subjects when matching on the propensity score. In doing so, several decisions must be made. First, one must choose between matching without replacement and matching with replacement (Rosenbaum, 2002). When using matching without replacement, once an untreated subject has been selected to be matched to a given treated subject, that untreated subject is no longer available for consideration as a potential match for subsequent treated subjects. As a result, each untreated subject is included in at most one matched set. In contrast, matching with replacement allows a given untreated subject to be included in more than one matched set. When matching with replacement is used, variance estimation must account for the fact that the same untreated subject may be in multiple matched sets (Hill & Reiter, 2006)."

So, this means ...

Matching with replacement: Comparison students will be matched with multiple treatment students.

Matching without replacement: One comparison student will be matched with one treatment student.

Also the definition of weights from R documentation:

weights

A vector of length n that provides the weights assigned to each unit in the matching process. Unmatched units have weights equal to . Matched treated units have weight 1. Each matched control unit has weight proportional to the number of treatment units to which it was matched, and the sum of the control weights is equal to the number of uniquely matched control units.

Source: http://www.inside-r.org/packages/cran/MatchIt/docs/matchit

I wrote this Excel program to calculate sample size for surveys.

www.nippondream.com/file/sample size calculation 11 30 2015.xlsx

Simple random sampling given the population size

Confirm that regular regression analysis produces larger standard errors. Using the total sample size as a part of the modeling process, PROC SURVEYREG achieves smaller standard errors (more precise measurement).

data IceCream;

input Grade Spending Income Kids @@;

datalines;

7 7 39 2 7 7 38 1 8 12 47 1

9 10 47 4 7 1 34 4 7 10 43 2

7 3 44 4 8 20 60 3 8 19 57 4

7 2 35 2 7 2 36 1 9 15 51 1

8 16 53 1 7 6 37 4 7 6 41 2

7 6 39 2 9 15 50 4 8 17 57 3

8 14 46 2 9 8 41 2 9 8 41 1

9 7 47 3 7 3 39 3 7 12 50 2

7 4 43 4 9 14 46 3 8 18 58 4

9 9 44 3 7 2 37 1 7 1 37 2

7 4 44 2 7 11 42 2 9 8 41 2

8 10 42 2 8 13 46 1 7 2 40 3

9 6 45 1 9 11 45 4 7 2 36 1

7 9 46 1

;

run;

proc surveyreg data=IceCream total=4000;

model Spending = Income / solution;

run;

proc reg data=icecream;

model spending=income;run;

Stratified Sampling

http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_surveyreg_sect004.htm

data IceCream;

input Grade Spending Income Kids @@;

datalines;

7 7 39 2 7 7 38 1 8 12 47 1

9 10 47 4 7 1 34 4 7 10 43 2

7 3 44 4 8 20 60 3 8 19 57 4

7 2 35 2 7 2 36 1 9 15 51 1

8 16 53 1 7 6 37 4 7 6 41 2

7 6 39 2 9 15 50 4 8 17 57 3

8 14 46 2 9 8 41 2 9 8 41 1

9 7 47 3 7 3 39 3 7 12 50 2

7 4 43 4 9 14 46 3 8 18 58 4

9 9 44 3 7 2 37 1 7 1 37 2

7 4 44 2 7 11 42 2 9 8 41 2

8 10 42 2 8 13 46 1 7 2 40 3

9 6 45 1 9 11 45 4 7 2 36 1

7 9 46 1

;

run;

data StudentTotals;

input Grade _TOTAL_;

datalines;

7 1824

8 1025

9 1151

;run;

data IceCream2;

set IceCream;

if Grade=7 then Prob=20/1824;

if Grade=8 then Prob=9/1025;

if Grade=9 then Prob=11/1151;

Weight=1/Prob;

run;

proc surveyreg data=IceCream2 total=StudentTotals;

strata Grade /list;

class Kids;

model Spending = Income / solution;

weight Weight;

run;

proc reg data=icecream2;

model spending=income;

weight Weight;

run;