Survey Methodology
4. Weighting and Variance Estimation

This section provides a description of the weights developed for analyzing the 2002 National Transportation Availability and Use Survey data. Weighting is a process that attempts to make the estimates from the survey representative of the total population that was sampled. It does this by accounting for the chances of selecting units into the sample and making adjustments for imperfections in the sample frame.

Another purpose of weighting is to produce population estimates of counts, such as the total number of trips taken by persons with disabilities.  This estimate could not be produced without weights that sum to the population.

Although this section deals with weighting adjustments for this survey sample, it begins with the general reasons why fully adjusted weights should be used. It also describes the details, advantages, and disadvantages of weighting.

4.1 Weighting Approach

The weighting process begins with a base weight which is adjusted to account for nonresponse and undercoverage. The base weight is the inverse of the probability of selection of the sampled unit. During the weighting process, additional information from external sources, such as the Census, can be used to benchmark the weights and achieve consistency between totals from the survey data and the external sources. In order to produce estimates, weights are applied to sample data to estimate aggregate statistics. In particular, survey data are weighted to accomplish the following objectives:

• Compensate for differential probabilities of selection for households and persons;
• Reduce biases occurring because nonrespondents may have different characteristics from respondents;
• Adjust, to the extent possible, for undercoverage in the sampling frames and in the conduct of the survey; and
• Reduce the variance of the estimates by using auxiliary information.

Each final weight is the result of a series of sequential adjustments made to the base weights. As part of the weighting process, a household weight is created for all households that completed the screener interview. This household-level weight is the base weight computed as the inverse of the probability of selection of the sample telephone number adjusted for:

• Unknown residential status;
• Screener interview nonresponse;
• Multiple telephone numbers;
• Subsampling for disability status; and
• Household poststratification.

Details of the household-level weighting adjustments are described in Section 4.2, below.

The poststratified household-level weight is adjusted to create an individual-level (person) weight for each extended interview. The expressions for the weighting factors and adjustments for the individual-level weights are given in Section 4.3, below. The adjustments incorporate the within-household probability of selection of the sampled person and account for nonresponse. Similar to the creation of the household-level weights, each of the adjustments corresponds to a multiplicative weighting factor applied to the individual-level weight. For the individual-level weights the following factors are included:

• Probability of selection of the person;
• Extended interview nonresponse adjustment;
• Trimming; and
• Raking to person-level control totals.

The weights are adjusted for persons in households without a telephone number through an additional raking dimension. Details are found in Section 4.3 and 4.4, below. The derivation of the population control totals and description of raking is included in Section 4.4.

Following the description of each weighting adjustment this documentation presents the sum of the weights and the coefficient of variation, or CV, of the weights. The CV of the weights is typically used to measure variation in the weights. As the variation in the survey weights increases, reliability of survey estimates decreases. Most weighting adjustments made to the survey weights increase the variation in the weights. For example, when making the multiple telephone weighting adjustment, any household with more than one telephone has its corresponding weight multiplied by a factor less than one to adjust for its increased selection probability. This typically increases the variation in the weights. Reporting the CV of the weights following each weighting adjustment indicates which adjustments contribute significant variation to the weights and which adjustments decrease the variation in the weights.

The appendix to this section contains tables that show the effect of each step of the weighting process at the household and person levels. Throughout this report, reference is made to specific tables and rows in the appendix that indicate how the weights were adjusted.

Weighting Adjustment Method

In an ideal survey, all the units in the inference population have a chance of being selected into the sample and all those that are selected participate in the survey. In practice, neither of these conditions occurs. Some units have no chance of being selected for the sample (undercoverage) and some of the sampled units do not respond (nonresponse). If undercoverage and nonresponse are not addressed, then the estimates from the survey will be biased.

Nonresponse results in biases in survey estimates when the characteristics of respondents differ from those of nonrespondents. The size of the bias depends on the magnitude of this difference and on the response rate (Groves, 1989). The purpose of adjusting for nonresponse is to reduce the bias. A weighting class adjustment method (Brick and Kalton, 1996) is the type of nonresponse adjustment procedure used in this survey. In this procedure, nonresponse adjustments were computed and applied separately within weighting classes, where a weighting class was defined using characteristics known for both nonrespondents and respondents. For example, one knows from the telephone number what the associated Census Division is. Thus, division can be used to define weighting classes, and weighting adjustments can be computed separately for each of these classes. The adjustment reduces bias if either response rates or the survey characteristics are more similar within the weighting classes. In this survey, weighting classes were used to adjust for different types of nonresponse at different stages of data collection. At the screener interview, nonresponse occurred when residents at some telephone numbers could not be contacted (unknown residential status of the telephone) or when persons in already identified households refused to complete the screener interview (screener interview nonresponse). Nonresponse also occurred at the extended level when sampled persons refused to complete the extended interview (extended interview nonresponse). Details of the nonresponse adjustments and the weighting classes are presented in Sections 4.2 and 4.3.

The drawback to nonresponse adjustment is that it increases the variability of the weights, increasing the sampling variance of the estimates (Kish, 1992). A nonresponse adjustment is beneficial only when the reduction in bias more than compensates for the increase in variance. When the weighting classes contain sufficient cases and the adjustment factors do not become inordinately large, the effect on variances is often modest. Very large adjustment factors usually occur in weighting classes with small numbers of respondents. To avoid this situation, weighting classes with few cases are “collapsed” or combined with similar classes to form a new weighting class with a larger number of cases. Oh and Scheuren (1983) discuss some of the statistical features associated with making these adjustments.

In this survey, before creating weighting classes to adjust for nonresponse (i.e. unknown residential status, screener and extended interview nonresponse), a set of univariate profiles was produced for available variables to explore the response propensity at the different levels. These profiles are useful for identifying variables that are related to response rates. The drawback is that some of the characteristics may be correlated and the univariate profiles do not explore these relationships. A multivariate analysis is more appropriate for examining complex relationships of the characteristics and the response. To that end, the categorical search algorithm CHAID (Chi-squared Automatic Interaction Detector) (Kass, 1980) was used to create the weighting classes. Given a set of categorical predictors of response probabilities, CHAID attempts to divide the data set into groups in a stepwise fashion so that the response rates between cells are as different as possible. By fitting a log-linear model, CHAID identifies the most important predictor of response and splits the data set into categories. Each of those categories is further segmented based on other predictors. Categories of a variable that are not significantly different can be merged together. The merging and splitting continues until no more statistically significant predictors are found or until a user-specified stopping rule is met. Weighting classes with fewer than 30 respondents were combined with another “nearby” class before running CHAID. The study team also examined the cells formed in CHAID that had unusually large adjustment factors. These cells were combined with other similar cells to form new cells with smaller adjustment factors.

As noted above, weighting classes can be formed only if data are available for both responding and nonresponding units. Since nonresponse adjustments are carried out for each stage of data collection, the available data for forming classes are different for each stage. For this survey, most of the available variables are at the telephone exchange level (demographic variables such as percentage Hispanic population in the exchange, percentage of renters/owner in the exchange, etc.) in addition to the geography where the telephone is located (e.g., Census region, and Metropolitan status). A variable that indicated if a pre-notification letter was sent to the household was also at the screener (household) level. Combinations of these variables were used for the creation of weighting classes for nonresponse during the screener interview (unknown residential status and screener interview nonresponse). At the extended interview (person level), additional variables collected during the screener interview (e.g., disability status during the screener, number of people in the household, number of persons with and without disabilities, age and gender of the sampled person) were used to create weighting classes. The definitions of the weighting classes are presented in tables in the sections that describe the respective weighting adjustments in detail.

The approach to adjusting for undercoverage is somewhat different because uncovered units or persons (e.g., persons in households without a telephone) were never included in the frame from which the sample was drawn. The weights are adjusted for undercoverage by using data from external sources (control totals) in a process called poststratification (Holt and Smith, 1979). The primary objective of poststratification is to dampen potential biases arising from a combination of response errors, sampling frame undercoverage, and nonresponse. A secondary objective is to reduce sampling errors. In general, the sample is poststratified to as many independent figures as possible, subject to some constraints. In this discussion the term poststratification is used loosely and includes raking, a form of multidimensional poststratification (Brackstone and Rao, 1979). For this survey, the control totals are derived from the Census 2000 Summary Files 1 and 3 for the United States published by the U.S. Census Bureau. Details of the creation of the control totals at the person level are described in Section 4.4.

4.2 Household-level Weights

This section is divided into seven sub-sections each describing the steps involved in creating the household-level weights. The first sub-section reviews the creation of household-level base weights as the inverse of the probability of selection of the telephone number. The four subsequent sub-sections describe the adjustments made to the base weights. These adjustments account for unknown residential status, screener interview nonresponse, subsampling of households occupied by only persons without disabilities, and households with multiple telephone numbers. The sixth sub-section describes how the household-level weights are poststratified to control totals for the number of households in the U.S. The final sub-section reviews the adjustment that reflects the subsampling of households that are occupied by both persons with and persons without disabilities.

(1) Base Weights

The first step in the weighting process for the data from this survey was creating a household weight for each completed screener interview. Because the screener captured data mainly for sampling purposes, this weight was not used for analytical purposes. However, this weight was a key element in the computation of the person weights.

The RDD sample was drawn using a list-assisted approach from a frame of 100 banks[11] with at least one listed telephone number. Using this approach, a bank is drawn for the frame and two digits are randomly generated to complete the sampled telephone number. The base weight of a telephone number is then computed as the inverse of the probability of selecting the number, that is the ratio of the total number of 100 banks multiplied by 100 to the number of telephone numbers sampled.

The base weight BSW_i  for the i-th telephone number is

where

n = the number of telephone numbers sampled; and

N = the number of banks.

For this survey there were 40,000 telephone numbers sampled from a total of 2,585,275 working banks. Thus, the base weights of all households sampled were originally the same prior to the adjustments for unknown residential status, screener nonresponse, further subsampling of households, etc.

Telephone numbers of households where only persons without disabilities reside were subsampled at a rate of approximately one-third. If a household had at least one resident with disabilities and at least one resident without disabilities then the household was selected to interview a person with disabilities. Such households were subsampled at a rate of approximately one-third for the interview of a person without disabilities. Sub-section 4 discusses the weighting adjustment that accounts for the subsampling of the households with only residents without disabilities. Sub-section 7 addresses the adjustment that accounts for subsampling of the households with residents without disabilities and residents with disabilities for the purpose of interviewing a person without disabilities.

(2) Unknown Residential Status Adjustment

At the end of data collection some telephone numbers could not be classified as residential despite being dialed many times. The unresolved numbers are considered to have an unknown residential status. They are telephone numbers that reached only answering machines (screener disposition code of NM) or were never answered (screener disposition of NA, ring no answer). Prior to adjusting the RDD weights for screener interview nonresponse, the study estimated the number of eligible residential telephone numbers among those numbers with unknown residential status. CASRO guidelines were used when making this estimation. Based on these guidelines, the proportion of residential telephone numbers among the numbers with an unknown residential status is estimated using the number of residential and nonresidential cases in the sample. This proportion, Pres, is computed as:

where

n_r = the number of residential numbers in the sample; and

n_nr = the number of nonresidential numbers in the sample.

For this survey, n_r = 14,460 and n_nr = 21,525, leading to an estimated proportion of 0.4018. Thus, using CASRO guidelines, we assume that 40.18 percent of the undetermined (NA and NM) cases are residential (see Table A1 row 2.1d in the appendix).

The estimated proportion of residential households among the unknown residential telephone numbers is used to adjust the weights for unknown residential status. The residential status adjusted weight, HHA1W_i , for the sample is

HHA1W_i = HHA1C_c1 ·  BSW_i

where

,

where the subscript RES denotes telephone numbers identified as residential, NRES denotes telephone numbers identified as nonresidential, and UNK_RES denotes telephone numbers with unknown residential status. The subscript c₁ is the indicator for the unknown residential status weighting class. The weighting classes were created using variables for Census region and MSA status. The classes are shown in Table 4.1.

After this adjustment, only known residential telephone numbers had positive weights. The nonresidential numbers and unknown residential status numbers (i.e., households with weights of zero) were removed from the weighting process and were not further adjusted after this step. The sum of the base weights of the residential households after this adjustment is 103,885,165 (see appendix Table A1, row 2.3).

(3) Screener Nonresponse Adjustment

Because some residential households did not complete the screener interview, it was necessary to adjust the weights for screener non-response. In this step, the household weight is adjusted within groups of similar households to account for households that did not complete the screener interview. For the RDD sample, the screener nonresponse adjusted household weight, HHA2W_i , is

HHA2W_i = HHA2F_c2 · HHA1W_i ,

where

,

and SC_R is the set of screener respondents, SC_NR is the set of screener nonrespondents, and c₂ is the indicator for the screener nonresponse weighting class. The nonresponse weighting classes were created using variables that indicated if a pre-notification letter was sent to the household[12], Census region, MSA status, percentage of rented households in the telephone exchange, and the population percentage of whites in the telephone exchange. Table 4.2 shows the screener nonresponse weighting classes. After the screener nonresponse adjustment, the sum of weights remains at 103,885,165, while the coefficient of variation (CV) increases from 1.65 to 7.53 (see appendix Table A1, rows 2.4, 3.2, and 3.3).

(4) Households With Only Persons Without Disabilities Subsampling Adjustment

Following the screener nonresponse adjustment, the weights were adjusted to account for the subsampling of households with only residents without disabilities[13]. These households were subsampled at a rate of about one-third. The households with only persons without disabilities subsample adjusted weight,  HHA3W_i , is

HHA3W_i = HHA3F_c3 · HHA2W_i ,

where

and D is the set of households with at least one resident with disabilities, ND_S is the set of households with only residents without disabilities retained in the sample and ND_NS is the set of households with only residents without disabilities that were not retained, and where c₃ is the indicator for the subsample nonresponse weighting class. The weighting classes were created using variables for Census region and MSA status. The sum of weights do not change due to this weighting adjustment, however, the weights become more variable. Notice that the weights of households with only residents without disabilities were the only weights affected by this step. A separate CV is reported for the weights associated with these households (CV_ND = 10.66) and for the weights associated with households with at least one resident with disabilities (CV_D = 7.50) (see appendix Table A1 rows 4.3 and 4.4).

(5) Multiple Telephone Adjustment

At the end of the screener interview, the interviewer collected information about the existence of additional telephone numbers and their use in the household (screener interview question SC20). If the additional telephone number was used for residential purposes (telephone not used solely for business, computer use, etc.), then the household had a greater probability of selection because it could have been selected through the other number. Approximately 12 percent of the households reported having more than one telephone used for residential purposes. Approximately 2 percent (106 households), had more than two telephones used for residential purposes. For these households, the household weight is adjusted to reflect the increased probability of selection. The multiple telephone adjusted household weight, HHA4W_i , is

HHA4W_i = HHA4F_i · HHA3W_i ,

where

This adjustment assumes that there is at most two additional telephone numbers[14]. This adjustment reduced the sum of weights from 103,885,165 to 97,106,619. The variation within type of household increased as a result of this weighting adjustment. The CV for households with only residents without disabilities increased to 21.52, while the CV for the households with residents with disabilities increased to 19.48 (see appendix Table A1 rows 5.4 and 5.5).

(6) Household Poststratification

The next step in weighting the screener interviews was to poststratify the household weights to household control totals from the Census 2000 data (Census 2000 Summary File 3 released by the U.S. Census Bureau). The poststratification adjustment reduces potential bias related to different response rates and telephone coverage for households in different regions of the United States or MSA status.

The household poststratification weight, HHA5W_i , is

HHA5W_i = HHA5F_k * HHA4W_i ,

where

where CNT_k is the control total for cell k . The poststratification cells were created using variables for Census region and MSA status.

The sum of weights before and after household poststratification are 97,106,619 and 105,480,101 respectively leading to an overall poststratification adjustment factor of 1.09 (see appendix Table A1 rows 6.2 to 6.4). The magnitude of this adjustment is sometimes used as a measure of the undercoverage of the estimate of the total number of households. The CVs for households with at least one resident with disabilities and households with only residents without disabilities are 42.53 and 43.55 respectively (see appendix Table A1 rows 6.5 and 6.6). Additional detail regarding postratification is provided in Section 4.2.

(7) Persons without Disabilities Living in Households with Persons with Disabilities Subsampling Adjustment

If a household had both persons with and without disabilities residing in it, the household was retained in the sample for the purpose of obtaining an interview of a person with disabilities. However, the household was subsampled at a rate of about one-third for the purpose of obtaining an interview of a person without disabilities. In order to account for this subampling, we adjusted the household poststratified weights. The persons without disabilities living in households with persons with disabilities subsample adjusted weight, HHA6W_i , is

HHA6W_i = HHA6F_c6 · HHA5W_i ,

where

and ND is the set of households with only residents without disabilities, D_S is the set of households with persons with and persons without disabilities and a person without disabilities was selected, and D_NS is the set of households with residents with and residents without disabilities where a person without disabilities was not selected. And c₆  is the indicator for the subsample nonresponse adjustment class. The classes for this weight adjustment were created using variables that indicate Census region and MSA status. The overall sum of the weights was not affected by this adjustment.

4.3 Person Weights

A person level final weight was created for all persons completing the extended interview. The initial person weight is the product of the final household weight and the reciprocal of the probability of selecting the respondent from all persons in the household who are of the same type (e.g. the number of persons with disabilities in the household). The initial person weight is then adjusted for nonresponse. After the person nonresponse adjustment, the variation in the weights is reduced using a procedure called trimming. The final step rakes the weights to known control totals. To deal with undercoverage of persons that could not be interviewed because they reside in nontelephone households, the raking adjustment was modified to reduce the bias from this source. Details on creating the person weights follow.

Person Initial Weight

The initial person weight is the product of the final household weight and the inverse of the probability of selecting the person within that household. For persons with disabilities and for persons without disabilities living in households with only residents without disabilities, the final household weight is given by HHA5W_i , while for persons without disabilities living in households with persons with disabilities, the final household weight is HHA6W_i . Thus, the expression for the person initial weight, PRA0W_i , is

where

and D is the set of persons with disabilities, ND is the set of persons without disabilities living in households where no person with disabilities resides, and ND_D is the set of persons without disabilities living in a household where a person with disabilities also resides. NUMDIS is the number of persons with disabilities living the household, and NUMNODIS is the number of persons without disabilities living in the household. The sum of the initial person weights for persons with disabilities and persons without disabilities are 30,515,402 and 255,332,502 respectively, and the corresponding CVs are 41.65 and 59.96 respectively (see appendix Table A2 rows 1.2 and 1.3).

Extended Interview Nonresponse Adjustment

In some households the screener interview was completed but the sampled person did not complete the extended interview. To account for sampled persons who did not complete the extended interview, we adjusted the person initial weight for extended interview nonresponse. The extended interview person nonresponse adjusted weight, PRA1W_i , is

PRA1W_i = PRA1F_c  ·  PRA0W_i  .

where

and ER is the set of eligible respondents, IN is the set of ineligible persons (deceased persons, sampled persons unknown in the household, and enumeration errors, i.e. sampled persons who were not a member of the household), NR is the set of extended interview nonrespondents; and c indicates the extended interview nonresponse weighting class. The weighting classes were created using disability status, Census region, age, the population percentage of Hispanics in the telephone exchange, and the population percentage of whites in the telephone exchange. The extended interview nonresponse weighting classes are shown in the Table 4.3. The set of ineligible persons was removed following this weighting step.

The sum of weights is not affected by this adjustment (see appendix Table A2 row 2.3). Also, the relative variation after the extended interview nonresponse adjustment is almost the same compared to the variation before the adjustment. The CV for the weights of persons with disabilities is 41.60 and the CV for the weights of persons without disabilities is 59.96 after the extended interview nonresponse adjustment (see appendix Table A2 row 2.5).

Disability Status and the Need for Trimming

Before raking, it was necessary to examine the distribution of the sample weights based on the disability status reported at the extended interview[15]. The disability status at the extended interview is used to classify persons with and without disability before raking. The disability status at the extended interview is a more reliable measure of disability because the sampled person is not always the same person who completed the screener interview. Row 3.3 in the appendix, Table A2 shows the CV before trimming. Examining the distribution of the weights by self-reported disability showed a presence of records with very large weights. For persons with disabilities, we trimmed eight weights that were larger than 250,000. For persons without disabilities we trimmed four weights greater than 350,000[16]. The trimming factor, t_i , ranged from 0.44 to 0.96 for persons with disabilities and 0.53 to 0.95 for persons without disabilities.

The trimmed weight TRMW_i is computed as

TRMW_i = TRMF_i · PRAW_i

where

where 0 < t_i < 1 .

The sum of weights after trimming for persons with disabilities was reduced from 33,453,727 to 32,865,554 and the sum of weights for persons without disabilities was reduced from 243,936,274 to 243,526,307.

Most of the weights that needed to be trimmed were large as a result of having an inordinately large number of persons of the same type in their household. After re-classifying the records based on self-reported disability, the CVs for these groups are larger. The relative variation in the weights of persons with disabilities decreased following the trimming adjustment (CV_D = 98.78). The CV for the weights of persons without disabilities is 65.99 (see appendix Table A2 rows 3.1 through 3.5).

Approximately 1.5 percent (78) of the respondents were sampled and screened as not having disabilities, but self-reported as having at least one disability during the extended questionnaire interview. These respondents represent 3.6 percent of the weighted total. The mean weight of these 78 respondents is about 79,000, while the mean weight of those sampled and screened as and self-reported as having disabilities is about 13,500.

Raked Weight

The final step in the weighting was raking the trimmed weights to population control totals to produce estimates consistent with the Census 2000 results. The specific control totals and the approach used to create them are described in Section 4.4. Raking is a commonly used estimation procedure in which estimates are controlled to marginal population totals. It can be thought of as a multidimensional poststratification procedure because the weights are poststratified to one set of control totals (a dimension), then these adjusted weights are poststratified to another dimension. The procedure continues until all dimensions are adjusted. The process is then iterated until the control totals for all the dimensions are simultaneously satisfied (at least within a specified tolerance).

The raked weight, RAKEDW_i , can be expressed as

RAKEDW_i = RAKEDF_k· TRMW_i .

The factor RAKEDW_k is determined to satisfy the conditions

,

and CNT_k is the control total for raking dimension k. The description of raking and how the control totals ( CNT_k) are created are found in Section 4.4. The sum of weights after the raking adjustment for persons with disabilities and persons without disabilities are 49,499,318 and 224,143,955 respectively (see appendix Table A2 row 4.2). The respective final CVs are 113.70 and 77.00 (see appendix Table A2 row 4.4). The CV for persons with disabilities is relatively large. One reason for this is that the raking adjustment factor for the weights of persons with disabilities is also large (1.48) (see appendix Table A2 row 4.3).

Nontelephone Adjustment

Since this was a telephone survey, persons in households without telephones did not have a chance of being selected. To reduce this bias, a special adjustment was included in the weighting process. A version of the Keeter adjustment developed by Brick, Flores-Cervantes, Wang and Hankins (1999) was implemented in this survey. Keeter (1995) noted that the telephone status of a household changes over time and households with interruptions in telephone service are similar to households without telephones. Brick, Waksberg and Keeter (1996) took this idea and translated it into a weighting method.  In general, the method works by adjusting the weights of sampled persons in telephone households who have had telephone service interruptions. The person weights for persons with interruptions in telephone service are adjusted upwards to represent persons without a telephone. In this survey, the adjustment was implemented by raking to an additional dimension created using telephone interruption, as shown in Table 4.4, and household tenure (i.e., renter or owner).

As mentioned, the Keeter adjustment was implemented by raking with an additional dimension (DIM5). The control total for DIM5 are  and , where CNT_m is the total number of noninstitutionalized persons in the United States in cell m (e.g., renter or owner) as determined by the SF3, t₄ is the percentage of persons in nontelephone households in cell class m, and  is the estimated percentage of persons in telephone households with an interruption[17] in service also in cell m. In surveys that do not collect data from nontelephone households, the percentage of persons in nontelephone households (t₄) cannot be obtained directly. Data from the March 2001 CPS and Census 2000 were used to compute t₄. On the other hand, the proportion of persons in households with an interruption in service ( ) was estimated using the sample. To reflect the variability in  (computed using the sample), replicate estimates of  were computed to generate variable control totals to be used for each replicate. Then the weights for each replicate were raked to the corresponding replicate dimensions (see codebook variables RAKEDW01_80 and the description for using them following Table A2, below).

4.4 Household Postratification, Raking and Control Totals

This section provides further details (see section 4.2.6) on the procedure used when poststratifying the household weights to household level control totals, and the development of these control totals. It also describes the procedure used to rake the person weights and the development of the raking control totals for this survey sample. Poststratification and raking are typically used to reduce the variance of the estimates, or to correct for survey undercoverage of units. The first part of this section gives a general overview of poststratification and raking. The second part describes the dimension used to poststratify the household weights. It also describes the derivation of the control totals used for poststratification. The third part describes the five dimensions used in the raking for this survey. Four of the dimensions use geography and demographic variables such as sex, age, race, and ethnicity. The fifth dimension was created to reduce the bias associated with households without a telephone. The third part of this section also describes how the control totals for the raking dimensions were derived from the 2000 Census files.

(1) The Poststratification and Raking Procedures

Poststratification is an estimation procedure in which the weights of respondents are adjusted so that the sums of the adjusted weights are equal to known population totals. The poststratified weight can be written as , where W_c  is the pre-poststratified weight of an observation in poststratification cell c, and  is a factor that represents the effect of the variable. can be written as = where W_c  represents the control total in class c and  is the sum of the weights in cell c before poststratifying.

Raking is an estimation procedure in which estimates are controlled to marginal population totals. In this survey, the adjustment to population control totals at the person level uses a raking procedure so that more auxiliary information can be included. For example, if poststratification were used, only some age/race/sex categories could be used in the adjustments, while with raking more levels of these variables and important geographic level data such as region of the country can also be included. As mentioned earlier, raking can be thought of as a multidimensional poststratification procedure, because the weights are basically poststratified to one set of control totals (a dimension), then these weights are poststratified to another dimension. After all dimensions are adjusted, the process is iterated until the control totals for all the dimensions are simultaneously satisfied (at least within a specified tolerance). The raking estimator is design-unbiased in large enough samples and is very efficient in reducing the variance of the estimates if the estimates in the cross-tabulation are consistent with a model that ignores the interactions between variables.

The raked weight can be written as , where W_cd  is the pre-raked weight of an observation in class (c, d) of the cross-tabulation,  is the effect of the first variable, and is the effect of the second variable. Note that in this formulation there is no interaction effect. In this sense, the weights are determined by the marginal distributions of the control variables. As a result, the sample sizes of the marginal distributions are the important determinants of the stability of the weighting procedure, not the classes formed by the crossing of the variables. This means that deficient classes (classes with small sample sizes) are defined by looking at the sample sizes of the margins.

(2) Poststratification Cells and Control Totals

The cells used to poststratify the household weights combined region and MSA status. Table 4.5 shows the dimension description and the household control totals.

(3) Raking Dimensions and Control Totals

The five raking dimensions used in this survey are shown in Table 4.6.

The first four dimensions in the table are created by combining demographic variables (age, sex, race, ethnicity, disability status[18], home ownership) and region. Dimension 5 is created to adjust the weights for persons in households without a telephone. The control totals for the raking dimensions were derived from the 2000 Census files except for dimension 5. Section 4.3, above, has more details on the nontelephone adjustment and the variables used to create the levels for dimension 5.

Two imputation procedures were used in this survey to fill in missing responses needed to create the raking dimensions. The first imputation technique is a completely random selection from the observed distribution and was used to impute missing responses for age and sex. For example, when imputing the missing values for self-reported age, the distributions of the responses for age were used to randomly assign an age using probabilities associated with these distributions.

The second technique is hotdeck imputation. Hotdeck imputation was used to impute race, ethnicity, the telephone interruption indicator, the length of an interruption, and rent/own status of a household. Missing values for race and ethnicity were imputed because these variables were used when creating raking dimensions (see item (2), above, for details regarding dimension creation). The other variables that were imputed using the hot deck procedure were done so in order to be able to carry out the nontelephone adjustment (see Section 4.3, sub-section (2), above, for details on this adjustment). The hotdeck approach is probably the most commonly used method for assigning values for missing responses in large-scale household surveys.

Using a hotdeck approach, a value reported by a respondent for a particular item is assigned or donated to a “similar” person who did not respond to that item. To carry out hotdeck imputation for this survey data, the respondents to an item form a pool of donors, while the nonrespondents are a group of recipients. A recipient is matched to the subset pool of donors with the same household structure. The recipient is then randomly imputed the same ethnicity or race (depending on the items that need to be imputed) from one of the donors in the pool. Once a donor is used, it is removed from the pool of donors. Table 4.7 shows the variables with imputed values, imputed counts and percentages.  These imputed values were used for weighting purposes only and not included in the data set.

The raking factor was computed as the ratio of the control total to the sum of weights before trimming and raking. This factor is, in some sense, a measure of the magnitude of the bias correction for estimates of totals.

Since the weights were already adjusted for nonresponse, the raking adjustment factor could be used as an indirect measure of under or overcoverage (Montaquila, et al., 1996). The adjustment factors confound several factors such as reporting error and residual nonresponse error, but still may be used as a rough indicator of within-household coverage error. A factor greater than unity suggests undercoverage, and a factor less than unity suggests overcoverage (these are all relative measures).

Table 4.8 shows that the raking factor of persons with disabilities is relatively large. The large adjustment factor for persons with disabilities may be caused by the mode in which the data was collected. The Census data was collected using a mail survey while the survey data for this survey was collected using a telephone survey. Males were slightly undercovered while females are overcovered, an expected result as females historically tend to respond to household surveys at a higher rate compared to males. A similar result can be seen when comparing the factors of home owners to renters.

The control totals used in the raking are derived from the Summary File 1 (SF1) and Summary File 3 (SF3) from the 2000 Census released by the U.S. Census Bureau. These files contain information referred to as the 100 percent data, which is compiled from the questions asked of all people in every housing unit. Population items included sex, age, race, ethnicity (Latino), household relationships, and group quarters.

One of the limitations of using the summary files for the control totals is the inability to produce counts that exclude the group quarters population for some dimensions used in this survey. The eligible population for this survey includes only persons in residential households (not including those in group quarters—housing units with nine or more unrelated persons). Institutionalized persons in group quarters are also excluded. These persons include those living in prisons, jails, juvenile detention facilities, psychiatric hospitals and residential treatment programs, and nursing homes for the disabled and aged, or in military barracks.

The group quarters population should be excluded from the counts in the summary files when deriving the control totals for the survey as they are not part of the population of inference. As Table 4.9 shows, the group quarters population represented 2.8 percent of the total population in the United States; as a result, approximately 7,780,000 persons must be removed from the overall population counts from the SF3.

Group quarters counts from the SF1 are only available for three age groups (less than 18, 18 to 64, and 65 years old or older). The first dimension in Table 4.6 (DIM1) for this survey requires separate counts for six age groups by gender. The next three dimensions in Table 4.6 also require counts not provided by the Census. The following is an explanation of how to calculate the counts for each dimension.

For DIM3, the totals were computed by subtracting the group quarter counts separately for each race category for the three age groups (under 18 years old, 18 to 64 years old, and 65 years or older). The new totals for the under 18 years old and 65 years or older categories were directly obtained by this subtraction. However, the totals for 18 to 64 years old were then allocated following the distribution of the DIM3 age groups (18 to 29, 30 to 49 and 50 to 64 years old) by race. Table 4.10 shows the count of people in group quarters by self-reported age groups. The dimension 1, 2, and 4 (DIM1, DIM2, and DIM4) control totals were computed using a similar procedure used for DIM3.

The last dimension (DIM5) was used to adjust for households without a telephone. The description and rationale for the dimension is given in Section 4.3. The control totals for the dimension are derived by allocating the overall control total from the SF3 using sample information. The computed total number of persons not living in group quarters for each person type was then applied to these percentages to produce the control total of the class.

References

Brackstone, G.J., and Rao, J.N.K. (1979). An investigation of raking ratio estimation. Sankhya C (41), 97‑114.

Brick, J.M., Flores-Cervantes, I., Wang, K., Hankins, T., (1999). Evaluation of the use of data on interruptions in telephone service. Proceedings of the Survey Research Methods Section of the American Statistical Association.

Brick, J.M., and Kalton, G. (1996). Handling missing data in survey research. Statistical Methods in Medical Research, 5, 215-238.

Brick, J.M., Waksberg, J., and Keeter, S. (1996). Using data on interruptions in telephone service as coverage adjustments, Survey Methodology, 22(2), 185-197.

Groves, R.M. (1989). Survey Errors and Survey Costs. New York, John Wiley and Sons.

Holt, D., and Smith, T. (1979). Poststratification. Journal of Royal Statistical Society A (142), 33-46.

Kass, G. (1980). An exploratory technique for investigating large quantities of categorical data. Applied Statistics 29, 119-127.

Keeter, S. (1995). Estimating telephone noncoverage bias from a phone survey. Public Opinion Quarterly, 59, 196-217.

Kish, L. (1992). Weighting for unequal pi. Journal of Official Statistics 8, 183-200.

Montaquila, J. M., Mohadjer, L., Waksberg, J., and Khare, M. (1996). A detailed look at coverage in the third National Health and Nutrition Examination Survey (NHANES III, 1998-1994. Proceedings of the Survey Research Methods Section of the American Statistical Association. pp. 532-537.

Oh, H.L., and Scheuren, F.J. (1983). Weighting adjustments for unit nonresponse. In Incomplete Data in Sample Surveys Volume 2, W.G. Madow, I. Olkin, and D.B. Rubin (Eds.). Academic Press, 143‑184.

[11]  A bank is defined as 100 consecutive telephone numbers with the same first eight digits including area code.

[12] Households that receive an pre-notification letter, i.e. telephone numbers with a mailable address, respond at a higher rate compared to households that do not receive a letter.

[13] This adjustment was made before poststratification for households that were not selected for extended interviews because multiple-telephone information was not collected for these cases.

[14] No household was reached through two or more different telephone numbers.

[15] Prior to this adjustment disability was classified using information from the screener interview.

[16]  The trimming was done prior to the raking adjustment; After the trimming and raking, the distribution of the weights were examined again and no further trimming was needed. The number of trimmed weights reported here is at the completion of this process.

[17] Interruption in service of one week or more.

[18] Disability status is defined using the Census 2000 definition.