INTRODUCTION

The estimates presented by the 2012 CFS may differ from the actual, unknown population values. The difference between the estimate and the population value is known as the total error of the estimate. When describing the accuracy of survey results, it is convenient to discuss total error as the sum of sampling error and nonsampling error. Sampling error is the average difference between the estimate and the result that would be obtained from a complete enumeration of the sampling frame conducted under the same survey conditions. Nonsampling error encompasses all other factors that contribute to the total error of a sample survey estimate.

The sampling error of the estimates in this publication can be estimated from the selected sample because the sample was selected using probability sampling. Common measures related to sampling error are the sampling variance, the standard error, and the coefficient of variation (CV). The sampling variance is the squared difference, averaged over all possible samples of the same size and design, between the estimator and its average value. The standard error is the square root of the sampling variance. The CV expresses the standard error as a percentage of the estimate to which it refers.

Nonsampling errors are difficult to measure and can be introduced through inadequacies in the questionnaire, nonresponse, inaccurate reporting by respondents, errors in the application of survey procedures, incorrect recording of answers, and errors in data entry and processing. In conducting the 2012 CFS, every effort has been made to minimize the effect of nonsampling errors on the estimates. Data users should take into account both the measures of sampling error and the potential effects of nonsampling error when using these estimates.

SAMPLING ERROR

Because the estimates are based on a sample, exact agreement with results that would be obtained from a complete enumeration of all shipments made in 2012 from all establishments included on the sampling frame using the same enumeration procedures is not expected. However, because probability sampling was used at each stage of selection, it is possible to estimate the sampling variability of the survey estimates. For CFS estimates, sampling variability arises from each of the three stages of sampling (See Appendix C).

The particular sample of shipments used in this survey is one of a large number of samples of the same size that could have been selected using the same design. If all possible samples had been surveyed under the same conditions, an estimate of a population parameter of interest could have been obtained from each sample. These samples give rise to a distribution of estimates for the population parameter. A statistical measure of the variability among these estimates is the standard error, which can be approximated from any one sample. The standard error is defined as the square root of the variance. The coefficient of variation (or relative standard error) of an estimator is the standard error of the estimator divided by the estimator. For the CFS, the coefficient of variation also incorporates the effect of the noise infusion disclosure avoidance method (see Disclosure Avoidance below). Note that measures of sampling variability, such as the standard error and coefficient of variation, are estimated from the sample and are also subject to sampling variability and, technically, we should refer to the estimated standard error or the estimated coefficient of variation of an estimator. However, for the sake of brevity, we have omitted this detail. It is important to note that the standard error only measures sampling variability. It does not measure systematic biases of the sample. The Census Bureau recommends that individuals using estimates contained in this report incorporate this information into their analyses, as sampling error could affect the conclusions drawn from these estimates.

An estimate from a particular sample and the standard error associated with the estimate can be used to construct a confidence interval. A confidence interval is a range about a given estimator that has a specified probability of containing the result of a complete enumeration of the sampling frame conducted under the same survey conditions. Associated with each interval is a percentage of confidence, which is interpreted as follows. If, for each possible sample, an estimate of a population parameter and its approximate standard error were obtained, then:

1. For approximately 90 percent of the possible samples, the interval from 1.833 standard errors below to 1.833 standard errors above the estimate would include the result as obtained from a complete enumeration of the sampling frame conducted under the same survey conditions.

2. For approximately 95 percent of the possible samples, the interval from 2.262 standard errors below to 2.262 standard errors above the estimate would include the result as obtained from a complete enumeration of the sampling frame conducted under the same survey conditions.

The 1.833 and 2.262 values, used to compute the 90 percent and 95 percent confidence intervals, are taken from the t-distribution with nine degrees of freedom. This takes into account the uncertainty in the estimates of the CVs and standard errors produced using the random group method with ten random groups.

To illustrate the computation of a confidence interval for an estimate of total value of shipments, assume that an esti-mate of total value is $10,750 million and the coefficient of variation for this estimate is 1.8 percent, or 0.018. First obtain the standard error of the estimate by multiplying the value of shipments estimate by its coefficient of variation. For this example, multiply $10,750 million by 0.018. This yields a standard error of $193.5 million. The upper and lower bounds of the 90 percent confidence interval are computed as $10,750 million plus or minus 1.833 times $193.5 million or $354.7 million. Consequently, the 90 percent confidence interval is $10,395 million to $11,105 million. If corresponding confidence intervals were con-structed for all possible samples of the same size and design, approximately 9 out of 10 (90 percent) of these intervals would contain the result obtained from a com-plete enumeration.

NONSAMPLING ERROR

Nonsampling error encompasses all other factors that con-tribute to the total error of a sample survey estimate and may also occur in censuses. It is often helpful to think of nonsampling error as arising from deficiencies or mistakes in the survey process. In the CFS, nonsampling error can be attributed to many sources:

• Inability to obtain information about all units in the sample.
• Response errors.
• Differences in the interpretation of the questions.
• Mistakes in coding or keying the data obtained.
• Other errors of collection, response, coverage, and processing.

Although no direct measurement of the potential biases due to nonsampling error has been obtained, precautionary steps were taken in all phases of the collection, processing, and tabulation of the data in an effort to minimize their influence. The Census Bureau recommends that individuals using estimates in this report incorporate this information into their analyses, as nonsampling error could affect the conclusions drawn from these estimates.

Some possible sources of bias that are attributed to respon-dent-conducted sampling include:

• Misunderstanding the definition of a shipment.
• Constructing an incomplete frame of shipments from which to sample.
• Ordering the shipment sampling frame by selected shipment characteristics.
• Selecting shipment records by a method other than the one specified in the questionnaire’s instructions.

The respondents who reported a shipment with unusually large value or weight when compared to the rest of their reported shipments were often contacted for verification. In such cases, if we were able to collect information on all of the large shipments a respondent had made either for a particular reporting week or for the entire quarter, we then identified those large shipments as certainty shipments.

A potential source of bias in the estimates is nonresponse. Nonresponse is defined as the inability to obtain all the intended measurements or responses from all units in the sample. Four levels of nonresponse can occur in the CFS:

• Item
• Shipment
• Quarter (reporting week)
• Establishment

Nonresponse

Item nonresponse occurs either when a particular ship-ment data item is unanswered or the response to the question fails computer or analyst edits. Nonresponse to the shipment value or weight items is corrected by imputa-tion, which is the procedure by which a missing value is replaced by a predicted value obtained from an appropriate model. (See Appendix C for a description of the imputation procedure.)

Shipment, quarter, and establishment nonresponse describe the inability to obtain any of the substantive measurements about a sampled shipment, quarter, or establishment, respectively. Shipment and quarter nonre-sponse are corrected by reweighting (see Appendix C for the descriptions of the shipment and quarter nonresponse weights). Reweighting allocates characteristics to the non-respondents in proportion to the characteristics observed for the respondents. The amount of bias introduced by this nonresponse adjustment procedure depends on the extent to which the nonrespondents differ, characteristically, from the respondents.

Establishment nonresponse is corrected during the estima-tion procedure by the industry-level adjustment weight. In most cases of establishment nonresponse, none of the four questionnaires have been returned to the Census Bureau after several attempts to elicit a response.

Response Rate

The CFS produces four different response rates: a participation response rate, a unit response rate, a weighted unit response rate, and a total quantity (item) response rate. The first three are based on the responses of the establishments selected into the survey. These unit response rates are shown in Table 1 below.

Table 1.
2012 CFS Unit Response Rates

¹ Participation Response Rate (PRR)—The Participation Response Rate is the total number of unweighted establishments that provided usable data divided by the total number of establishments in the sample (102,565) (expressed as a percentage). “Usable data” means that an establishment provided at least one shipment that was used in the tabulation of published estimates.
² Unit Response Rate (URR)—The Unit Response Rate is defined as the percentage of the total unweighted number of establishments that provided usable data to the total number of establishments that were eligible (or potentially eligible) for data collection. URRs are indicators of the performance of the data collection process in obtaining usable responses.
³ Weighted Unit Response Rate (WRR)—The Weighted Unit Response Rate
is defined as the percentage of the total weighted 2012 Economic Census
adjusted receipts of establishments that provided usable data to the total weighted economic census adjusted receipts of establishments that were eligible (or potentially eligible) for data collection. This incorporates the size of the establishment as well as its sample weight into the measure of response.

The fourth rate is based on the quality of the individual shipment data reported by the responding establishments. These total quantity response rates for the 2012 CFS are shown in Table 2 below.

Table 2.
2012 CFS Total Quantity Response Rates

¹ Total Quantity Response Rate (TQRR)—The Total Quantity Response Rate is defined as the percentage of the estimated (weighted) total of a given data item (Value, Tons, or Ton-miles) that is based on reported shipment data or from sources determined to be of equivalent-quality-to-reported. The TQRR is an item-level indicator of the “quality” of each estimate. In contrast to the Unit Response Rate (URR), these weighted response rates are computed for individual data items, so CFS produces several TQRRs. The TQRR for the CFS is based on the weighting adjustments made for establishment, quarter, or shipment nonresponse.

DEFINITIONS OF TERMS

Confidentiality

Title 13 of the U.S. Code authorizes the Census Bureau to conduct censuses and surveys. Section 9 of Title 13 requires that any information collected from the public under the authority of Title 13 be maintained as confidential. Section 214 of Title 13 and Sections 3559 and 3571 of Title 18 of the U.S. Code provide for the imposition of penalties of up to 5 years in prison and up to $250,000 in fines for wrongful disclosure of confidential census information. In accordance with Title 13, no estimates are published that would disclose the operations of an individual firm.

The Census Bureau’s internal Disclosure Review Board sets the confidentiality rules for all data releases. A checklist approach is used to ensure that all potential risks to the confidentiality of the data are considered and addressed.

Disclosure Avoidance

Disclosure is the release of data that have been deemed confidential. It generally reveals information about a specific individual or establishment or permits deduction of sensitive information about a particular individual or establishment. Disclosure avoidance is the process used to protect the confidentiality of the survey data provided by an individual or firm.

Using disclosure avoidance procedures, the Census Bureau modifies or removes the characteristics that put confidential information at risk of disclosure. Although it may appear that a table shows information about a specific individual or business, the Census Bureau has taken steps to disguise or suppress the original data while making sure the results are still useful. The techniques used by the Census Bureau to protect confidentiality in tabulations vary, depending on the type of data.

For the CFS, the primary method of disclosure avoidance is Noise Infusion. Noise Infusion is a method of disclosure avoidance in which values for each shipment are perturbed prior to tabulation by applying a random noise multiplier to shipment value and weight. Disclosure protection is accomplished in a manner that causes the vast majority of cell values to be perturbed by at most a few percentage points. For sample-based tabulations, such as CFS, the estimated relative standard error for a published cell includes both the estimated sampling error and the amount of perturbation in the estimated cell value due to noise. In extremely rare circumstances, some individual cells may be suppressed on a case-by-case basis for additional disclosure avoidance. In these cases, the data are replaced with a “D” in the tables. Other cells in the table may be suppressed because the quality of the data does not meet publication standards. By far, the most common reason for suppressing a cell is a high coefficient of variation (greater than 50 percent). These suppressed cells are shown with an “S” in the tables.

Unpublished Estimates

Estimates that had high sampling variability or poor response quality were not published. Some of these unpublished estimates can be derived directly from the CFS tables by subtracting published estimates from their respective totals. However, the (unpublished) estimates obtained by such subtraction would be subject to poor response, high sampling variability, or other factors that may make them potentially misleading. Estimates derived in this manner should not be attributed to the Census Bureau.

Individuals who use estimates in these tables to create new estimates should cite the Census Bureau as the source of only the original estimates.

More detailed descriptions for the 2012 CFS can be found in the sampling and nonsampling errors sections (see Sampling and Nonsampling Error in Appendix B).