 |
Appendix
Safety Data Analysis Survey
Overview
This Appendix presents the instrument for the survey conducted by the working team from December 2001 through January 2002. The goal for conducting the survey is to acquire information on methods used to analyze transportation safety data. The purpose is to develop recommendations for facilitating improvements in transportation safety data analysis. Toward these ends, the specific survey objectives are to identify the following.
- Methods that U.S. Department of Transportation (DOT) agencies use to analyze the safety of road, air, rail, water and pipeline transportation systems
- Research problems that the agencies address with safety data analysis
- Impact of safety data analyses on Federal transportation policies
- DOT analysis resources and capabilities
- Needs for improvement.
The survey principally focuses on quantitative analysis of safety data, though it also covers qualitative analysis methods. The survey acquires information on analysis used to address safety problems, to identify areas for transportation safety improvement, and to develop recommendations for facilitation of data analysis for all transportation modes. This includes, for example, the development of resources for information on and providing training in the use of analysis methods, developing guidelines for use of these methods, and developing standards or "best practices" for research.
Safety Data Analysis Purpose and Analysis Objectives
In conducting the survey, the team assumed that the purpose of safety data analysis is to produce knowledge in order to save lives, prevent injuries, and protect property from hazards in transportation. To achieve this purpose, public and private highway, aviation, rail, marine, and pipeline safety organizations sponsor, conduct, and use the results of transportation safety research worldwide. Safety professionals analyze data to assess the extent, investigate the circumstances, and determine the causes of transportation-related deaths, injuries, and property damage. The results of their analyses inform the development and implementation of policies for hazard mitigation and measures to minimize the consequences of accidents or other mishaps for all modes of transportation.
Areas Considered for Possible Improvement
Successful analysis requires capabilities to produce accurate and useful information on hazards and risks in an increasingly complicated transportation environment. However:
- Resources for safety analysis may presently be insufficient, perhaps because of a decline in recognition of the value of data analysis as a tool for improving transportation safety.
- Since data analysis methods and proficiency levels are diverse, generally accepted standards for their application or the evaluation of their adequacy may not exist.
- Fora for intermodal interchange of safety research methods, analysis tools, and information on best practices for improving safety data analysis do not presently exist.
The team hypothesized that these or other deficiencies might constrain the productivity of transportation safety research, with adverse impacts on the effectiveness of safety programs. One purpose of the survey was to evaluate the extent of possible deficiencies, and recommend steps to improve safety data analysis for all transportation modes.
Method
To acquire information on transportation safety data analysis, the team surveyed organizations that conduct and sponsor safety research. Questions were developed to facilitate these discussions about data analysis purposes, programs, and the analysis used. These interviews will identify quantitative and qualitative safety analysis methods, areas of research, resources used to conduct analysis, and recommendations for actions that might be taken to improve safety data analysis for each of the transportation modes.
The following survey instrument was sent to prospective modal contacts in preparation for their interviews.
Survey Instrument
The Bureau of Transportation Statistics (BTS) requests your participation in the survey of transportation safety professionals conducted by its Safety Data Initiative, Safety Data Analysis Working Team. Participants represent public and private organizations involved in the sponsorship or conduct of safety data analysis or the use of analysis products. It is essential that recommendations in the forthcoming report on Transportation Safety Data Analysis reflect your knowledge and experience, and BTS very much appreciates your consideration and participation in the survey to ensure this occurs. To arrange for your participation, BTS will contact you to arrange a time for you to meet with Dr. Lynn Weidman, Ms. Paulette Grady, and Dr. Alex Blumenstiel. The meeting will be an opportunity to discuss critical issues in safety data analysis. The questions in the attachment to this document are intended to facilitate discussion at this meeting.
The purpose of the survey is to ensure that professionals who sponsor or conduct analysis, or use analysis products join in exploring critical issues affecting transportation safety data analysis and participate in the development of recommendations for improvement. The information that you provide by participating in this survey will be crucial in forming recommendations for solving these problems.
Safety Data Analysis Issues
The goal of safety data analysis is to produce knowledge in order to save lives, prevent injuries, and protect property from hazards in transportation. To reach this goal, public and private highway, aviation, rail, marine, and pipeline safety organizations sponsor, conduct, and use the results of transportation safety research worldwide. Safety professionals analyze data to assess the extent, investigate the circumstances and determine the causes of transportation-related deaths, injuries and property damage. The results of their analyses inform the development and implementation of policies for mitigation of accidents and measures to minimize their consequences for all modes of transportation.
Achievement of the goal requires the capability to produce accurate and useful information on hazards and risks in an increasingly complicated environment. However, insufficient resources and capabilities even now constrain safety data analysis and limit its benefits for the development of effective transportation policies and hazard controls. The future viability and contribution of data analysis to further improvements in transportation safety remain uncertain. Reasons for this uncertainty include:
- Resources for safety analysis are presently insufficient, apparently because of a decline in recognition of the value of data analysis as a tool for improving transportation safety.
- Data analysis methods and proficiency levels are diverse, with generally accepted standards for neither the application of these methods nor the evaluation of their adequacy.
- Fora for intermodal interchange of safety research methods, analysis tools, and information on best practices for improving safety data analysis do not presently exist.
Specific Safety Data Analysis Questions
The Safety Data Analysis Working Team offers the attached list of questions to facilitate discussion and the formulation of recommendations for improvement of data analysis tools and analytic expertise for transportation safety research. The questions request information on:
- Data sources, problems addressed by and uses of transportation safety data analysis products
- Methods and tools used to identify and measure the consequences of causes and circumstances of transportation-related deaths, injuries and property damage for each mode of transportation
- Expertise and resources applied and needed to support the analysis of safety data
- Best practices and lessons learned within organizations and by individuals conducting and applying the results of safety research for each of the transportation modes
- Needs for and the effectiveness of training.
The working team will use the information you provide to formulate:
- Recommendations for standards, technologies, training, evaluation and other requirements to improve safety data analysis capabilities for each mode
- Recommendations for venues and protocols for interchanges of safety data analysis best practice and lessons-learned.
- A strategy for program development and coordination of fora, training, and other options for information exchanges on data analysis efforts, statistical and other analysis tools, best practices and lessons-learned, impacts, and resource requirements.
Your assistance in this effort is crucial for success of the initiative to improve safety data analysis in all modes of transportation.
We very much appreciate your time and effort in contributing to the initiative and will call you to arrange a meeting.
Thank you.
Sincerely,
Dave Balderston
Federal Aviation Administration Office of Systems Safety
Chair, Safety Data Analysis Working Team
Name: __________________________________________________
Title: ___________________________________________________
Agency/Organization: _____________________________________
Address: ________________________________________________
Phone: __________________________________________________
Fax: ____________________________________________________
E-mail: _________________________________________________
Please summarize your own interests and efforts in conducting, sponsoring or using products of safety data analysis.
Interview Questions
| 1.1 |
What organizations use your agency’s information systems for safety research?
|
| 1.1.1 |
In house
|
| 1.1.2 |
Other agencies
|
| 1.1.3 |
Private research, academic or other non-profit organizations
|
| 1.1.4 |
Commercial enterprises
|
| 1.2 |
What are the principal problems that your agency has addressed by analyzing safety data over the last five years?
|
| 1.3 |
What are the principal sources of the data used for this analysis?
|
| 1.3.1 |
Your agency’s information systems (list)
|
| 1.3.2 |
Other DOT, government or other organizations’ information systems (identify)
|
| 1.4 |
What kinds of analysis were (are) used?
|
| 1.5 |
What standards are applied to evaluate the analysis results?
|
| 1.6 |
Has your agency established requirements or guidelines to ensure that analysis results are used to inform policy development?
|
| 1.6.1 |
What are the requirements or guidelines?
|
| 1.7 |
What recent operational, development, enforcement or other policies or requirements have been informed by the results of safety data analysis?
|
| 2 |
Methods and Tools
|
| 2.1 |
What kinds of analytical methods and tools does your agency use? Examples are:
|
| 2.1.1 |
Hazard analysis
|
| 2.1.2 |
Risk analysis and modeling
|
| 2.1.3 |
Probability statistics
|
| 2.1.4 |
Correlations and analysis of variance
|
| 2.1.5 |
Other statistical tests
|
| 2.1.6 |
Geographical information system
|
| 2.1.7 |
Video or graphical analysis
|
| 2.1.8 |
Simulation
|
| 2.2 |
Does your agency recommend, require or promote the use of particular tools specifically for safety assessment, risk analysis or other safety-related analysis applications?
|
| 2.2.1 |
If it does, what are these tools and how are they used?
|
| 2.3 |
Are analysis methods or tools developed specifically or applied for purposes that are unique to your agency?
|
| 2.3.1 |
If so, what are these methods or tools, what are their sources, and how are they used?
|
| 2.4 |
What analysis methods and tools does your agency use that can be used by other DOT modes?
|
| 2.4.1 |
For what purposes does your agency use these tools?
|
| 2.4.2 |
What other agencies use the tools? For what purposes?
|
| 2.4.3 |
For what other purposes could your agency and other agencies use the tools?
|
| 2.4.4 |
Are there studies that you have wanted but haven’t been able to find the methods to do properly?
|
| 2.4.5 |
Does your agency develop or sponsor the development of new tools for safety analysis? What tools has it developed?
|
| 2.4.6 |
Who uses them?
|
| 2.4.7 |
For what are they used?
|
| 2.4.8 |
What methods or tools is your agency currently developing or testing?
|
| 2.5 |
Does your agency have a policy for use of particular types of analytic tools to answer specific types of safety questions? For example, the FAA develops and applies blunder models to assess impacts of aircraft separation standards. It uses hazard assessment to assess the potential impacts on safety of new aeronautical technologies. Does your agency use similar tools for similar purposes? Other examples?
|
| 2.6 |
What are limitations of the analysis tools that you agency uses?
|
| 2.7 |
Are there actions that BTS can take to help improve the agency’s analysis capabilities?
|
| 3 |
Expertise and Resources
|
| 3.1 |
What proportion of your agency’s safety analysis expertise is in-house?
|
| 3.2 |
What is the composition of this expertise, in skills and qualifications?
|
| 3.3 |
How many FTEs on average has your agency devoted overall to safety-related analysis work during the past 5 years?
|
| 3.3.1 |
In house
|
| 3.3.2 |
Other
|
| 3.4 |
Does your agency have a specific organization unit with the mission to develop and apply safety analysis methods and tools?
|
| 3.4.1 |
Size of the unit (FTEs)?
|
| 3.4.2 |
Funding?
|
| 3.4.3 |
Other responsibilities?
|
| 3.5 |
What has your agency’s annual level of funding for safety data analysis been over the past five years?
|
| 3.6 |
What is its FY 2002 level of funding for this activity?
|
| 3.7 |
Does your agency need to change the number of FTEs, funding, skill mix or staff qualifications devoted to safety analysis? If so, in what ways and why have priorities changed?
|
| 3.8 |
What is your agency’s plan for safety analysis support for the next five years?
|
| 4 |
Lessons Learned and Best Practices
|
| 4.1 |
How does your agency measure and evaluate productivity in its in-house and sponsored or contracted research and analysis programs?
|
| 4.2 |
Does your agency have a policy in place to formalize best practices for safety research and analysis?
|
| 4.3 |
How does your agency:
|
| 4.3.1 |
Identify lessons learned?
|
| 4.3.2 |
Formalize best practices for productive safety research and analysis efforts?
|
| 4.4 |
If lessons learned and best practices have been identified and formalized, have they improved safety analysis? If so, how?
|
| 4.5 |
Is documentation of lessons-learned and productivity improvements available?
|
| 4.6 |
In what form?
|
| 4.7 |
Has the documentation been useful?
|
| 4.8 |
What improvements in the application of best practices and lessons-learned may be needed?
|
| 4.9 |
What resources (FTEs and budget) are applied to evaluate your agency’s safety analysis methods, tools and results?
|
| 4.10 |
What are your agency’s plans for improving safety data analysis over the next five years?
|
| 4.11 |
Training methods
|
-
Does your agency have specific goals for improving its safety analysis capabilities that can be achieved through training? If so, what are these goals?
-
What improvements are needed or planned to help your agency achieve its goals?
-
What level of funding is needed for these improvements?
-
Does your agency currently provide training in safety data analysis?
-
Formal Courses (Internal, External)
-
Training materials
-
Mentoring
-
Does your agency have a policy regarding the use of specific training methods for improving its safety analysis capability?
-
In the past 5 years, what resources has your agency devoted on average for training its staff in safety data analysis?
|
| 5.6.1 |
Courses
|
| 5.6.2 |
Training materials
|
| 5.6.3 |
Other
|
| 5.7 |
How is staff selected for this training? How is the effectiveness of the training measured? What have been the results?
|
|
|
