Research Article

Communication of Public Health Risk to Society: Strategic Proposed Guide

Alberto Frutos Pérez-Surio1* and Gema Sancho Monllor2

1Department of Preventive Medicine and Public Health, University of Zaragoza, Spain

2Expert in Documentation and Information Systems, Atlantis Project Site Manager, Zaragoza, Spain

*Corresponding author: Alberto Frutos Pérez-Surio, Department of Preventive Medicine and Public Health, University of Zaragoza, Spain, Tel: +34 659073704; E-mail: ajfrutos@unizar.es

Citation: Frutos Pérez-Surio A, Sancho Monllor G (2017) Communication of Public Health Risk to Society: Strategic Proposed Guide. J Glob Epidemiol Environ Health 2017: 46-53.

Received: 05 June, 2017; Accepted: 12 December, 2017; Published: 29 December, 2017

Abstract

The strategic aim of communication in cases of Public Health crisis is to approximate, as far as possible, the perception of risk by the society and the situation of real risk, in order to reduce the anxiety about the crisis by evidence based information, according to the seriousness of the Public Health alert. The citizen’s perception of risk is a legitimate reality that we must accept and learn to manage better. We present a case study of the Public Health Alert System (PHAS), which aims to ensure the control of Public Health alerts in the Aragon region outside normal working hours. Therefore, along with the activities carried out in the normal working hours, the alerts are attended to 24 hours a day for the entire year. The PHAS is designed to meet all the urgent situations (confirmed or suspected) that may affect the collective health of the population.

Keywords: Global Epidemiology; Guide; Public Health; Risk Analysis; Risk Communication; Society

Introduction

Over the past two decades, our society has faced a marked increase in Public Health risk situations. From the outbreak of massive poisoning by Spanish toxic oil syndrome in the early 1980s to the current possibility of bird flu, Ebola or Zika virus’s outbreaks, other crises of different natures have occurred. Controversies about risks to Public Health regularly hit the news, whether about food safety, environmental issues, medical interventions, or ‘lifestyle’ risks [1]. Thereby, the contamination of chickens and other foods by dioxins occurred in Belgium, exposure to depleted uranium in the Balkan War, or the breakthrough in the transgenic food market, hardly impacted Public Health organizations, policy and regulations [2]. The most obvious common element in these situations, once removed from the high pressure of the moment, is that the population perceives the situations as high risk, and demands an immediate response from health professionals.

On the other hand, these crises are usually followed by criticism from the population and the media. The most general claim is that information is hidden, or other interests could influence how the media receives and understands the information given. Although they are not habitual, there is some experience of crisis situations that shows how vulnerable the position of institutions can be in the face of poorly managed problems, and the costly recovery of lost confidence [3]. The current technological trend, which favours communication and immediacy in circulations information, joins the social aspiration to receive answers to the demands in the face of situations that possibly risk the health of future generations or the environment. In this context, the adequate transmission of information and communication of risk become essential in any system implemented to solve Public Health crises; the methodology used will contribute to the well-being of the population [4]. Risk communication, i.e., about environmental hazards [5], “is the attempt of science and Public Health professionals to provide information that allows an individual or an entire community to make the best possible decisions during a Public Health crisis. This communication must be made with time constraints and requires public acceptance of the imperfect nature of the choices available for action” [6].

The strategic aim of communication in cases of Public Health crises is to approximate, as far as possible, the perception of risk by the society and the situation of real risk. This approximation can help to reduce the anxiety about the crisis by evidence based information, according to the seriousness of the Public Health alert. The citizen’s perception of risk is a legitimate reality that we must accept and learn to manage better overall.

We present, as research article, a case study of the Public Health Alert System (PHAS) aims to ensure the control of Public Health alerts in Aragon Region outside normal working hours. The Autonomous Region of Aragon is located in the north-eastern Spain, and plays a key role between the regions of southern Europe. With a 136 kilometre border with France, it is the central gateway to Europe from Spain and Portugal. It consists of the provinces of Huesca, Zaragoza and Teruel. Aragon has a surface area of 47,500 m2, almost a tenth of Spain. Its strategic location makes it the natural crossroads between two of Europe’s most important axes of development: Atlantic and Mediterranean. Aragon’s total population (2016 census) is 1,308,563.

Therefore, along with the activities carried out in the normal work hours, the alerts are attended 24 hours a day, every day of the year. The PHAS is designed to meet all these urgent situations (confirmed or suspected) that may affect the collective health of the population:

  • Urgent diseases of an urgent nature;
  • Epidemic situations and outbreaks;
  • Risks related to water supplies;
  • Risks that affect food safety;
  • Risks related to chemical, biological or physical contamination that may affect the health of the population;
  • And other Public Health alerts.

The basic principles of action before a Public Health alert are:

  • Interdisciplinary team working;
  • Coordination and continuous communication between team members;
  • Communication and coordination with Agencies and Institutions involved;
  • Information to affected or exposed in the alert;
  • Confidentiality;
  • Application of the Precautionary and Sanitary Authority Principle.

Regulation is a very important topic on risk management [2,7-9]. Therefore, specific aims are:

  • To provide a rapid response to control and management of Public Health alerts, initiating appropriate measures;
  • To respond to the intervention needs, planned and non-urgent, that need to be carried out outside normal working hours, and which are not regulated at the moment;
  • To improve information on emerging risks in order to plan medium and long-term actions that prevents its occurrence;
  • To facilitate the training of technicians involved in the PHAS;
  • To provide the necessary information to citizens who may be affected by Public Health alerts.

In order to attend to the alerts, the PHAS is structured mainly by the weekly guard team and has a series of working means, performing an activity based on strategic and operational protocols of action. The PHAS will be evaluated periodically to measure its adequacy to the objective of attending Public Health alerts. This document describes the organization of this structure, activity and evaluation related to Public Health risk communication.

The aim of the paper is to describe the methodology of PHAS, audience, and characteristics, of the key messages for an adequate transmission of external information to the society in a situation of crisis or Public Health risk.

The internal communication systems are not collected, neither how nor whom, and it is necessary to notify an alert situation as we understand that it is already included in a general way in the protocol of basic principles and in those of each particular risk situation. The audience of the information are: healthcare professionals affected / exposed citizens, professional organizations, trade unions, public opinion, and the society [10].

Structure of the Guard Team

Composition: Each week, the guard will consist of: 1 Regional coordinator and 2 or 3 individuals responsible for each province. The composition of the guard team (Figure 1) will take into account the following: The coordinators will be the Provincial Deputy Directorate of Public Health, heads of service and the advisor of the General Directorate of Public Health, who will take turns; 33% of the guards will correspond to each area of intervention (surveillance, environmental and food), according to the approximate observed distribution of alerts in the first two years of the System’s operation; Each week on shifts, there will be at least two people from each intervention area on the guard team, not counting the coordinator (2 from surveillance department, 2 from environmental health department, 2 from food safety department); The other elements of organization will be at the discretion of each Provincial Deputy Directorate of Public Health.

Composition of the guard team

Figure 1: Composition of the guard team.

H: Huesca, PG: Post-graduate, T: Teruel, Z: Zaragoza

 

Participant’s profile: Professionals interested in participating in the PHAS should have the following requirements:

  • Technician of group A or B, with health training (Medical doctors, Veterinarians, Pharmacists and Technicians of Health Administration);
  • Assigned to the units of Public Health;
  • With at least two years of experience in the units of Public Health;
  • With a minimum of one hundred hours of training in Public Health disciplines related to alert management (both in previous courses and in specific PHAS courses);
  • With flexible thinking, a desire to learn and the ability to be open-minded and adapt to new changing situations;
  • With capacity for organization, coordination, management and decision-making.

Team Functions: The coordinator of Aragon region will have the following responsibilities:

  • Risk assessment: magnitude and severity;
  • Making decisions to intervene;
  • Coordination of actions;
  • Deciding the mobilization of human resources, both the PHAS and others;
  • Communication with those responsible for the implementation of intervention measures;
  • Data reception, control, pre-treatment and aggregation of data;
  • Technical communication with other entities and Agencies (emergency services, health care services, other departments, other communities, Ministry of Health, Communication Office);
  • Preparation of risk communication for the society with the communication cabinet;
  • Management of the information system of Public Health alerts: updating the databases of management and introduction of the information of the alerts attended;
  • Distributing information to the Public Health units on the next working day, and preparation of the information for next Monday, when the guard team changes;
  • Preparation of the weekly report, including the alerts served and justifying, where appropriate, the time of physical presence guarded by professionals;
  • Preparation of the annual evaluation with other coordinators;
  • Organization of non-urgent work outside the normal working hours to be performed that week, always giving priority to alerts.

The professional and technician will have the following functions:

  • Additional risk assessment;
  • Measures of action and control, including mobilization of external resources (assistance, environmental protection or others), in collaboration with the coordinator;
  • Effective resolution of the alert, which will be the main responsibility;
  • Coordinated team actions with other guard officers;
  • Communication with the coordinator during and after the management of alert;
  • Information system: data entry, report writing.

Communication channels: They will be articulated through the Department’s of Communication Office. The communication channels are very varied and will be used depending on the needs of the situation. The members of the PHAS will communicate the information the alert is finally decided and specifically ordered. The channels used can be: News; Press conferences; Interviews; Institutional press releases; Informational publications; Advertising; Web page; Television and radio; Informative meetings.

Although it is always desirable to have a single spokesperson, it may be necessary in some cases to gain the support of an expert with sufficient skills for transmitting the information. It will generally be done with interviews, reports, conferences, etc.

Keys to transmit the message: The spokesperson must always be the same communicator during crises, it will be decided if a politician or a technician is to transmit the information;

The information will be:

  • clear and sincere and we will say what we know, but also the unknowns that we face;
  • It will include all points of view (affected groups, industry, administration, consumers);
  • The message needs to be adapted to the different interlocutors;
  • It will be made clear that the purpose of the administration is to promote and protect the society’s health;
  • Databases and other proven scientific and technical sources will be consulted to obtain evidence-based information;
  • It will facilitate the meeting with affected groups whenever it is possible.

Message features: Key messages are based for applying the STARCC principle (Simple, Timely, Accurate, Relevant, Credible, and Consistent) in your communication [11]. The public message in a crisis must be:

  • Simple (frightened people don’t want to hear big words);
  • Timely (frightened people want information as soon as possible);
  • Accurate (frightened people won’t understand nuances, so present it directly);
  • Relevant (answer their questions and give action steps);
  • Credible (empathy and openness are key to credibility);
  • Consistent (the slightest change in the message is upsetting and dissected by all).

There are two levels of communication. First level for alert with impact on public opinion: When a situation of habitual alert is not considered as a crisis, the coordinator understands that it can still have impact in the public opinion, or be perceived risk. In these cases, the coordinator will report the situation to the Press Office of the Department with as much data as possible.

The valuation will be related to:

  • The area in which the alert situation occurs (school, hospital, work, etc.);
  • The number of people affected;
  • The features of the target entity itself (meningitis, legionelosis, food alert, etc.).

For the second level, where there is a crisis situation, or certainty of real risk: The coordinator will immediately inform the General Director and, thereafter, the Crisis Office.

The Composition of the Crisis Cabinet will constitute an interdisciplinary team that will include at least:

  • A representative of health authority (director), to transmit and make decisions;
  • Those responsible for the units that needs to intervene resolving the crisis;
  • An expert in the search of scientific documentation;
  • A representative of the Communication Cabinet of the Department of Health;
  • Administrative staff.

In this team, it will be decided, in matters related to the transmission of external information, which will be the spokesperson and what the message is to be transmitted, and its characteristics (press release, press conference, etc.) The opportunity to call together information meetings with affected groups, groups at risk, representatives of consumer associations, etc. In some cases, it will be necessary to expand the components of the Crisis Cabinet with external experts, both scientific and technical, as well as legal experts.

Likewise, based on the characteristics of the crisis, meetings will be organized with representatives of other entities involved.

Four phases can be distinguished in the elaboration of the message:

  1. Compilation of all available information concerning the crisis: Background, current situation, possible evolution;
  2. Analysis of the situation;
  3. Organization of response;
  4. Elaboration of information.

The media and crisis situations: The public depends heavily on the media to obtain information and in crisis situations significantly increases the use of them.

The relationships established between Public Health professionals and the media are articulated through the Department’s Communication Office. Good relations with the media are crucial in a crisis situation. There are two types of media relations, the reactive one (answering the questions of the media, on demand) and the preventive one, which consists of making an effort to organize and disseminate the information to be transmitted before we are asked.

There are several ways to get information from the media, depending on the time and circumstances the transmission channels can be statements or news) communications, or Press conference. Statements communications are useful when it is desired to disclose and record certain current information, in a clear and concise manner without immediate urgency. It is distributed to all media. Press conference: In crisis situations it is very useful when it is necessary to transmit new data. Conferences should only be organized when they are able to offer relevant and consolidated information. The institution summons the written and audio-visual media to facilitate the information and at the same time, process the public opinion. The press conference has an interactive character because it allows immediate feedback with the journalists. This is a part of a dialogue with media professionals. It is important to prepare support material for later delivery to journalists, especially when it comes to numerical data, which should be reflected in clear tables, and even with graphics to facilitate their understanding [12]. In both cases, the initiative starts from the institution itself that wants to inform the media. Other types of risk communication include interviews and reports. Interviews: are a type of information journalism that reflects the responses of the interviewee, whose opinions, due to their social relevance, the position they occupy, or their involvement in current news events, are of general interest. The interviewee can be a politician or a technician / scientific expert, depending on the point of view that one wants to deal with. Reports are articles in which a journalist broadly reports on a current topic, provides data, analyses causes and interviews people. It is, therefore, an expository text that generally adopts a descriptive form or a narrative form.

In these cases, the initiative is part of the media and lacking the urgency of the press conference occurs because the transmitted information is not something new and timely. Other communication tools are Institutional website, and informal publications. Institutional website: The media and the general public can also obtain information about a crisis on the Public Health website. It will be essential to keep updated information for the duration of the crisis and when it is resolved, a final report will be posted elaborating all phases of the process, involved, decision making, research methodology, etc. Informational publications: In certain situations, predictable in advance, there may be social alarm, as in the case of the emergence of extreme climatology related to heat waves. On these occasions, we can use this type of information media, in a programmed way and without the immediacy demanded by crisis situations. For more information, we may include full-text scientific and technical articles, links to pages of institutions with authority on the subject, as well as any further information that may be relevant. A very comprehensive revision about patient decision aids and risk decision making was published in manuscripts published in 2013 in BMC Medical Informatics and Decision Making [13-24].

Analysis

The PHAS must be permanently evaluated and, depending on the results, adapted to the changing needs. This includes both the structure of the system (human resources, training and technical issues), as well as the process (the activity carried out), and finally the results (response time and degree of control of alerts). In addition, an assessment will be made by the participants. At least once a year, an evaluation of the suitability and results of the PHAS will be carried out with a written report that will be incorporated into the general evaluation of the Public Health units. This report will be presented at an annual evaluation day which will be open to participants.

Parameters Included

The analysis of the PHAS is based on obtaining indicators of evaluation of the structure, process and results from the related sources of data. Structure: administrative databases with human and material or economic resources of the PHAS. Process (or activity): This data is obtained mainly from the alerts database, which includes a series of variables of all alerts served. This database is completed by coordinators, and includes reports from the responsible professionals. Results: One part of the indicator is obtained from the alerts database, and another from a specific survey of three evaluators about importance, quality of intervention and degree of control achieved in the alerts [25].

In terms of results according to criteria of importance, quality of intervention and degree of control, the alerts might be evaluated by three evaluators. Importance (for Public Health): degree in which the alert is a Public Health problem. It is categorised in 5 different levels.

  1. Very little importance: administrative problem or with very little impact, or does not require immediate action.
  2. Little importance: little importance because of its low impact or requires a small action.
  3. Important: repercussion for Public Health and need for action.
  4. Quite important: greater impact for Public Health, need for immediate and / or large measures.
  5. Very important: great impact for Public Health, very urgent measures and / or huge impact.

Quality of the intervention: grade in which activities performed conform to the best procedures that control it. This is also classified in 5 levels.

  1. Very scarce: hardly any measures have been taken and / or have not been adequate.
  2. Scarce: few measures have been taken and / or are ineffective.
  3. Median: measures have been taken, reasonably adjusted.
  4. Good: measures have been taken, of good efficacy.
  5. Very good: all recommended measures have been taken with maximum effectiveness.

Grade of control achieved (by the PHAS): way in which the alert has been satisfactorily resolved by the system, regardless of whether it is followed up by Public Health units. A Likert scale can be used to classify from 1 to 5 levels.

  1. Very low: the problem has not been controlled.
  2. Scarce: very little control.
  3. Medium: it has been controlled to some degree.
  4. Appreciable: it has been controlled for most part.
  5. Complete: The problem has disappeared.

According with the participant evaluation survey, there will be the following variables: Work area: Food hygiene, Environmental health, epidemiological surveillance or programs. Dependence: peripheral (Veterinary Zones, Basic Health Areas, etc.), Subdirectories of health, General direction. Occupation: veterinarian, pharmacist, technician, medicine doctor.

Discussion

Global evaluation of PHAS: Each participant would be asked to rate the following aspects of the PHAS on a Likert scale from 1 to 5 (1 being the worst assessment; 5 being better). Protocols: the adequacy of the protocols and applicability for the resolution of alerts. Materials / equipment: the adequacy of technical support material and its location. Transportation: availability and adequacy of means of displacement. Hours: suitable to cover any alert that occurs. Effective: The number of people is adequate to give correct coverage.

Communications: the media inside the guard team and with the outside are adequate. Work load: The work load of the PHAS is 1 to 5 (1: low - 5 excessive). Organization of turn and guard teams. Effectiveness: The PHAS has an adequate capacity to solve Public Health alerts. In addition, it will be possible to add a brief written commentary to each of the previous points as well as in a final section of other aspects.

Indicators include structure, process and results aspects. These indicators should include:

  • a total number of members, in a province or in an area of work;
  • total rotation of responsible, in a province or in an area of work;
  • total cost, workers (located guards and physical presence), and other costs.
  • Process / activity indicators include:
  • Number of alerts; Incidence of alerts per 100,000 people per year;
  • Distribution of alerts (type, origin, communication, severity, intervention...);
  • Number of people affected by alert and percentage of hospitalized cases;
  • Percentage of alerts investigated and with final report;
  • Percentage of alerts using protocol;
  • Percentage of alerts resolved by the on-call/guard team;
  • Regular off-hours work by the on-call/guard team.

Results indicators: This is the most difficult aspect to measure in the PHAS, as discussed for other indicators (Pérez, 2014). An easy but limited indicator to measure, is the delay of action, which can be obtained from the date and time of call and intervention data respectively. Regarding the results indicators according to the criteria of importance of the alert, quality of the intervention and grade of control reached, the total scoring means and by type of alert, province, action, scope, communication, source, physical presence and resolution.

Evaluation survey indicators include:

  • Response rate to the survey by area, dependency, profession and province;
  • Scoring averages for each question;
  • Comparison of scoring means of each question by scope, dependency, profession and province;
  • Evaluation of comments written in free text.
  • For the qualitative and quantitative assessment of risk, table 1 shows several indicators.

Event

Capacity morbidity

Affected location

Nº affected

Appearance of symptoms

Age affected

Serious

Lethality

National/

International

>100

Acute

Old people

Medium

Serious illness

Autonomous community / province

16 - 99

 

Children

Low

Benign

Municipal / regional

<15

Chronic

Adults

*Indicate the risk in each column and with all of them make an overall assessment of the risk of the alert.

Table 1: Qualitative and quantitative assessment of risk.

 

An essential component of the Rapid Risk Assessment process is risk communication: The interactive exchange of information and opinions throughout the risk analysis process concerning hazards and risks, risk-related factors and risk perceptions, as well as risk assessors, risk managers, consumers, industry, and the academic community and other interested parties. These include the explanation of risk assessment findings and the basis of risk management decisions. It is essential to make people know the harms they are exposed to. This communication, in order to be effective, must be grounded on a science-based approach. It must be a continued and interactive activity carried out from the beginning of the assessment. It is a process in which researchers, interested parties, as well as those prospectively interested during the assessment, analyze all information gathered during the analysis and propose measures adequate to the possible risks and harms. Subsequently, they are passed along to competent authorities and interested parties in order for them to make informed decisions. In order to guarantee the success of communication some requisites must be met: transparency of the process, consensus reaching, adequate information flow so as for all the relevant available information to be integrated in the assessment [26]. Risk communication attempts to put each particular risk into context, evaluate similarities among risks, include suggestions on the minimization of risks and encourage the dialogue between the sender and recipient of the message.

There are two important components for risk communication: operational communication and communication with the public [27]. Operational communication: The structured communication that organizations use to meet their work goals and strategic objectives, including internal coordination and coordination with people outside the organization. Communication with the public: Communication to provide key findings from risk assessments at regular intervals.

It is a necessary two-way communication between the risk management team and the stakeholders previously identified (by the team, at the start of the risk assessment cycle). There must be a coordination of response: Risk and crisis communication, to be adapted to Member State needs and circumstances, aimed at providing consistent and coordinated information in the Union to the public and to health care professionals. But also with protection of personal data: The personal data will be communicated to the competent national authorities for the location of contacts.

There are some limitations, and future researchers should be building on other attempts; WHO has made numerous attempts to “grade” the threat to convey the level of concern. WHO currently seems more focused on phases of a pandemic rather than level of threat. It is difficult, even for influenza to reach consensus on conveying the threat of a complex epidemic (varying pathogenicity, varying age groups most severely affected, etc.) through a simple alert system. To do that for diseases from a new tickborne encephalitis virus to antimicrobial resistant gonorrhea would be even more difficult. The idea of weekly rotating teams to grade epidemics also may be difficult; without strict criteria, the teams are likely to grade epidemics differently. There is enough trouble to find agreement on whether there is a public health emergency for a disease at WHO with experts. With rotating 24/7 coverage in the US, the covering public health official may be called on to deal with an outbreak of water-borne disease when they are an obesity expert and have little technical expertise. Would HIV have been considered high alert when it first started and cases of PCP and KS were dribbling in? Part of the problem (and fear) has to do with the unknown (long-term effects, transmission modes, pathogenicity, and virulence). There is controversy as to when to open an emergency operations center and it is not always based on the alert level. CDC opened its EOC for polio. Is that high alert?

Acknowledgment

We would like to thank the General Directorate for Public Health for providing the key messages reported here. We would also like to acknowledge Taylor Thornburg (Admissions Coordinator at the Atlantis Project) for her edition of the manuscript.

References

  1. Bennett P, Calman K, Curtis S, Fischbacher-Smith D (2009) Risk communication and Public Health. 2ndedn, Oxford University Press, New York, USA. Pg no: 264.
  2. Demeritt D, Rothstein H, Beaussier AL, Howard M (2015) Mobilizing risk: Explaining policy transfer in food and occupational safety regulation in the UK. Environment and Planning A 47: 373-391.
  3. Dodds A, Kodate N (2011) Accountability, organisational learning and risks to patient safety in England: Conflict or compromise? Health, Risk & Society 13: 327-346.
  4. Bardsley M (2016) Learning how to make routinely available data useful in guiding regulatory oversight of hospital care. BMJ Quality & Safety 26: 120-130.
  5. Baker F (1990) Risk communication about environmental hazards. J Public Health Policy 11: 341-359.
  6. Centers for Disease Control and Prevention (2011) Public Health Emergency Response Guide for State, Local, and Tribal Public Health Directors - Version 2.0. US Department of Health and Human Services, Centers for Disease Control and Prevention, Atlanta, USA.
  7. Beaussier AL, Demeritt D, Griffiths A, Rothstein H (2016) Accounting for failure: risk-based regulation and the problems of ensuring healthcare quality in the NHS. Health Risk Soc 18: 205-224.
  8. Black J (2010) Risk-based regulation: Choices, practices and lessons learnt. In: OECD. Risk and regulatory policy: Improving the governance of risk. OECD Publishing, Paris, France. Pg no: 185-224.
  9. Phipps DL, Noyce PR, Walshe K, Parker D, Ashcroft DM (2011) Risk-based regulation of healthcare professionals: What are the implications for pharmacists? Health, Risk & Society 13: 277-292.
  10. Kruk ME (2012) Globalisation and global health governance: Implications for Public Health. Glob Public Health 7: 54-62.
  11. Reynols B (2004) Crisis and Emergency Risk Communication by Leaders for Leaders. Centres for Disease Control and Prevention, Atlanta, GA, USA.
  12. Remington PL (1998) Communicating epidemiologic information. In: Brownson RC, Petitti DB (eds.). Applied Epidemiology: Theory to Practice. Oxford University Press, New York, USA. Pg no: 323-348.
  13. Abhyankar P, Volk RJ, Blumenthal-Barby J, Bravo P, Buchholz A, et al. (2013) Balancing the presentation of information and options in patient decision aids: An updated review. BMC Med Inform Decis Mak 13: 6.
  14. Barry MJ, Chan E, Moulton B, Sah S, Simmons MB, et al. (2013) Disclosing conflicts of interest in patient decision aids. BMC Medical Informatics and Decision Making 13: 3.
  15. Bekker HL, Winterbottom AE, Butow P, Dillard AJ, Feldman-Stewart D, et al. (2013) Do personal stories make patient decision aids more effective? A critical review of theory and evidence. BMC Medical Informatics and Decision Making 13: 9.
  16. Coulter A, Stilwell D, Kryworuchko J, Mullen PD, Ng CJ, et al. (2013) A systematic development process for patient decision aids. BMC Med Inform Decis Mak 13: 2.
  17. Fagerlin A, Pignone M, Abhyankar P, Col N, Feldman-Stewart D, et al. (2013) Clarifying values: an updated review. BMC Med Inform Decis Mak 13: 8.
  18. Feldman-Stewart D, O’Brien MA, Clayman ML, Davison BJ, Jimbo M, et al. (2013) Providing information about options in patient decision aids. BMC Medical Informatics and Decision Making 13: 1-9.
  19. Hoffman AS, Volk RJ, Saarimaki A, Stirling C, Li LC, et al. (2013) Delivering patient decision aids on the Internet: definitions, theories, current evidence, and emerging research areas. BMC Med Inform Decis Mak 13: 13.
  20. Montori VM, LeBlanc A, Buchholz A, Stilwell DL, Tsapas A (2013) Basing information on comprehensive, critically appraised, and up-to-date syntheses of the scientific evidence: a quality dimension of the International Patient Decision Aid Standards. BMC Med Inform Decis Mak 13: 5.
  21. Sepucha KR, Borkhoff CM, Lally J, Levin CA, Matlock DD, et al. (2013) Establishing the effectiveness of patient decision aids: key constructs and measurement instruments. BMC Medical Informatics and Decision Making 13: 1-11.
  22. Stacey D, Kryworuchko J, Belkora J, Davison BJ, Durand MA, et al. (2013) Coaching and guidance with patient decision aids: A review of theoretical and empirical evidence. BMC Med Inform Decis Mak 13: 11.
  23. Trevena LJ, Zikmund-Fisher BJ, Edwards A, Gaissmaier W, Galesic M, et al. (2013) Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers. BMC Medical Informatics and Decision Making 13: 1-15.
  24. Volk RJ, Llewellyn-Thomas H, Stacey D, Elwyn G (2013) Ten years of the International Patient Decision Aid Standards Collaboration: evolution of the core dimensions for assessing the quality of patient decision aids. BMC Medical Informatics and Decision Making 13: 1-7.
  25. Donabedian A (1988) The quality of care. How can it be assessed? JAMA 260: 1743-1748.
  26. Demeritt D, Nobert S, Cloke HL, Pappenberger F (2013) The European Flood Alert System and the communication, perception, and use of ensemble predictions for operational flood risk management. Hydrological Processes 27: 147-157.
  27. OECD (2010) Risk and regulatory policy: Improving the governance of risk. OECD Publishing, Paris, France.