| Abstract|| |
Background and Aim: The role of the cardiac anaesthesiologists extends beyond mere patient wellbeing to diagnostic input and active participation in decision making during cardiac surgery. The quality of service provision should therefore be judged not only by patient satisfaction but also by the satisfaction of cardiac surgeons. Unfortunately, quantification of cardiac surgeon satisfaction remains a challenge due to the absence of a reliable and validated tool. We therefore attempted to develop a robust, validated, pilot psychometric questionnaire, to measure satisfaction of cardiac surgeons' to cardiac anesthesia services.
Methods: The questionnaire was developed with the help of senior cardiac anesthesiologist, cardiac surgeon and statistician with database search in PubMed and the Cochrane Library. The questionnaire was tested for content validity, comprehensibility, and identification of new items. This generated the second version of the questionnaire with nine socio-demographic and professional questions, 46 Likert type questions, an abridged Marlowe Crowne Social Desirability scale and one open ended question. This questionnaire was e-mailed to 100 cardiac surgeons requesting them to participate via a web-based survey application.
Results: Content validity of the responses was tested by Aiken's content validity coefficient (V). Internal consistency was tested with Chronbach's alpha. Fifty-two cardiac surgeons participated in the survey. Twelve Likert type questions were deleted due to low V values. Excellent Chronbach's alpha (0.94) was obtained in the remaining 34 items.
Conclusions: We have developed a questionnaire that includes 34 variables and allows quantifying surgeon satisfaction in a reliable fashion and is validated for the purpose.
Keywords: Cardiac anesthesia, cardiac surgery, quality assessment, question survey
|How to cite this article:|
Guhabiswas R, Chowdhury SR, Narayan P. Validated tool for quality assessment of anesthesia services by cardiac surgeons. Ann Card Anaesth 2021;24:190-6
|How to cite this URL:|
Guhabiswas R, Chowdhury SR, Narayan P. Validated tool for quality assessment of anesthesia services by cardiac surgeons. Ann Card Anaesth [serial online] 2021 [cited 2022 Jan 24];24:190-6. Available from: https://www.annals.in/text.asp?2021/24/2/190/314120
| Introduction|| |
In recent times, numerous studies have been conducted to measure the quality of anesthesia services. Patient satisfaction has been identified as one of the key quality indicators of anesthesia provision.,, However, there are many aspects of anesthesia services provision and patient satisfaction is merely one of them. The surgeon is a direct and important recipient of anesthesia services and despite that, surgeon satisfaction has rarely been incorporated as a tool to assess the quality of anesthesia services.,
It has been shown that quality of patient care depends on effective teamwork between the surgeon and the anesthetist. This is especially true in the cardiac surgical procedures where apart from providing anesthesia, the cardiac anesthesiologist co-ordinates between the surgeons, perfusionists and allied teams and provides valuable inputs in surgical decision-making. It is therefore imperative to study satisfaction from the surgeons viewpoint, to have a more accurate assessment of the quality of anesthesia services provided. However, very limited number of studies have attempted to measure surgeon satisfaction. When attempted, the measurement of satisfaction of surgeons with the anesthetic services is further hampered by the lack of a reliable and validated tool. Aim of this study was to therefore develop a simple, easily reproducible pilot questionnaire tool that is reliable and validated using stringent and standardized psychometric techniques, to measure satisfaction of cardiac surgeons to cardiac anesthesia services with the idea to utilize this as a quality indicator in future.
| Methods|| |
The study was approved by the institutional ethics committee of our hospital. (Ethics committee approval was obtained ON 27th April 2015. Reference - RTIIC/EC/AP/2015)
The principle phases of development of the questionnaire are shown in [Figure 1]. The survey items of first draft of the questionnaire were identified from review of existing literature and expert opinion of one senior cardiac surgeon, a senior cardiac anesthesiologist and a statistician experienced in analysis of psychometric data. The literature search was conducted using PubMed and the Cochrane Library. We restricted the search to English language publications. The keywords used for literature search were: Cardiac Surgical Procedures; Anaesthesia; Personal Satisfaction; Psychometrics; Questionnaires and Social Desirability. Publications not adopting stringent psychometric methodology were excluded. Backward snowballing was performed to identify additional relevant literature from the reference list of the initially identified articles. Indicators of quality and safety in anesthesia care from clinical indicator programs of government and professional bodies, including attributes of quality of care as identified by the National Library of Healthcare indicators, were incorporated as survey items in the questionnaire.
The format of the questionnaire was inspired by the previously published Surgeon Satisfaction with Anaesthesia Services (SSAS) scale.
One senior cardiac anesthesiologist and a senior cardiac surgeon were interviewed during generation of the first draft of the questionnaire. The cardiac anesthesiologist ensured that elements of perioperative care, which are linked to anesthesia quality, were not missed out. This first version was pretested on a separate group of five cardiac surgeons to evaluate read ability, comprehensibility, and qualitatively measure the face validity of the questionnaire.
Following the expert consultation, none of the items were deleted due to repetitiveness or irrelevance. However, they suggested inclusion of additional items specific to practice of cardiac anesthesia, specifically items on provision and quality of perioperative transoesophageal echocardiography services.
This led to the generation of the second version of the questionnaire which has nine questions related to socio-demographic and professional variables, 46 Likert items, one Likert item on satisfaction to overall anesthesia services (global satisfaction score), Reynold Short Form C of the Marlowe Crowne Social Desirability (SD) scale and one open ended question (What other things can improve the quality of services provided by cardiac anesthesiologists to the department of cardiac surgery?). [Appendix 1].
The open-ended question was included to allow the cardiac surgeons to freely express their expectations on additional areas of quality improvement which might not have been identified through a pre-determined and enforced set of questions. Eleven out of the 46 questions were reversed to prevent bias of acquiescence. The Reynold's short form C was used to rule out the bias of social desirability.
Conceptual themes that emerged during item generation were identified as dimensions of anesthesia care in the second draft of questionnaire. These dimensions covered technical knowledge and skill, interpersonal relationship, communication, and attitude to work, availability of anesthesiologists and continuity of care.
This final version was electronically mailed to a different cohort of 100 cardiac surgeons. These surgeons were selected as the first hundred names from a list of members of their professional society. (Convenience sample).
They were explained the importance of evaluating satisfaction as an important quality initiative in anesthesia services and the dearth of research in this area. Surgeons were requested to respond to the questionnaire via an web-based application (SurveyMonkey, California, USA). The questionnaire was uploaded to the online portal and a link was generated to the anonymous survey for attachment to the request e-mail. A first and a second reminder were sent by e-mail between August and September of 2015 to encourage higher response rate.
Statistical analyses were performed with SPSS 20 (Armonk, NY; IBM Corp). Prior to scoring, responses for “negative” statements were reversed. Reliability was checked with Chronbachs alpha. Chronbach's alpha value of 0.7 is an accepted cut-off for high internal consistency. Content validity of the pilot questionnaire was quantitatively evaluated with Aiken's method to calculate V value for each item., Cut-off for low V scores was taken at 0.5. For Social Desirability, mean and standard deviation was calculated after calculating total SD score for each respondent. One and a half times calculated SD was added to the calculated mean to get the cut-off value for high social desirability. The survey responses with high social desirability scores were rejected in our study. Relationship between socio-demographic and professional variables and surgeons' global satisfaction score was assessed with Fischer's exact test. A P value of <0.05 was considered statistically significant.
| Results|| |
A total of 52% of cardiac surgeons responded to the questionnaire. Eleven of these 52 responses were rejected as incomplete. In the remaining 41 completed questionnaires, two were rejected due to high social desirability scores of greater than 11.67. Demographic and professional variables of cardiac surgeons are detailed in [Table 1]. Eleven questions were reversed to prevent bias of acquiescence. Among the 46 Likert items, 12 items returned low V values and were excluded [Appendix 1]. V values of the remaining items ranged between 0.52 and 0.79 with a mean V value of 0.65.
These V values indicated that the second version of the pilot questionnaire, with 34 variables, was a valid instrument for measuring satisfaction of cardiac surgeons with anesthetic care.
An excellent Chronbach's alpha value of 0.94 was obtained from our study which denoted high internal consistency and proved that all the items in the questionnaire together created a single dimension which could be considered to be a quality anesthesia service. Among all the socio-demographic and professional variables none correlated with the global satisfaction score.
Responses to open ended question are summarized in [Table 2]. Several respondents had more than one suggestion and each suggestion was counted separately.
| Discussion|| |
Surgeon satisfaction is an underutilized but extremely important marker of quality of anesthesia service provision. One of the important issues in assessing surgeon satisfaction was the lack of a reliable, validated tool to assess cardiac surgeons' satisfaction. In this study, we have developed a reliable and validated pilot questionnaire, with 34 variables, designed to measure satisfaction of cardiac surgeons to cardiac anesthesia services.
Satisfaction scores are important to improve patient outcomes. Many different scores have been used to assess, correlate, and improve outcomes in patients. Human resources, staff burnout, inter-personal relationships, team work and communication in the operating room have all been shown to be related to patient outcomes.,,,,,,, Surgeon satisfaction scores are an amalgamation of all these different factors and have been shown to be an important measure of anesthetic outcome. Valid and reliable survey data on satisfaction can help to identify important deficiencies in standard of care and consequently leads to quality improvement and it has been recommended that the satisfaction of all recipients of anesthesia services, including surgeons, health administrators and patients should be measured and we should not concentrate merely on services to patients.
Designing of a reliable and validated questionnaire to assess satisfaction among cardiac surgeons with regards to cardiac anesthesia services is a challenging prospect. This is mainly due to the inherent complexity and incompletely understood psychological basis involved in defining satisfaction. Also, for this questionnaire to be valid and reliable a stringent psychometric methodology is necessary. Both face validity and content validity which are important for psychometric was carried out on the questionnaire. Chronbach's alpha was used in this study after ensuring content validity. Adequate measures were taken to negate the effect of social desirability and bias of acquiescence. Social desirability is another inherent impediment to a proper objective assessment of questionnaire surveys. It has been observed that social desirability response when analyzed in studies influenced almost half of them. The context of social desirability is important in the survey of cardiac surgeons as they may have a tendency to respond with favorable answers because of the inter-dependent nature of their professional relationship with cardiac anesthesiologists. A short version of the Marlowe and Crowne social desirability scale (Reynold's short form C) was used in our study to identify cardiac surgeons with high social desirability responses. The survey responses with high social desirability scores were rejected in our study.
Acquiescence bias is a category of response bias, which denotes the tendency of a survey responder to agree to a statement irrespective of the content of the statement particularly when in doubt.
Some of the other issues with a questionnaire are response rates, and the completeness of response and the adequate length of the questionnaire. The response rate to the survey in our study was 52% which is comparable to a similar study reporting a response rate of 32.6%.
Measurement of quality in anesthesia services is important. Apart from being a quality improvement tool, regulatory requirements, and ability to objectively demonstrate its practice standards to external assessors makes measuring satisfaction an important requirement. It is however not a tool to financially incentivize or dis-incentivize surgeons or anesthesiologists. While the tool we have created appears satisfactory for the assessment of surgical satisfaction to the cardiac anesthesia services it is important to regularly revise and update the questionnaire. The process of scale development is never final and the perception of satisfaction varies due to advances in technology; clinical outcomes and social values. Therefore, in the next phase of the study we plan to perform an exploratory factor analysis to determine the underlying factor structure of questions and analyze the construct validity of our questionnaire.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| Appendix 1: Final version of the Pilot Questionnaire|| |
Demographic & Professional Questions:
1. What is your Age in Years?
<40 40-50 51-60>60
2. What is your Gender?
3. What is your Marital status?
Married Single Widowed/Widower Divorced/Separated
4. Kindly mention your Professional Qualifications:
5. Years of experience in Cardiac surgical practice:
<10 10-20 21-30>30
6. Type of practice:
Private sector Government sector
7. No. of Consultant Cardiac Anesthesiologists in your cardiac surgical unit:
8. No. of Consultant Cardiac Anesthesiologists working with you in a week:
9. Number of Operations performed in a Week:
<5 5-10 > 10
The Cardiac Anesthesiologists in your opinion:
(Kindly rank in a scale of 1 -4 where is 1 is Disagree strongly, 2 is Disagree, 3 is Agree & 4 is Agree strongly)
10. Are confident & reliable: 1 2 3 4
11. Are relaxed and reassuring in nature: 1 2 3 4
12. Remain respectful towards your patients: 1 2 3 4
13. Are knowledgeable and professional: 1 2 3 4
14. Are agreeable to criticism and constructive feedback: 1 2 3 4
15. Perform Pr-Anesthesia Assessment satisfactorily: 1 2 3 4
16. Advice appropriate premedication for your patients: 1 2 3 4
17. Appropriately manage patient's preoperative medications in the perioperative period: 1 2 3 4
18. Provide adequate information regarding anesthesia to your patients: 1 2 3 4
19. Show empathy towards your patients: 1 2 3 4
20. Are punctual: 1 2 3 4
21. Quickly position patients and induce anesthesia : 1 2 3 4
22. Maintain hemodynamic stability during surgery: 1 2 3 4
23. Communicate effectively with you during surgery: 1 2 3 4
24. Remain present in operation theaterthroughout surgery: 1 2 3 4
25. Remain pro-active to reduce incidence of intra-operative Complications: 1 2 3 4
26. Perform appropriate airway management: 1 2 3 4
27. Remain alert to prevent anesthetic medication error: 1 2 3 4
28. Administer prophylactic antimicrobials in time: 1 2 3 4
29. Adopt standard procedures to ensure sterility: 1 2 3 4
30. Appropriately manage invasive lines: 1 2 3 4
31. Actively and appropriately control patient temperature in the intra-operative Period : 1 2 3 4
32. Achieve satisfactory peri-operative glucose control: 1 2 3 4
33. Advice appropriate blood transfusion: 1 2 3 4
34. Make you satisfiedwith their decision making during Trans-Esophageal Echocardiography (TEE) assessment: 1 2 3 4
35. Provide TEE assessment which agree with surgical findings most of the time:
1 2 3 4
36. Rarely make you call the cardiologist due to unsatisfactory TEE assessment:
1 2 3 4
37. Give due consideration to your professional opinion: 1 2 3 4
38. Manage to stay calm during emergencies: 1 2 3 4
39. Act productively during emergencies: 1 2 3 4
40. Are willing to adjust their availability according to the your or your patients' clinical requirements: 1 2 3 4
41. Manage mechanical ventilation to your satisfaction in the post-operative period:
1 2 3 4
42. Effectively manage post-operative pain in your patients: 1 2 3 4
43. Are not responsible for unplanned re-admission of patients to ICU within 24 hours of discharge: 1 2 3 4
44. Make you feel satisfied with their overall services: 1 2 3 4
Kindly comment if the following statements, when applied to you, are True (T) or False (F):
1. It is sometimes hard for me to go on with my work if I am not encouraged: T F
2. I sometimes feel resentful when things don't get done Exactly the way I want them to: T F
3. On a few occasions, I have given up doing something because I thought it was beyond mycapability: T F
4. There have been times when I felt like rebelling against people in Authority eventhough I knew they were right: T F
5. No matter who I'm talking to, I'm always a Good Listener: T F
6. There have been occasions when I took advantage of someone: T F
7. I'm always willing to admit it when I make a mistake: T F
8. I sometimes try to get even rather than forgive and forget: T F
9. I am always courteous, even to people who are disagreeable: T F
10. I have never been irked when people expressed ideas very different from my own: T F
11. There have been times when I was quite jealous of the good fortune of others: T F
12. I am sometimes irritated by people who ask favors of me: T F
13. I have never deliberately said something that hurt someone's feelings: T F
What Other things can improve the quality of services provided by cardiac anesthesiologists to the department of cardiac surgery:
| References|| |
Heidegger T, Saal D, Nuebling M. Patient satisfaction with anaesthesia care: What is patient satisfaction, how should it be measured, and what is the evidence for assuring high patient satisfaction? Best Pract Res Clin Anaesthesiol 2006;20:331-46.
Bell DM, Halliburton JR, Preston JC. An evaluation of anesthesia patient satisfaction instruments. AANA J 2004;72:211-7.
Fung D, Cohen MM. Measuring patient satisfaction with anesthesia care: A review of current methodology. Anesth Analg 1998;87:1089-98.
Schaefer HG, Helmreich RL, Scheidegger D. Safety in the operating theatre--part 1: Interpersonal relationships and team performance. Curr Anaesth Crit Care 1995;6:48-53.
Le May S, Dupuis G, Harel F, Taillefer MC, Dubé S, Hardy JF. Clinimetric scale to measure surgeons' satisfaction with anesthesia services. Can J Anaesth J Can Anesth 2000;47:398-405.
Eagle CJ, Davies JM. Current models of “quality”--an introduction for anaesthetists. Can J Anaesth J Can Anesth 1993;40:851-62.
Haller G, Stoelwinder J, Myles PS, McNeil J. Quality and safety indicators in anesthesia: A systematic review. Anesthesiology 2009;110:1158-75.
Aiken LR. Three coefficients for analyzing the reliability and validity of ratings. Educ Psychol Meas 1985;45:131-42.
Aiken L. Content validity and reliability of single items or questionnaires - Lewis R. Aiken, 1980. Educ Psychol Meas 1980;40:955-9.
Choi H-H, Han K-T, Nam CM, Moon KT, Kim W, Park E-C. Association between human resources and risk of hospitalisation in end-stage renal disease outpatients receiving haemodialysis: A longitudinal cohort study using claim data during 2013-2014. BMJ Open 2016;6:e011319.
Hall LH, Johnson J, Watt I, O'Connor DB. Association of GP wellbeing and burnout with patient safety in UK primary care: A cross-sectional survey. Br J Gen Pract 2019;69:e507-14.
Papaspyros SC, Javangula KC, Adluri RKP, O'Regan DJ. Briefing and debriefing in the cardiac operating room. Analysis of impact on theatre team attitude and patient safety. Interact Cardiovasc Thorac Surg 2010;10:43-7.
Manser T. Teamwork and patient safety in dynamic domains of healthcare: A review of the literature. Acta Anaesthesiol Scand 2009;53:143-51.
Lee A, Lum ME. Measuring anaesthetic outcomes. Anaesth Intensive Care 1996;24:685-93.
Duncan P. Quality: A job well done! - Abstract - Europe PMC. Can J Anesth 1993;40:813-5.
Mortel van de TF. Faking it: Social desirability response bias in self-report research. Aust J Adv Nurs 2008;25:40-8.
Edens JF, Buffington JK, Tomicic TL, Riley BD. Effects of positive impression management on the psychopathic personality inventory. Law Hum Behav 2001;25:235-56.
Watt S, Simpson C, McKillop C, Nunn V. Electronic course surveys: Does automating feedback and reporting give better results? Assess Eval High Educ 2002;27:325-37.
Epstein KR, Laine C, Farber NJ, Nelson EC, Davidoff F. Patients' perceptions of office medical practice: Judging quality through the patients' eyes. Am J Med Qual 1996;11:73-80.
Department of Cardiac Surgery, NH Rabindranath Tagore International Institute of Cardiac Sciences, Kolkata - 700 099, West Bengal
Source of Support: None, Conflict of Interest: None
[Table 1], [Table 2]