Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals
Objectives To extend reliability of WHO Behaviourally Anchored Rating Scale (WHOBARS) to measure the quality of WHO Surgical Safety Checklist administration using generalisability theory. In this context, extending reliability refers to establishing generalisability of the tool scores across populat...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2019-01-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/9/1/e022625.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823858104554487808 |
|---|---|
| author | Jennifer M Weller Carmen Skilton Derryn A Gargiulo Oleg N Medvedev Simon J Mitchell Alan F Merry |
| author_facet | Jennifer M Weller Carmen Skilton Derryn A Gargiulo Oleg N Medvedev Simon J Mitchell Alan F Merry |
| author_sort | Jennifer M Weller |
| collection | DOAJ |
| description | Objectives To extend reliability of WHO Behaviourally Anchored Rating Scale (WHOBARS) to measure the quality of WHO Surgical Safety Checklist administration using generalisability theory. In this context, extending reliability refers to establishing generalisability of the tool scores across populations of teams and raters by accounting for the relevant sources of measurement errors.Design Cross-sectional random effect measurement design assessing surgical teams by the five items on the three Checklist phases, and at three sites by two trained raters simultaneously.Setting The data were collected in three tertiary hospitals in Auckland, New Zealand in 2016 and included 60 teams observed in 60 different cases with an equal number of teams (n=20) per site. All elective and acute cases (adults and children) involving surgery under general anaesthesia during normal working hours were eligible.Participants The study included 243 surgical staff members, 138 (50.12%) women.Main outcome measure Absolute generalisability coefficient that accounts for variance due to items, phases, sites and raters for the WHOBARS measure of the quality of WHO Surgical Safety Checklist administration.Results The WHOBARS in its present form has demonstrated good generalisability of scores across teams and raters (G absolute=0.83). The largest source of measurement error was the interaction between the surgical team and the rater, accounting for 16.7% (95% CI 16.4 to 16.9) of the total variance in the data. Removing any items from the WHOBARS led to a decrease in the overall reliability of the instrument.Conclusions Assessing checklist administration quality is important for promoting improvement in its use, and WHOBARS offers a reliable approach for doing this. |
| format | Article |
| id | doaj-art-2a9bee1c3a164369b378cc09607cf31a |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2019-01-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-2a9bee1c3a164369b378cc09607cf31a2025-02-11T13:55:09ZengBMJ Publishing GroupBMJ Open2044-60552019-01-019110.1136/bmjopen-2018-022625Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitalsJennifer M Weller0Carmen Skilton1Derryn A Gargiulo2Oleg N Medvedev3Simon J Mitchell4Alan F Merry5Department of Anaesthesia, Auckland City Hospital, Auckland, New Zealand1 Center for Medical and Health Sciences Education, University of Auckland, Auckland, New Zealand2 Department of Anaesthesiology, University of Auckland, Auckland, New Zealand1 Center for Medical and Health Sciences Education, University of Auckland, Auckland, New Zealand2 Department of Anaesthesiology, University of Auckland, Auckland, New ZealandDepartment of Anaesthesiology, The University of Auckland School of Medicine, Auckland, Auckland, New ZealandObjectives To extend reliability of WHO Behaviourally Anchored Rating Scale (WHOBARS) to measure the quality of WHO Surgical Safety Checklist administration using generalisability theory. In this context, extending reliability refers to establishing generalisability of the tool scores across populations of teams and raters by accounting for the relevant sources of measurement errors.Design Cross-sectional random effect measurement design assessing surgical teams by the five items on the three Checklist phases, and at three sites by two trained raters simultaneously.Setting The data were collected in three tertiary hospitals in Auckland, New Zealand in 2016 and included 60 teams observed in 60 different cases with an equal number of teams (n=20) per site. All elective and acute cases (adults and children) involving surgery under general anaesthesia during normal working hours were eligible.Participants The study included 243 surgical staff members, 138 (50.12%) women.Main outcome measure Absolute generalisability coefficient that accounts for variance due to items, phases, sites and raters for the WHOBARS measure of the quality of WHO Surgical Safety Checklist administration.Results The WHOBARS in its present form has demonstrated good generalisability of scores across teams and raters (G absolute=0.83). The largest source of measurement error was the interaction between the surgical team and the rater, accounting for 16.7% (95% CI 16.4 to 16.9) of the total variance in the data. Removing any items from the WHOBARS led to a decrease in the overall reliability of the instrument.Conclusions Assessing checklist administration quality is important for promoting improvement in its use, and WHOBARS offers a reliable approach for doing this.https://bmjopen.bmj.com/content/9/1/e022625.full |
| spellingShingle | Jennifer M Weller Carmen Skilton Derryn A Gargiulo Oleg N Medvedev Simon J Mitchell Alan F Merry Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals BMJ Open |
| title | Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals |
| title_full | Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals |
| title_fullStr | Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals |
| title_full_unstemmed | Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals |
| title_short | Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals |
| title_sort | examining reliability of whobars a tool to measure the quality of administration of who surgical safety checklist using generalisability theory with surgical teams from three new zealand hospitals |
| url | https://bmjopen.bmj.com/content/9/1/e022625.full |
| work_keys_str_mv | AT jennifermweller examiningreliabilityofwhobarsatooltomeasurethequalityofadministrationofwhosurgicalsafetychecklistusinggeneralisabilitytheorywithsurgicalteamsfromthreenewzealandhospitals AT carmenskilton examiningreliabilityofwhobarsatooltomeasurethequalityofadministrationofwhosurgicalsafetychecklistusinggeneralisabilitytheorywithsurgicalteamsfromthreenewzealandhospitals AT derrynagargiulo examiningreliabilityofwhobarsatooltomeasurethequalityofadministrationofwhosurgicalsafetychecklistusinggeneralisabilitytheorywithsurgicalteamsfromthreenewzealandhospitals AT olegnmedvedev examiningreliabilityofwhobarsatooltomeasurethequalityofadministrationofwhosurgicalsafetychecklistusinggeneralisabilitytheorywithsurgicalteamsfromthreenewzealandhospitals AT simonjmitchell examiningreliabilityofwhobarsatooltomeasurethequalityofadministrationofwhosurgicalsafetychecklistusinggeneralisabilitytheorywithsurgicalteamsfromthreenewzealandhospitals AT alanfmerry examiningreliabilityofwhobarsatooltomeasurethequalityofadministrationofwhosurgicalsafetychecklistusinggeneralisabilitytheorywithsurgicalteamsfromthreenewzealandhospitals |