Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19
Objectives To reliably quantify the radiographic severity of COVID-19 pneumonia with the Radiographic Assessment of Lung Edema (RALE) score on clinical chest X-rays among inpatients and examine the prognostic value of baseline RALE scores on COVID-19 clinical outcomes.Setting Hospitalised patients w...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2023-01-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/13/1/e066626.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1823857102028800000 |
|---|---|
| author | Andrew I Kim William Bain Alison Morris Barbara Methe Bryan J McVerry Georgios D Kitsios Daniel Kotok Janet S Lee Nameer Al-Yousif Christine E Girard Jana J Jacobs Konstantin Golubykh Hafiz Qurashi Michael Lu Saketram Komanduri Anatoliy Korzhuk Halimat O Lawal Nicholas Abourizk Caitlin Schaefer Kevin J Mitchell Catherine M Dietz Ellen K Hughes Clara S Brandt Georgia M Fitzgerald Robin Joyce Asmaa S Chaudhry Jose D Rivera Shruti Shettigar Allen Lavina Samantha R Gillenwater Anas Hadeh Faraaz A Shah Matthew Bittner Niall Prendergast John Evankovich Navitha Ramesh Cathy Kessinger |
| author_facet | Andrew I Kim William Bain Alison Morris Barbara Methe Bryan J McVerry Georgios D Kitsios Daniel Kotok Janet S Lee Nameer Al-Yousif Christine E Girard Jana J Jacobs Konstantin Golubykh Hafiz Qurashi Michael Lu Saketram Komanduri Anatoliy Korzhuk Halimat O Lawal Nicholas Abourizk Caitlin Schaefer Kevin J Mitchell Catherine M Dietz Ellen K Hughes Clara S Brandt Georgia M Fitzgerald Robin Joyce Asmaa S Chaudhry Jose D Rivera Shruti Shettigar Allen Lavina Samantha R Gillenwater Anas Hadeh Faraaz A Shah Matthew Bittner Niall Prendergast John Evankovich Navitha Ramesh Cathy Kessinger |
| author_sort | Andrew I Kim |
| collection | DOAJ |
| description | Objectives To reliably quantify the radiographic severity of COVID-19 pneumonia with the Radiographic Assessment of Lung Edema (RALE) score on clinical chest X-rays among inpatients and examine the prognostic value of baseline RALE scores on COVID-19 clinical outcomes.Setting Hospitalised patients with COVID-19 in dedicated wards and intensive care units from two different hospital systems.Participants 425 patients with COVID-19 in a discovery data set and 415 patients in a validation data set.Primary and secondary outcomes We measured inter-rater reliability for RALE score annotations by different reviewers and examined for associations of consensus RALE scores with the level of respiratory support, demographics, physiologic variables, applied therapies, plasma host–response biomarkers, SARS-CoV-2 RNA load and clinical outcomes.Results Inter-rater agreement for RALE scores improved from fair to excellent following reviewer training and feedback (intraclass correlation coefficient of 0.85 vs 0.93, respectively). In the discovery cohort, the required level of respiratory support at the time of CXR acquisition (supplemental oxygen or non-invasive ventilation (n=178); invasive-mechanical ventilation (n=234), extracorporeal membrane oxygenation (n=13)) was significantly associated with RALE scores (median (IQR): 20.0 (14.1–26.7), 26.0 (20.5–34.0) and 44.5 (34.5–48.0), respectively, p<0.0001). Among invasively ventilated patients, RALE scores were significantly associated with worse respiratory mechanics (plateau and driving pressure) and gas exchange metrics (PaO2/FiO2 and ventilatory ratio), as well as higher plasma levels of IL-6, soluble receptor of advanced glycation end-products and soluble tumour necrosis factor receptor 1 (p<0.05). RALE scores were independently associated with 90-day survival in a multivariate Cox proportional hazards model (adjusted HR 1.04 (1.02–1.07), p=0.002). We replicated the significant associations of RALE scores with baseline disease severity and mortality in the independent validation data set.Conclusions With a reproducible method to measure radiographic severity in COVID-19, we found significant associations with clinical and physiologic severity, host inflammation and clinical outcomes. The incorporation of radiographic severity assessments in clinical decision-making may provide important guidance for prognostication and treatment allocation in COVID-19. |
| format | Article |
| id | doaj-art-ee2eebba292e4aff996a75ee343c60db |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2023-01-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-ee2eebba292e4aff996a75ee343c60db2025-02-12T01:15:10ZengBMJ Publishing GroupBMJ Open2044-60552023-01-0113110.1136/bmjopen-2022-066626Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19Andrew I Kim0William Bain1Alison Morris2Barbara Methe3Bryan J McVerry4Georgios D Kitsios5Daniel Kotok6Janet S Lee7Nameer Al-Yousif8Christine E Girard9Jana J Jacobs10Konstantin Golubykh11Hafiz Qurashi12Michael Lu13Saketram Komanduri14Anatoliy Korzhuk15Halimat O Lawal16Nicholas Abourizk17Caitlin Schaefer18Kevin J Mitchell19Catherine M Dietz20Ellen K Hughes21Clara S Brandt22Georgia M Fitzgerald23Robin Joyce24Asmaa S Chaudhry25Jose D Rivera26Shruti Shettigar27Allen Lavina28Samantha R Gillenwater29Anas Hadeh30Faraaz A Shah31Matthew Bittner32Niall Prendergast33John Evankovich34Navitha Ramesh35Cathy Kessinger36Department of Pulmonary and Critical Care, Cleveland Clinic Florida, Weston, Florida, USADivision of Pulmonary, Allergy and Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USACenter for Medicine and the Microbiome, University of Pittsburgh, Pittsburgh, Pennsylvania, USADivision of Pulmonary, Allergy, Critical Care and Sleep Medicine, Department of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USAPulmonary, Allergy, and Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania, USADivision of Pulmonary, Allergy and Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USAWeston Hospital, Cleveland Clinic Florida, Weston, Florida, USAAcute Lung Injury Center of Excellence, University of Pittsburgh, Pittsburgh, Pennsylvania, USAGeorge and Fay Yee Center for Healthcare Innovation, University of Manitoba, Winnipeg, Manitoba, CanadaDepartment of Internal Medicine, Cleveland Clinic Florida, Weston, Florida, USADivision of Infectious Diseases, Department of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USAInternal Medicine Residency Program, UPMC Pinnacle Harrisburg, Harrisburg, Pennsylvania, USADepartment of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USACardiovascular Imaging Research Center, Massachusetts General Hospital, Boston, Massachusetts, USAInternal Medicine Residency Program, UPMC Pinnacle Harrisburg, Harrisburg, Pennsylvania, USAInternal Medicine Residency Program, UPMC Pinnacle Harrisburg, Harrisburg, Pennsylvania, USAInternal Medicine Residency Program, UPMC Pinnacle Harrisburg, Harrisburg, Pennsylvania, USAInternal Medicine Residency Program, UPMC Pinnacle Harrisburg, Harrisburg, Pennsylvania, USAAcute Lung Injury Center of Excellence, University of Pittsburgh, Pittsburgh, Pennsylvania, USAComputer Vision Group, Veytel LLC, Pittsburgh, Pennsylvania, USAComputer Vision Group, Veytel LLC, Pittsburgh, Pennsylvania, USAComputer Vision Group, Veytel LLC, Pittsburgh, Pennsylvania, USAComputer Vision Group, Veytel LLC, Pittsburgh, Pennsylvania, USAComputer Vision Group, Veytel LLC, Pittsburgh, Pennsylvania, USAComputer Vision Group, Veytel LLC, Pittsburgh, Pennsylvania, USAComputer Vision Group, Veytel LLC, Pittsburgh, Pennsylvania, USADepartment of Pulmonary and Critical Care, Cleveland Clinic Florida, Weston, Florida, USADepartment of Pulmonary and Critical Care, Cleveland Clinic Florida, Weston, Florida, USADepartment of Pulmonary and Critical Care, Cleveland Clinic Florida, Weston, Florida, USADepartment of Pulmonary and Critical Care, Cleveland Clinic Florida, Weston, Florida, USADepartment of Pulmonary and Critical Care, Cleveland Clinic Florida, Weston, Florida, USADivision of Pulmonary, Allergy and Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USAInternal Medicine Residency Program, University of Pittsburgh, Pittsburgh, Pennsylvania, USADivision of Pulmonary, Allergy and Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USADivision of Pulmonary, Allergy and Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USADepartment of Pulmonary and Critical Care, UPMC Pinnacle Harrisburg, Harrisburg, Pennsylvania, USADivision of Pulmonary, Allergy and Critical Care Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania, USAObjectives To reliably quantify the radiographic severity of COVID-19 pneumonia with the Radiographic Assessment of Lung Edema (RALE) score on clinical chest X-rays among inpatients and examine the prognostic value of baseline RALE scores on COVID-19 clinical outcomes.Setting Hospitalised patients with COVID-19 in dedicated wards and intensive care units from two different hospital systems.Participants 425 patients with COVID-19 in a discovery data set and 415 patients in a validation data set.Primary and secondary outcomes We measured inter-rater reliability for RALE score annotations by different reviewers and examined for associations of consensus RALE scores with the level of respiratory support, demographics, physiologic variables, applied therapies, plasma host–response biomarkers, SARS-CoV-2 RNA load and clinical outcomes.Results Inter-rater agreement for RALE scores improved from fair to excellent following reviewer training and feedback (intraclass correlation coefficient of 0.85 vs 0.93, respectively). In the discovery cohort, the required level of respiratory support at the time of CXR acquisition (supplemental oxygen or non-invasive ventilation (n=178); invasive-mechanical ventilation (n=234), extracorporeal membrane oxygenation (n=13)) was significantly associated with RALE scores (median (IQR): 20.0 (14.1–26.7), 26.0 (20.5–34.0) and 44.5 (34.5–48.0), respectively, p<0.0001). Among invasively ventilated patients, RALE scores were significantly associated with worse respiratory mechanics (plateau and driving pressure) and gas exchange metrics (PaO2/FiO2 and ventilatory ratio), as well as higher plasma levels of IL-6, soluble receptor of advanced glycation end-products and soluble tumour necrosis factor receptor 1 (p<0.05). RALE scores were independently associated with 90-day survival in a multivariate Cox proportional hazards model (adjusted HR 1.04 (1.02–1.07), p=0.002). We replicated the significant associations of RALE scores with baseline disease severity and mortality in the independent validation data set.Conclusions With a reproducible method to measure radiographic severity in COVID-19, we found significant associations with clinical and physiologic severity, host inflammation and clinical outcomes. The incorporation of radiographic severity assessments in clinical decision-making may provide important guidance for prognostication and treatment allocation in COVID-19.https://bmjopen.bmj.com/content/13/1/e066626.full |
| spellingShingle | Andrew I Kim William Bain Alison Morris Barbara Methe Bryan J McVerry Georgios D Kitsios Daniel Kotok Janet S Lee Nameer Al-Yousif Christine E Girard Jana J Jacobs Konstantin Golubykh Hafiz Qurashi Michael Lu Saketram Komanduri Anatoliy Korzhuk Halimat O Lawal Nicholas Abourizk Caitlin Schaefer Kevin J Mitchell Catherine M Dietz Ellen K Hughes Clara S Brandt Georgia M Fitzgerald Robin Joyce Asmaa S Chaudhry Jose D Rivera Shruti Shettigar Allen Lavina Samantha R Gillenwater Anas Hadeh Faraaz A Shah Matthew Bittner Niall Prendergast John Evankovich Navitha Ramesh Cathy Kessinger Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19 BMJ Open |
| title | Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19 |
| title_full | Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19 |
| title_fullStr | Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19 |
| title_full_unstemmed | Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19 |
| title_short | Inter-rater reliability and prognostic value of baseline Radiographic Assessment of Lung Edema (RALE) scores in observational cohort studies of inpatients with COVID-19 |
| title_sort | inter rater reliability and prognostic value of baseline radiographic assessment of lung edema rale scores in observational cohort studies of inpatients with covid 19 |
| url | https://bmjopen.bmj.com/content/13/1/e066626.full |
| work_keys_str_mv | AT andrewikim interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT williambain interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT alisonmorris interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT barbaramethe interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT bryanjmcverry interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT georgiosdkitsios interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT danielkotok interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT janetslee interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT nameeralyousif interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT christineegirard interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT janajjacobs interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT konstantingolubykh interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT hafizqurashi interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT michaellu interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT saketramkomanduri interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT anatoliykorzhuk interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT halimatolawal interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT nicholasabourizk interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT caitlinschaefer interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT kevinjmitchell interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT catherinemdietz interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT ellenkhughes interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT clarasbrandt interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT georgiamfitzgerald interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT robinjoyce interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT asmaaschaudhry interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT josedrivera interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT shrutishettigar interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT allenlavina interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT samanthargillenwater interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT anashadeh interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT faraazashah interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT matthewbittner interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT niallprendergast interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT johnevankovich interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT navitharamesh interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 AT cathykessinger interraterreliabilityandprognosticvalueofbaselineradiographicassessmentoflungedemaralescoresinobservationalcohortstudiesofinpatientswithcovid19 |