Development and validation of a postoperative nausea and vomiting intensity scale (2024)

Search for other works by this author on:

Article history

Abstract

The incidence of postoperative nausea and vomiting (PONV) in an untreated adult surgical population receiving general anaesthesia is around 20–30%,^1–10 but this increases up to 80% in patients with two or more risk factors for PONV.³ Although PONV is sometimes viewed as a ‘minor’⁷ problem, and patients may be willing to forego effective PONV prophylaxis in preference for better analgesia,¹¹ there are possible serious consequences.^1,2,12,13 Nevertheless, it must be acknowledged that some episodes of PONV are mild, transient, and inconsequential.

There have been attempts to create a score that could be used to rate the severity of nausea, vomiting, and retching in patients undergoing chemotherapy.^14,15 However, no score or index has been validated to define clinically important PONV, that is, at what point does PONV become a significant and important postoperative complication, and more importantly, can this be defined from a patient's perspective? Techniques used in the past to measure PONV have included absolute counts or incidence rate, Likert scales, visual analogue scales (VAS), and treatment response. Some studies make reference to ‘severe’ PONV without any definition or justification of the cut-off values that discriminate severe PONV from what may otherwise be a trivial event. This issue has similarities with the measurement of pain intensity, in which it is generally accepted that a clinically important reduction in pain intensity is reflected by a reduction in a 100 mm VAS score of at least 15,¹⁶ and severe pain being represented by a score greater than 70.¹⁷

A clinician's perspectives and preferences may differ from that of a patient.^18,19 We aimed to use psychometric methods to develop and validate a measurement scale to define clinically important PONV which takes into account patient views and expert medical and nursing opinion in its development and evaluation. Prospective validation of the derived scale was then undertaken.

Methods

This prospective observational study had two phases: (i) literature review, participant survey, and development of a PONV Intensity Scale (‘development phase’); and (ii) prospective psychometric evaluation of the scale in a second cohort of patients (‘validation phase’). Ethics committee approval and consent was obtained from all subjects.

Patients were excluded from the study if they had a psychiatric disturbance that precluded complete cooperation, a history of alcohol or drug dependence, distress or any severe pre-existing medical condition that limited objective assessment after operation, or the presence of any life-threatening postoperative complication.

The participant interviews were in the most part conducted by a single fulltime investigator (T.M. for development phase, R.W. for the validation phase). However, on some occasions the participants were interviewed by an experienced anaesthesia research nurse or registrar. All the interviews were conducted according to the study protocol.

Development of the PONV Intensity Scale

The developmental phase was preceded by a literature review, and then a structured questionnaire asking clinicians (anaesthetists, nurses, surgeons), patients, and their family members to identify features of PONV that would describe its intensity and clinical importance. Key features included the intensity, pattern, and duration of nausea. For vomiting, participants were asked how many times a patient would have to vomit before they would be regarded as having severe PONV. The developmental phase concluded with the composition of a PONV scale that reflected the views provided by the survey. Further details are provided in the Supplementary Appendix.

Validation of the PONV Intensity Scale

The second phase of validation study evaluated the psychometric properties of the derived scale. A further 183 patients were approached in the postoperative period after being identified as having PONV, requiring treatment for PONV, or being at high risk of PONV.^3,5 We chose to include a sequential cohort of adult patients from each of the following types of surgery:We used the recommended psychometric techniques.^20–26 These included tests of content, construct, discriminant, and predictive validity; inter-rater reliability; internal consistency; responsiveness and effect size. Further details are provided in the Supplementary Appendix.

• Ambulatory surgical patients, where hospital discharge was planned on the same day of surgery.

• Inpatients undergoing minor surgery, defined by a duration <2 h and expected blood loss <500 ml, or a planned hospital stay of <2 days.

• Inpatients undergoing major surgery, defined by having at least 3 of: (i) surgery with a duration >2 h, (ii) planned hospital stay of >2 days, (iii) intracranial or body cavity (i.e. abdominal or thoracic) surgery, or (iv) expected blood loss >500 ml, or the need for postoperative i.v. patient-controlled analgesia or epidural block.

Data collection

In both phases of the study we recorded baseline data including patient anxiety and depression using the validated hospital anxiety and depression (HAD) Scale,²⁷ for which scores <7 indicate normal functioning. Nausea and vomiting were recorded as present or absent, number of vomits, and extent and duration of nausea. Pain after surgery was measured using a 100 mm VAS, and was recorded at rest and during movement (coughing, deep breathing, or walking). The quality of recovery (QoR) after surgery was measured using the QoR score or QoR-40, both validated measures of quality of recovery after surgery and anaesthesia.^28,29 All measurements and ratings were completed at the time when patients rated their PONV intensity.

We also recorded factors known to be associated with the risk of developing PONV, including known risk factors for PONV.³ Apfel scores were then calculated to ascertain the percentage risk for patients developing PONV, with scores of 2 or greater indicating high risk.³ Other indices or consequences of what may constitute clinically important PONV were recorded; these included wound dehiscence, the need for i.v. fluids because of an inability to tolerate oral intake, electrolyte imbalance, pneumothorax, s.c. emphysema, and any other adverse effects identified by patients or investigators.

Patients were asked to provide a global rating of their nausea intensity using a 100 mm nausea VAS. The limits of the nausea VAS were ‘no nausea’ to ‘nausea as bad as it possibly could be’. The association between the nausea VAS and the PONV Intensity Scale was used as a measure of construct validity. We also measured duration of hospital stay and expected time off work.

Given that vomiting and dry-retching reflect similar physiological processes and are similarly distressing to patients, and our early experience with measuring PONV intensity identified occasional uncertainty as to how to rank severity of nausea, we modified the PONV Intensity Scale to clarify these issues (Fig.1). In order to ascertain whether the revision otherwise provided comparable results with that of the first version of the scale, we measured the agreement using both versions of the scale in the final 63 patients recruited into the study.

The sample size selected for this study was guided by previous studies^28,29 and in part chosen for convenience, given that power calculations cannot be readily made with correlational analysis. A post hoc analysis of the derived score was planned for the development phase in order to test these assumptions, and this was used in the sample size estimation for the validation phase in order to achieve at least 80% power for a comparison of QoR-40 scores.

Statistical analysis

Data are summarized as mean (sd), median [inter-quartile range (IQR)], or number (%). Differences between proportions were analysed using the χ² test, and risk estimates were done with odds ratio (OR) or risk ratio (RR) and 95% confidence intervals (CI). For normally distributed data, Student's t-test was used to compare means of two groups. When data were not normally distributed, Wilcoxon signed-rank test was used. In selected patients, non-normal data were log-transformed to calculate geometric means, than enabling the use of t-tests. Rating differences in the questionnaires between the groups (patients/relatives, nursing, anaesthetic and surgical staff) were investigated using non-parametric Kruskall–Wallis analysis of variance. Associations were measured using Spearman rank correlation, ρ. Internal consistency was measured using Cronbach's α.²⁵ Agreement was measured using intraclass correlation (ICC) for continuous variables³⁰ and the kappa statistic for dichotomous variables. Responsiveness was measured using standardized response means, calculated as the mean change divided by its sd.²⁶ All statistical analyses were performed using SPSS for Windows V16.0 (SPSS Inc., Chicago, IL, USA). A P-value of <0.05 was considered significant; no correction was made for multiple comparisons.

Results

In the development phase, a total of 191 participants were approached for the study; 188 were eligible and 180 were enrolled (recruitment rate 96%). In the validation phase, a further 183 patients were approached; 171 were eligible and 163 were enrolled (recruitment rate 95%; Supplementary Figs1 and 2).

Development phase

Patient, surgical, and anaesthetic characteristics are presented in the Supplementary Tables1–4. Of the postoperative patients who experienced PONV (39%), 20 (35%) had nausea and six (11%) vomited; 10 (18%) of these patients received antiemetic treatment.

In most patients there were no significant differences between the groups (patients, family members, and clinical staff) (Supplementary Tables5–8) as to the duration of nausea regarded as clinically important (Table1). However, the duration of moderate varying and severe nausea were different between the groups. There was general consensus among family members and staff that three or more vomits was clinically important (Supplementary Table6). Staff were more likely to tolerate more intense nausea (in their patients) for longer than patients themselves. There was no significant correlation between the anxiety and depression score and the duration of nausea considered clinically important. Participants' past history of PONV had little impact on their determinations of the duration of clinically important nausea (results not shown).

In the development and pilot-testing, the duration data related to clinical importance were used to develop a PONV Intensity Scale. In general, we found that moderate nausea had twice the impact of mild nausea, and severe nausea had about 24 times the impact of moderate nausea. In addition, the ratio between the duration of constant and varying nausea was about 2:1. We thus created a scoring system that reflected the relationship between nausea intensity, pattern, and duration according to the relative durations reported by the participants. Although there were some differences between clinical staff and patients, particularly with the duration of severe nausea, we chose to place more emphasis on patient ratings.

We thus developed a score to measure PONV intensity, where a score ≥50 defined clinically important PONV: PONV Intensity Scale=severity of nausea (1=mild, 2=moderate, 3=severe)×pattern of nausea (1=varying, 2=constant)×duration of nausea (in hours).

The PONV intensity score was first pilot-tested on postoperative patients in the development phase of the study, with results available in the Supplementary material. This was then adapted to create an Intensity Scale (Fig.1) that underwent extensive prospective validation.

Validation phase

Patient, surgical, and anaesthetic data for the validation cohort (Table2) showed that this cohort represented a fairly high-risk group for developing PONV, with nearly 96% of patients being classified with an Apfel Scale of 2 or greater. Around 40% had a past history of PONV, motion sickness, or both. Despite being a high-risk population for developing PONV, 33% of patients did not receive antiemetic prophylaxis and a further 33% received only one antiemetic.

Nearly all (97%) patients experienced nausea in the postoperative period, and about half (45%) of them were in the severe category (Table3), and just more than half (55%) of the patients vomited in the postoperative period.

Validity testing

While 45% of participants were categorized as having ‘severe’ characteristics of PONV, less than half of them (18% of the cohort) had clinically important PONV as defined by a score of ≥50. The median (IQR) PONV Intensity Scale for all participants was 8.0 (0.9–40). The mean nausea VAS scale was 52 (sd 30).

Patients with clinically important PONV had significantly higher nausea VAS scales when compared with those without clinically important PONV (Table4). Patients with clinically important PONV scores also had a significantly poorer quality of recovery (Supplementary Fig.3). In addition, to ensure that a spurious correlation did not exist between the QoR-40 scales and patients with clinically important PONV, three items related to nausea and vomiting were removed from the QoR-40 scale and the significant difference remained marked: mean difference 15 (95% CI 9–21), P<0.0005. Patients with clinically important PONV needed twice as many doses of antiemetic (Table4), and had greatly increased risk of typical complications and other consequences of PONV (Fig. 2).

Women were more likely than men to have higher PONV intensity scores [17 (1.5–48) vs 1.8 (0.2–19), P=0.001], and to be classified as clinically important PONV [OR 3.0 (95% CI 1.1–8.2), P=0.032]. An increase in preoperative Apfel scores was associated with an increase in the proportion of patients with clinically important PONV: 1=0%, 2=14%, 3=31%, 4=55%; P=0.004.

The median (IQR) hospital stay for patients with clinically important PONV was 4 (3–5) days and for those without clinically important PONV 3 (2–6) days, P=0.50; the estimated time off work was 42 (28–105) days and 30 (14–60) days, respectively, P=0.15.

Nausea VAS

During the conduct of the study we found that the nausea VAS was quick to administer and easily understood by investigators and patients. Its common use in previous studies and our ease of use raised the possibility of this being a useful measure of nausea intensity, despite not having been previously validated. We thus evaluated its ability to discriminate clinically important PONV from unimportant PONV using two cut-off values:

1. Nausea VAS score ≥50. A cut-off value of 50 or more, being the mid-point and close to the calculated mean VAS score, was first arbitrarily used as an alternative criterion of clinically important PONV; this resulted in 116 patients (71%) being classified as having clinically important PONV (Table5). Although statistically significant, we found that the discriminatory ability was less than when using the PONV Intensity Scale. The agreement between both scales to identify clinically important PONV was poor (κ 0.29; P<0.0005).

2. Nausea VAS score ≥75. A cut-off value of 75 [being close to the 75th centile (VAS=76)] resulted in 47 patients (29%) being classified as having clinically important PONV. When using a nausea VAS of 75 or more (VAS₇₅), results were statistically significant, and the discriminatory ability was comparable with the PONV Intensity Scale (Table5). The agreement between both scales to identify clinically important PONV was moderately strong (κ 0.49; P<0.0005). However, if vomiting on three or more occasions is included in this definition, the agreement remains moderate (κ 0.39).

The PONV Intensity Scale had a stronger correlation (ρ 0.58, P<0.0005) with the total number of antiemetic doses needed when compared with the nausea VAS₇₅ (ρ 0.50, P<0.0005). There was no difference in agreement between the two measures with the total number of vomits (ρ 0.30 vs ρ 0.29; both P<0.0005); however, there was a large difference between the associations between the PONV Intensity Scale (ρ 0.83, P<0.0005) and VAS₇₅ (ρ 0.58, P<0.0005) with duration of nausea experienced. Further analyses were completed using consequences of PONV (Table6), which demonstrate that although both the Scales and VAS₇₅ were able to identify consequences of PONV, the scale was a more sensitive measure. If vomiting on three or more occasions is included in this definition the agreement is less (results not shown).

Revision of the PONV Intensity Scale

A total of 63 patients were tested with the original and revised versions of the PONV Intensity Scale. The revised PONV Intensity Scale identified 27% of this cohort as having clinically important PONV. There was a strong correlation between both versions of the scale (ρ 0.83; P<0.0005). Agreement between the versions to determine clinical significance (scores ≥50) were strong (κ 0.69; P<0.0005).

Consistent with results obtained with the original version of the PONV Intensity Scale, agreement between clinically important scales and VAS₇₅ was moderate (κ 0.46; P<0.0005). Correlation between the revised Intensity Scale and the VAS₇₅ was also moderate (ρ 0.58; P<0.0005).

Reliability testing

Test–retest and inter-rater reliability were completed in 22 patients using a single method. The reliability coefficient was excellent for the PONV Intensity Scale [ICC 0.99 (95% CI 0.99–1.0), P<0.0005], and strong for the nausea VAS₇₅ [ICC 0.91 (95% CI 0.78–0.96), P<0.0005]. We also used the revised PONV Intensity Scale in 16 patients, for which correlation was also excellent [ICC 0.97 (95% CI 0.91–0.99), P<0.0005].

The six items used to determine severity of nausea for the PONV Intensity Scale had very good internal consistency (Cronbach's α 0.78). The inter-item correlations are shown in Supplementary Table9. Most items had moderate to strong correlation, with no evidence of redundancy.

Responsiveness testing and effect size

The correlations between the change in PONV Intensity Scales [original version (n=36) and revised version (n=22)] with the reduction in number of antiemetics needed at the second interview were moderately strong (ρ 0.74, P<0.0005; and ρ 0.53, P=0.011, respectively). For those patients who had changed from having clinically important PONV scores to unimportant PONV scores, there was a significant reduction in the number of medications required since the first interview when using both versions of the PONV Intensity Scale (P=0.007 and P=0.036, respectively). Of the eight patients who changed from having clinically important to unimportant PONV using the original version of the scale, seven (88%) rated themselves as having good relief on responsiveness questioning. Similarly for the revised scale, five of six (83%) patients rated themselves as having good PONV relief. Good PONV relief was defined as a health status change that was either ‘satisfactory’, ‘good’, or ‘excellent’.

The differences in PONV Intensity Scale between the two interview times were then analysed. Log transformations of the scales were done owing to positively skewed data, and we compared geometric means. A significant difference was found in the ΔPONV Intensity Scales between patients who had a change from clinically important PONV to unimportant PONV (n=8) and those who had no clinical change (n=28). The geometric mean scores were 301 (3.7) and 4.6 (14), respectively (P<0.0005). Similar results were obtained when using the revised version of the scale (n=6 vs n=16), geometric means of 155 (3.1) and 3.6 (14) (P=0.001). The effect size for the change in PONV scales at the second interview when compared with the first, comparing patients who had a change from clinically important to unimportant PONV and those who had no clinical change was calculated using the difference in geometric means divided by their pooled standard deviation. The effect size was 0.82, illustrating excellent discriminatory ability.²⁹

Discussion

This study sought to develop a measurement tool for PONV intensity in order to identify clinically important PONV. The development phase found minimal differences between staff and patient opinion for the indices used to measure clinical importance. Patients are likely to prioritize their comfort and quality of recovery after surgery over other complications.^31,32 However, clinical staff are more likely to be mindful of other serious complications of PONV and so perhaps are less likely to place importance on severe nausea itself.

This study used a range of psychometric techniques to identify clinically important PONV. Content validity was achieved through literature review and consultation with experienced clinicians. The evidence of construct validity was strong, with the PONV Intensity Scale correlating with higher Apfel scores³ and having more severe and clinically important PONV. Inter-dimension correlations with factors known to be associated with PONV morbidity were consistently positive. Patients with clinically important PONV required more antiemetic therapy, had higher rates of complications associated with severe PONV, and took longer to recover from their surgery. Clinically important PONV, as determined by the PONV Intensity Scale, was associated with a poor quality of recovery.^23,33 The 18-point difference in the QoR-40 in those with and without clinically important PONV reflects a major difference in the quality of recovery, being comparable with the change in a patient's health status that occurs from immediately after cardiac surgery to full recovery at about 3 months.³³

Reliability and responsiveness were also confirmed. The reliability coefficients of the PONV Intensity Scale exceeded the established recommendations,²⁰ indicating that the scale can provide reliable assessment for both group and individual measurements, comparisons, or both. The strength of the PONV Intensity Scale to yield consistent responses when different raters are used effectively demonstrates the ease of use and clinical utility of the scale. To overcome difficulties in measuring responsiveness, we approached it in a number of ways, each adding strength to the ability of the scale to detect an important clinical change. We found that a reduction in the PONV Intensity Scale was accompanied by a significant reduction in antiemetic dosage and patient self-ratings of improvement. The effect size was large (0.82), illustrating that a clinically important change can readily be determined by the scale.

During the validation phase of the study we revised the PONV Intensity Scale to simplify the process by which a final score is obtained and to eliminate the need to make clinical decisions regarding severity. Furthermore, it was broken down into three time frames, 0–6 h, 6–24 h, and 24–72 h, to allow for specific PONV research in the future; each time frame can be used separately or combined to calculate a final scale. We then further evaluated the revised version and found it to be essentially interchangeable with the original PONV Intensity Scale. We believe that the revised scale was simpler to use, being less susceptible to misinterpretation or uncertainty. The PONV Intensity Scale can be readily modified into a questionnaire format for patients to complete at home, or by telephone interview.

In both phases of the study we found that about 18% of those with PONV could be classified as having clinically important PONV. That is, many episodes of PONV are unimportant (to patients and clinicians). We do not want to diminish the potential significance of PONV in perioperative practice, in fact we would emphasize that clinically important PONV deserves the same attention as other serious postoperative complications, and it is this that requires more stringent management than currently implemented.

There have been previous attempts to create a scale that could be used to rate the severity of emesis in patients undergoing chemotherapy,^{14,15,34–37} but none were developed for use in the postoperative setting. None of these measurement tools sought patient opinions as to what constitutes clinically important nausea and vomiting.

It was expected that patients with clinically important PONV would have higher VAS scores. We found that the nausea VAS₇₅, in particular, had good results on validity testing, but these were not as strong as for the PONV Intensity Scale. We therefore propose the VAS₇₅ as a screening tool in the clinical setting for rapid assessment or audit tool, and the PONV Intensity Scale is recommended for detailed research.

In our study, 70% of patients had three or more risk factors but only 35% received double- or triple-antiemetic prophylaxis. There are several reasons as to why this occurred. Anecdotally, there is a lack of preoperative identification of risk factors. Under-treatment also may be because of belief by the anaesthetist that PONV is not so important. It is hoped that a clearer understanding of what is clinically important PONV, and its risk prediction, will go a long way to redress any under-treatment issues in the future.

In summary, we used psychometric techniques to prospectively validate and test the reliability and responsiveness of the PONV Intensity Scale in a broad surgical setting. The PONV Intensity Scale can be used to identify clinically important PONV.

Supplementary material

Supplementary material is available at British Journal of Anaesthesia online.

Funding

The Alfred Hospital Research Trust, Melbourne, Victoria, Australia. Dr Myles is the recipient of an Australian National Health and Medical Research Council (NHMRC) Practitioner's Fellowship, Canberra, Australian Capitol Territory, Australia; and is supported by the NHMRC Clinical Research Excellence in Therapeutics (ID 219284), Monash University, Melbourne, Victoria, Australia.

Acknowledgements

The authors gratefully acknowledge the assistance of Ms Sophie Wallace, MPH, Research Manager in the Department of Anaesthesia and Perioperative Medicine, Alfred Hospital, Melbourne, Australia; and the cooperation of their anaesthetic, surgical, and nursing colleagues in the conduct of this study.

References

© The Author [2009]. Published by Oxford University Press on behalf of the British Journal of Anaesthesia. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournal.org

Issue Section:

Clinical Practice

Citing articles via

More from Oxford Academic