Effectiveness of ChatGPT in Identifying and Accurately Guiding Patients in Rhinoplasty Complications

Overview

Complications after rhinoplasty need prompt attention, and patients often have to judge for themselves whether something warrants a call. The authors tested whether a language model could make that triage judgment. Different complication scenarios were entered into ChatGPT, and its responses were sorted into two categories: contact surgeon, or follow postoperative instructions.

Sections of note

• Design: scenario testing of a language model. No patients were involved.
• Responses were categorized into contact surgeon or follow postoperative instructions.
• For infection, ChatGPT consistently advised immediate contact with the surgeon.
• For other complications, it recommended monitoring and adhering to instructions, while suggesting surgeon contact if concerns persisted.
• The abstract does not state how many scenarios were tested or what they were.
• The abstract does not state who judged the responses or against what standard.
• No accuracy rate, error rate or expert comparison is reported.

What it means for a patient

• The model directed patients toward the surgeon for infection, which is the complication where delay carries the most risk.
• For everything else it defaulted to watchful waiting with an escalation option, which is a conservative but non-specific position.
• No numbers are reported. Without knowing how many scenarios were tested or how correctness was judged, the reliability of these responses cannot be assessed.
• A language model cannot examine a nose. Any postoperative concern still needs a clinician.

Why this paper matters

Patients recovering from surgery increasingly ask software before calling a clinic, so what these tools say about complications matters regardless of whether they were designed for it. Testing that behavior systematically is the first step. Whether the model misses dangerous presentations, which is the risk that counts, is not established by this abstract.

Terms

• Large language model: software trained on text that generates human like written responses.
• ChatGPT: a publicly available conversational large language model.
• Postoperative complication: a problem arising after surgery, such as infection or bleeding.
• Triage: deciding how urgently a problem needs attention.
• Scenario testing: presenting a system with constructed situations to observe its responses.