A few years ago I pulled myself off an economic evaluation project. The management consultant leading the analysis kept pushing to include costs that don't belong in a standard economic evaluation, not because anyone made a math error, but because leaving them out made the final number less newsworthy. I have no idea what number they landed on, but I promise you it was bigger than it should have been.

That's not an isolated occurrence; I see a version of it constantly in GLP-1 research right now. A "cost benefit analysis" will show dramatic "savings," and if you actually look at what's in the model, the cost of the drug itself isn't included. Of course it looks conveniently cost-saving.

The opposite move happens too, just in the other direction: a scope so broad that it's not meaningful. Technically, best practice says a reference case analysis should use a lifetime horizon and a societal perspective, and it should, that's not wrong. But here's the thing: no single US payer is covering anyone for their whole life. People switch jobs, switch plans, age into Medicare, and so on. Decision makers also need a second, more honest analysis using a more realistic time horizon. And as GLP-1 indications keep expanding, the question of what even counts as a relevant cost or saving needs the same level of scrutiny.

(Slight tangent: some decision analytic models of GLP-1 discontinuation simulate the following handoff: patient stops taking the GLP-1, patient starts lifestyle treatment, like a baton pass. That's not how it works. Most people taking GLP-1s are already simultaneously engaged in lifestyle treatment. GLP-1s are a tool to support adherence to behavioral interventions, not a magic pill. The only transition happening is a transition off the drug. Fix your model.)

It also drives me a little crazy when people cite the CHEERS checklist as proof that a cost-effectiveness analysis was done well. In reality, CHEERS is a reporting checklist that tells you whether the authors described their methods clearly, not whether those methods were any good. You can report a bad study perfectly.

This isn't limited to GLP-1 research. During the pandemic, my colleague and I organized a journal club around a paper that had gotten a ton of media attention for modeling life expectancy lost from COVID-19 school closures. Big, scary, very shareable number. Also, seemingly a wrong number in my opinion. A few of us—people who actually do decision analytic modeling for a living—sat down and reviewed the manuscript with about 200 attendees at our journal club, debating the merits (some) and pitfalls (many) of the analysis, how to interpret the findings, and how certain (choice) assumptions the analysis made could meaningfully change the findings. I'm pretty sure (and proud) that this is the first time a journal club was referenced in an NPR piece. 

Why does this keep happening and what should we do about it?

The fix isn't complicated: report the reference case and outcomes to support realistic decision making, disclose what's excluded as clearly as what's included, and stop treating a reporting checklist as a seal of quality. None of that requires new methods. It requires demanding better from analysts (and peer review) and promoting the honest number instead of the big one, in addition to acknowledging uncertainty. The good actors aren't always the loudest people in the room, and a quiet, honest finding is never going to out-headline a made up sum of money.

I joined Dr. David Hill and Dr. Joanna Parga-Belinkie on Pediatrics on Call to talk through what's shifted in child and adolescent obesity treatment since the AAP's 2023 guidelines, which recommend obesity pharmacotherapy with medications like GLP-1s earlier and more broadly than a lot of families and even some clinicians expect.

Here's the thing I tried to convery in that conversation: none of this is a simple protocol you hand someone. There's no universal right answer for a given kid, because the right answer depends on that family's actual goals, constraints, resources, and values, not a checklist or chart. A pediatrician's job isn't "tell parents what to do." Their job is to help families recognize whether a child is a good candidate for some sort of obesity treatment program and to support families in making a decision about treatment initiation (and discontinuation for that matter) with accurate information about what the options really involve.

That's the whole reason shared decision-making exists as a concept. It's not a soft add-on to "real" medicine. It's the mechanism for handling exactly the kind of uncertainty and trade-off this field is full of.

[Listen to the episode →]

Most of what we know about GLP-1 effectiveness and utilization comes from clinical trial, EHR, and claims data; data that are automatically generated and are structured for research. If we're lucky, we may have data from patient surveys or even better, qualitative interviews, which can provide insight into how patients and providers approach decision making and collect data on patient-reported outcomes and decision satisfaction. Our aUDIO research project (Understanding Diabetes Drug Initiation and Offerings) took a different hypothesis-generating approach: what research questions can we generate if we just listened to actual conversations?

We used a database of real, recorded clinic visits (Verilogue's Point-of-Practice database with over 235,000 encounters) to study what genuinely gets discussed, and what doesn't, when GLP-1RAs and SGLT2i medications come up for patients with type 2 diabetes.

Some of what we're finding, presented this year at the Society for Medical Decision Making conference: cost and coverage get raised constantly, but often without the specifics a patient would need to actually plan around. Clinicians rarely bring up reproductive goals before prescribing to women of reproductive age, even though the effects of GLP-1s can impact fertility, fetal, and potentially reproductive health. Shared decision-making in the real-world often looks pretty different from best practice recommendations.

These conversations are fascinating and sometimes entertaining or heartbreaking.

None of this is about blaming anyone in the room. It's about the fact that if critical information isn't part of the actual conversation, no amount of good intention downstream fixes that. You can't decide well on information you were never given. We need tools to support informed, shared decision making for patients considering GLP-1s or other novel and expensive tests or treatments.

Surveys and claims data are great, but they're answering a different question. They tell you what people remember, or what got coded after the fact. Conversation data tells you what was actually said, in the room, in real time, before anyone summarized it into Qualtrics or a chart note. We need both. The gap we're pointing at is that almost nobody's been systematically analyzing the latter.

That's likely to change fast. As digital scribes and ambient documentation tools become standard in clinical practice, this kind of data (if retained) could go from something rare and resource intensive to something that is routinely captured at scale. If you're working on digital scribes, clinical decision support, or anything touching this data, I'd like to compare notes.

I didn't learn how to tell a story through reading or a research methods class. I honed my skills by recounting my personal shenanigans at parties, then on stage in front of hundreds of people, and more formally through directed readings and storytelling workshops. These performances have absolutely nothing to do with work, but have also been the most useful tool for improving my scientific communication skills. The exact same instincts apply to presenting research: how to hook an audience, what and when to hold back, why you might want to alter the narrative for flow, and how to stick the landing (aka the key research findings). 

I look for storytelling inspiration outside of academia on purpose. Attending live storytelling shows and reading books like The Moth's How to Tell a Story allow you to observe different storytelling styles, decide what does and doesn't work well, and importantly, learn how to stay under a time limit. Going to live shows is important— podcasts are curated and edited, but you learn a ton from watching raw or bad performances. Podcasts like Radiolab and This American Life are worth studying for how they handle pacing and create a sense of curiosity, tension, and suspense. I found Masterclasses on writing and communication by Malcolm Gladwell, Aaron Sorkin, and Robin Roberts helpful. Priya Parker's The Art of Gathering changed how I think about designing a conference room, not just a talk. None of this is about research. All of it is useful for research communication.

Many research talks are frankly, boring.† Seminars often bury the interesting part within the methods section or even worse, when the method is very technical, they don't present it in a way the listener can relate to. If you're in an interdisciplinary environment, this is...not good. Such talks often follow a rigid structure (background, methods, results, conclusions), not a structure that makes a listener care. A good talk does the opposite: it finds the actual question at stake, usually a human one, finds a way to relate to the audience, and lets the data and methods answer it. There is no single winning template; the structure should be tailored toward your audience, setting, and topic. 

Here are two tests you can use:

• Imagine the projector/screen dies and you don't have any slides. Maybe you have a whiteboard and a marker to sketch out some figures. Would people sit there and listen to you? 

• Could you translate your talk into a short YouTube explainer video [Kurzgesagt - In a Nutshell→], a TikTok meme, an SNL sketch, or even a dance [Dance Your PhD→]?

Most research talks fail these tests. Mine did too, plenty of times. What you want— at the end of your talk— is somone thinking, "That was so relatable. How did they do that?" 

Next time you're giving a talk, pretend you can't use slides. Focus on telling a story for your particular audience. A story starring Bad Bunny if you're really into the SNL idea (I mean, why not?). Once you figure that out, determine which visuals you absolutely need to tell that story. Remember: you don't have to mimic anyone else, but you should invest time in finding your own public speaking style. 

Let me know if this advice works for you! I have many more strong opinions and I'm happy to brainstorm other ideas.

† Footnote: Almost all talk titles are boring too! Your first hurdle is getting people in the room. So maybe you call your financial incentives lecture "Bribes and Prejudice: [Information that actually describes your research question]." At minimum, you need something that is descriptive but also generalizable to others. For example, if I see a talk about HIV, I might not be interested as an obesity researcher. But if I knew it was presenting a discrete choice experiment, I might think I'd get something out of that seminar. Showcase the approach or common thread in addition to the topic in your talk titles.