[Meta] the FLOW rule is doing its job and people should stop complaining
Putting this to the board: the FLOW rule is doing its job and people should stop complaining.
Why comparing across trials almost never works, with the specific failure modes.
Different populations: an obesity programme and a diabetes programme enrol different people with different baseline characteristics. Different endpoints: body weight change, glycaemic control and cardiovascular events are not convertible. Different durations: 68 weeks and 72 weeks are not the same, and the curves have not flattened by either.
Different analysis populations: one paper reports intention-to-treat, another emphasises completers. Different support: some trial designs include structured lifestyle contact that no member of this board receives.
Stack those and the "X beats Y" tables that circulate here are comparing five things at once and attributing the difference to the molecule.
How to read one of these papers in fifteen minutes, in the order that actually helps.
Start with the registered protocol and check the primary endpoint against what is reported. Then the methods: who was included, what the comparator was, how long the randomised phase ran. Then the discontinuation numbers, which are a tolerability result and are usually in a supplementary table.
Only then the efficacy figure, and read the interval rather than the point estimate. Finish with the limitations section, which is where the authors say what they actually think.
Fifteen minutes, and you will know more than any thread summarising it.
Argued for a week about a result and then read the limitations section, which conceded most of my opponent’s point.
Corrections welcome, especially the pedantic ones. Pedantry is how this board earns its reputation.
best — the order this archive was captured in
A confidence interval is the range of effects compatible with the data. Two trials with overlapping intervals have not disagreed, whatever their point estimates look like next to each other.
Agreed. Half the arguments on this site are two people quoting different trials at each other without noticing.
Yes. The appendix tables are where the subgroup and the adverse event detail actually live.
the confidence interval is the finding, the point estimate is the headline
a press release is not a publication
Quoted a figure here confidently, got asked whether it was ITT, went and checked, and it was not. Learned something.
Spent an evening with the appendix tables and found the subgroup detail that the entire thread had been speculating about.
Relative risk reduction without the baseline rate is uninterpretable. A large relative reduction on a small absolute risk is a small absolute benefit.
That is a relative risk reduction. Quoting it without the absolute numbers overstates the case considerably.
Right — trial participants get structured support. Comparing yourself to a trial mean is comparing across two different interventions.
Open-label extensions lose their randomisation. Anybody still enrolled at week 104 is a selected group and the numbers describe that group.
Compared myself to a trial mean for about six months before realising the trial arm had dietitian contact every fortnight.
a mean is not a promise
intention to treat versus completers changes the number substantially
trial populations get support that nobody on this board gets
Correcting myself upthread: I gave the completer figure and labelled it intention-to-treat.
The registered protocol is public. Comparing the registered primary endpoint with the reported one is a two-minute check and it is how outcome switching gets caught.
Same. A press release is a claim about a result; the publication is the result.
Careful with that mean. The distribution around it was wide enough that it describes very few individual participants.
Absolute or relative risk reduction?
This. Intention-to-treat versus completer analysis routinely moves the headline by several points.
Press-release posts get their own flair here. Nothing wrong with them, they are just a different kind of claim.
Do you have the publication or the press release?
Agreed on comparators. "Superior" means nothing until you know superior to what and at what dose.
Disagree — that figure is from the diabetes programme and you are quoting it as an obesity endpoint.
Correction: SURMOUNT is the obesity programme and SURPASS is the diabetes one. The figure you quoted belongs to the other one.
The discontinuation numbers were the most useful thing in the paper for me and they were in a supplementary table.
The discontinuation numbers were the most useful thing in the paper for me and they were in a supplementary table.
formulary_fighter is right about the programme names. They are different populations with different endpoints.
open-label extensions are not the same evidence as the randomised phase
- 1Open-label extensions lose their randomisation. Anybody still enrolled at…7 comments in this branch · started by u/ahmed_iyer