I think that this is your best Inkhaven post so far!
The idea feels novel and I feel like your personality comes out a bit more, which can make reading more enjoyable.
I saw that someone said one of your earlier posts felt like LLM writing and—while I didn’t think so—I could see why they said that. Your writing is always well structured, which makes it easy to follow, yet also can seem similar to LLM writing. This post lacks the listicle feel that I believe is what the other commenter was picking up on as LLM vibes, yet keeps the organization!
I think that “Future Work” bullet 3 could be its own post. There was a lot of distrust about the COVID vaccine(s) caused by how “rushed” people felt they were. Unfortunately, on a public relations level, feelings may matter significantly more than data….
This seems sensible if Phase 1 and 2 trials were properly run. Unfortunately I don't think they are (see one of the finalists in the ACX 2025 not-a-book review competition), and the current system seems to rely on Phase 3 to pick up most serious side effects. Maybe the anecdata of a single essay is not good evidence but I would be worried if we dropped Phase 3 trials without overhauling Phase 1 and 2 incentives which seem to discourage reporting side effects.
The question is one of cost-benefit: is the potential of rare side effects one can spot in Phase 3 trials worth the cost of delaying rollout by months or years?
Maybe if the human cost from malaria, the next pandemic, etc, is hundreds of lives, but not if the cost is tens of thousands of lives or hundreds of thousands.
Keep in mind that in my proposal you're conducting ongoing trials *during* the rollout, so you could always stop vaccine distribution if and when the costs are seen to outweigh the benefits.
I'm saying that the incentives (and anecdotal evidence) means that side effects and lack of efficacy are now much less likely to be picked up in Phase 1/2 anymore, because participants are paid for taking part so they have no reason to report any but the most life-threatening effects or lack of efficacy. Essentially, we are moving to a situation where Phase 3 is the only real trial. Now you are suggesting dropping that, based on a probably correct analysis of how Phase 1/2 should *in theory* be enough. In practice I don't think they are.
It might make sense to do adaptive adoption (skip all trials, instead monitor those getting the first few treatments intensively including acceptance of the first few fatalities as part of the learning process, with new statistical techniques to maximize the information extracted from the early experience of treatment) but you don't seem to be arguing for that.
I agree with the premise, if things stay as they do now. But I don't think it takes into perspective the effect this change in policy will have downstream on the parties involved, and might lead to the cost-benefit calculation shifting backwards. Vaccines are already not trusted by too many people to sometimes gain herd immunity. What you are doing now is reducing the average safety and efficacy of vaccines that hit the market and are used by the common people, lowering their reputation even more. I personally dont think the reason we do phase 3 trials is a net-benefit calculation based on deaths/rewards, but its to keep the (small level of existing) trustworthiness in the pharmaceutical industry. There would have to be extreme reasons (see COVID) to make people deem it worthy, and even they did Phase 3. Furthermore, phase 3 costs a lot of money. Letting pharmaceutical companies skip it might change the incentive in ways where we will get worse vaccines, because all you need to do is get past phase 1 and 2 to start making money. It might be that we will get the pharmaceuticals industry's version of a "replication crisis".
I think that this is your best Inkhaven post so far!
The idea feels novel and I feel like your personality comes out a bit more, which can make reading more enjoyable.
I saw that someone said one of your earlier posts felt like LLM writing and—while I didn’t think so—I could see why they said that. Your writing is always well structured, which makes it easy to follow, yet also can seem similar to LLM writing. This post lacks the listicle feel that I believe is what the other commenter was picking up on as LLM vibes, yet keeps the organization!
I think that “Future Work” bullet 3 could be its own post. There was a lot of distrust about the COVID vaccine(s) caused by how “rushed” people felt they were. Unfortunately, on a public relations level, feelings may matter significantly more than data….
This seems sensible if Phase 1 and 2 trials were properly run. Unfortunately I don't think they are (see one of the finalists in the ACX 2025 not-a-book review competition), and the current system seems to rely on Phase 3 to pick up most serious side effects. Maybe the anecdata of a single essay is not good evidence but I would be worried if we dropped Phase 3 trials without overhauling Phase 1 and 2 incentives which seem to discourage reporting side effects.
The question is one of cost-benefit: is the potential of rare side effects one can spot in Phase 3 trials worth the cost of delaying rollout by months or years?
Maybe if the human cost from malaria, the next pandemic, etc, is hundreds of lives, but not if the cost is tens of thousands of lives or hundreds of thousands.
Keep in mind that in my proposal you're conducting ongoing trials *during* the rollout, so you could always stop vaccine distribution if and when the costs are seen to outweigh the benefits.
I'm saying that the incentives (and anecdotal evidence) means that side effects and lack of efficacy are now much less likely to be picked up in Phase 1/2 anymore, because participants are paid for taking part so they have no reason to report any but the most life-threatening effects or lack of efficacy. Essentially, we are moving to a situation where Phase 3 is the only real trial. Now you are suggesting dropping that, based on a probably correct analysis of how Phase 1/2 should *in theory* be enough. In practice I don't think they are.
It might make sense to do adaptive adoption (skip all trials, instead monitor those getting the first few treatments intensively including acceptance of the first few fatalities as part of the learning process, with new statistical techniques to maximize the information extracted from the early experience of treatment) but you don't seem to be arguing for that.
I agree with the premise, if things stay as they do now. But I don't think it takes into perspective the effect this change in policy will have downstream on the parties involved, and might lead to the cost-benefit calculation shifting backwards. Vaccines are already not trusted by too many people to sometimes gain herd immunity. What you are doing now is reducing the average safety and efficacy of vaccines that hit the market and are used by the common people, lowering their reputation even more. I personally dont think the reason we do phase 3 trials is a net-benefit calculation based on deaths/rewards, but its to keep the (small level of existing) trustworthiness in the pharmaceutical industry. There would have to be extreme reasons (see COVID) to make people deem it worthy, and even they did Phase 3. Furthermore, phase 3 costs a lot of money. Letting pharmaceutical companies skip it might change the incentive in ways where we will get worse vaccines, because all you need to do is get past phase 1 and 2 to start making money. It might be that we will get the pharmaceuticals industry's version of a "replication crisis".