We're waiting for the big leap in the next AlphaGenome capabilities from Latysheva and the team! Curious to know if widespread use of agents will translate to faster progress in genomics
2) The peer review process is indeed a mess, but we may need to think about 3 possible tracks as solutions. Track 1 review is AI free all the way. Violators receive a 1 year ban for the first misdeed. A second violation merits a lifetime ban from the venue. Track 2 review is AI disclosure on both ends. Track 3 review is AI only review; that is, AI serves as both author and reviewer. Journals/funders/editors/authors may need to select which track is applicable under a given circumstance or pick a lane.
I think you're right that we should think about different tracks for evolving peer review. It seems like these questions are also downstream of the wider question about how science artefacts themselves should evolve in the AI era. I think the paper (and conventional peer review) has a lot going for it, despite its limitations, so we should be careful here. But it seems clear that we will have some new kinds of artefacts - e..g like 'skills' - that will need new forms of peer review. It's very hard to predict what is a more ephemeral vs permanent change, but I think we'll definitely need some new ideas here.
e.g. I thought Gavin Leech's idea of a journal for AI-asssisted proofs was interesting.
Missing the most obvious necessary things for the validation bottleneck:
1) Need to build career paths and incentives for researchers in validation. Right now peer review operates on goodwill. No wonder it is creaking.
2) More human researchers to do validation, auto-validation is where the technology is weakest is many important fields. Don’t get distracted by the value of auto-validation in mathematics and mathematically formalised fields, it does not translate across to more complex fields. Of course we may get huge technological breakthroughs in AI, but we should also respect the fundamental constraints of the technology as it stands.
Thanks Indy. On 2, that is the point that we are making here. We're not suggesting that auto-validation in maths will work across fields, and instead suggest investing more in experimental infrastructure to help with validation needs. That said, the idea that other fields have nothing to learn from how AI verification is progressing in maths, including in natural language maths, doesn't seem quite right?
We're waiting for the big leap in the next AlphaGenome capabilities from Latysheva and the team! Curious to know if widespread use of agents will translate to faster progress in genomics
This was a really informative read. I only have two comments:
1) Not only do models need epistemic humility, but AI researchers do, too. This means accepting that priors might be wrong and non-Western worldviews are equally valid. I wrote about the need for such an approach in AI & Society: https://idp.springer.com/authorize?response_type=cookie&client_id=springerlink&redirect_uri=https%3A%2F%2Flink.springer.com%2Farticle%2F10.1007%2Fs00146-023-01708-y
2) The peer review process is indeed a mess, but we may need to think about 3 possible tracks as solutions. Track 1 review is AI free all the way. Violators receive a 1 year ban for the first misdeed. A second violation merits a lifetime ban from the venue. Track 2 review is AI disclosure on both ends. Track 3 review is AI only review; that is, AI serves as both author and reviewer. Journals/funders/editors/authors may need to select which track is applicable under a given circumstance or pick a lane.
Thanks again!
Thanks for reading Josh.
I think you're right that we should think about different tracks for evolving peer review. It seems like these questions are also downstream of the wider question about how science artefacts themselves should evolve in the AI era. I think the paper (and conventional peer review) has a lot going for it, despite its limitations, so we should be careful here. But it seems clear that we will have some new kinds of artefacts - e..g like 'skills' - that will need new forms of peer review. It's very hard to predict what is a more ephemeral vs permanent change, but I think we'll definitely need some new ideas here.
e.g. I thought Gavin Leech's idea of a journal for AI-asssisted proofs was interesting.
https://docs.google.com/document/d/1XUBjP75Fl-W3qo2NZa8bMJV-UJY5vt6LJFHcqQaV3CI/edit?tab=t.0
From his newsletter here
https://gavin-leech.beehiiv.com/p/13-collywobbles
Missing the most obvious necessary things for the validation bottleneck:
1) Need to build career paths and incentives for researchers in validation. Right now peer review operates on goodwill. No wonder it is creaking.
2) More human researchers to do validation, auto-validation is where the technology is weakest is many important fields. Don’t get distracted by the value of auto-validation in mathematics and mathematically formalised fields, it does not translate across to more complex fields. Of course we may get huge technological breakthroughs in AI, but we should also respect the fundamental constraints of the technology as it stands.
Thanks Indy. On 2, that is the point that we are making here. We're not suggesting that auto-validation in maths will work across fields, and instead suggest investing more in experimental infrastructure to help with validation needs. That said, the idea that other fields have nothing to learn from how AI verification is progressing in maths, including in natural language maths, doesn't seem quite right?