Matthew Yates
@allomedna.bsky.social
Molecular Ecologist and Conservation Geneticist. I work a lot with fish, eDNA, and eRNA - with a soft spot for salmonids (very original, I know...)
Ludicrous. Absolutely ludicrous, especially coming from such a respected institution.
April 27, 2025 at 6:06 AM
Ludicrous. Absolutely ludicrous, especially coming from such a respected institution.
I’m honestly flabbergasted that this even occurred. The idea of an LLM acting as a SA survivor or a trained crisis councillor and then giving real people advice without them knowing the origin of that advice? Worse, being actively deceived about the origin of that advice?
April 27, 2025 at 6:06 AM
I’m honestly flabbergasted that this even occurred. The idea of an LLM acting as a SA survivor or a trained crisis councillor and then giving real people advice without them knowing the origin of that advice? Worse, being actively deceived about the origin of that advice?
BUT EVEN IF that weren’t true, this is a gross violation of informed consent, which is a component of research ethics that is *extremely important*.
April 27, 2025 at 6:06 AM
BUT EVEN IF that weren’t true, this is a gross violation of informed consent, which is a component of research ethics that is *extremely important*.
Before anyone tries to justify this on utilitarian grounds, the research is of dubious quality because Reddit is so riddled with bots that a substantial amount of their LLM interactions could have been bots talking to bots. So the data are fundamentally flawed from the get-go.
April 27, 2025 at 6:06 AM
Before anyone tries to justify this on utilitarian grounds, the research is of dubious quality because Reddit is so riddled with bots that a substantial amount of their LLM interactions could have been bots talking to bots. So the data are fundamentally flawed from the get-go.
How the hell the University ethics review process let this go is beyond me, but apparently the university is defending it?
April 27, 2025 at 6:06 AM
How the hell the University ethics review process let this go is beyond me, but apparently the university is defending it?
Nobody in the online community had any idea this was going on. The community also has mechanisms to score whether users were ‘convinced’ by the arguments of a poster (in this case AI). The LLM had more than 100 such comments that received this score. These are (theoretically, at least) real people.
April 27, 2025 at 6:06 AM
Nobody in the online community had any idea this was going on. The community also has mechanisms to score whether users were ‘convinced’ by the arguments of a poster (in this case AI). The LLM had more than 100 such comments that received this score. These are (theoretically, at least) real people.
See:
www.reddit.com/r/changemyvi...
The researchers had the LLMs pose as users with expertise or experience in various subject areas (including SA) who then provided advice or arguments to users posting in the subreddit.
www.reddit.com/r/changemyvi...
The researchers had the LLMs pose as users with expertise or experience in various subject areas (including SA) who then provided advice or arguments to users posting in the subreddit.
META: Unauthorized Experiment on CMV Involving AI-generated Comments
www.reddit.com
April 27, 2025 at 6:06 AM
See:
www.reddit.com/r/changemyvi...
The researchers had the LLMs pose as users with expertise or experience in various subject areas (including SA) who then provided advice or arguments to users posting in the subreddit.
www.reddit.com/r/changemyvi...
The researchers had the LLMs pose as users with expertise or experience in various subject areas (including SA) who then provided advice or arguments to users posting in the subreddit.
Edit: Post 6 should say 'note that you take the mean AFTER scaling with 'b'
March 12, 2025 at 3:14 PM
Edit: Post 6 should say 'note that you take the mean AFTER scaling with 'b'
And there you have it! Pretty happy with how this turned out, was a long collaboration with lots of different folks - huge thanks to Dr. Taylor Wilcox, Dr. Shannon Kay, Dr. Pedro Peres-Neto (@comecology.bsky.social), and Dr. Daniel Heath, who all were instrumental in developing this paper!
March 12, 2025 at 3:11 PM
And there you have it! Pretty happy with how this turned out, was a long collaboration with lots of different folks - huge thanks to Dr. Taylor Wilcox, Dr. Shannon Kay, Dr. Pedro Peres-Neto (@comecology.bsky.social), and Dr. Daniel Heath, who all were instrumental in developing this paper!
20) But slope values for eDNA/biomass relationships will be higher for protists compared to bacteria due to their larger cell sizes.
March 12, 2025 at 3:08 PM
20) But slope values for eDNA/biomass relationships will be higher for protists compared to bacteria due to their larger cell sizes.
19) Amongst single-cellular organisms, ‘b’ will equal 0 since we expect eDNA to scale with numerical abundance (each cell contains ~approximately~ the same number of gene copies).
March 12, 2025 at 3:08 PM
19) Amongst single-cellular organisms, ‘b’ will equal 0 since we expect eDNA to scale with numerical abundance (each cell contains ~approximately~ the same number of gene copies).
18) Compared to macro-organisms, eDNA/biomass slopes will be very low since capturing organisms whole produces lots of eDNA.
March 12, 2025 at 3:08 PM
18) Compared to macro-organisms, eDNA/biomass slopes will be very low since capturing organisms whole produces lots of eDNA.
17) Amongst multicellular microorganisms (e.g., plankton, microinvertebrates) ‘b’ will equal 1, as we expect eDNA to scale with biomass due to most eDNA being derived from capturing organisms whole-body.
March 12, 2025 at 3:08 PM
17) Amongst multicellular microorganisms (e.g., plankton, microinvertebrates) ‘b’ will equal 1, as we expect eDNA to scale with biomass due to most eDNA being derived from capturing organisms whole-body.
16) Plants and turtles? Hard external surfaces and low metabolism -> high eDNA/biomass slopes compared to metabolically active groups like fish.
March 12, 2025 at 3:08 PM
16) Plants and turtles? Hard external surfaces and low metabolism -> high eDNA/biomass slopes compared to metabolically active groups like fish.
15) Amongst macro-organisms ‘b’ will likely be close to ~0.75, and eDNA/biomass regression slopes are likely to be predominantly determined by metabolics and surface area characteristics.
March 12, 2025 at 3:08 PM
15) Amongst macro-organisms ‘b’ will likely be close to ~0.75, and eDNA/biomass regression slopes are likely to be predominantly determined by metabolics and surface area characteristics.
14) More broadly, the framework also lays out a foundation for modelling these relationships across taxonomic groups; we can thus make predictions about what the relative values of the beta coefficients in the relationship and the ‘b’ scaling parameters will look like across taxonomic groups.
March 12, 2025 at 3:08 PM
14) More broadly, the framework also lays out a foundation for modelling these relationships across taxonomic groups; we can thus make predictions about what the relative values of the beta coefficients in the relationship and the ‘b’ scaling parameters will look like across taxonomic groups.
13) If only one of those relationships produces a good regression, then that means you probably need to directly try to estimate the value of ‘b’ in your system!
March 12, 2025 at 3:08 PM
13) If only one of those relationships produces a good regression, then that means you probably need to directly try to estimate the value of ‘b’ in your system!
12) If you correlate eDNA with biomass, you are assuming a ‘b’ of 1 -> that inherently means that eDNA divided by mean population mass should correlate with organism abundance. If it doesn’t, you have a problem!
March 12, 2025 at 3:08 PM
12) If you correlate eDNA with biomass, you are assuming a ‘b’ of 1 -> that inherently means that eDNA divided by mean population mass should correlate with organism abundance. If it doesn’t, you have a problem!
11) If you correlate eDNA with N, you are assuming a ‘b’ of 0 -> that inherently means that eDNA multiplied by mean population mass should correlate well with biomass. If it doesn’t, you have a problem!
March 12, 2025 at 3:08 PM
11) If you correlate eDNA with N, you are assuming a ‘b’ of 0 -> that inherently means that eDNA multiplied by mean population mass should correlate well with biomass. If it doesn’t, you have a problem!
10) This is what most studies have done – previous researchers typically presented the ‘best-fit’ relationship for either eDNA/biomass or eDNA/abundance. We show that either assumption REQUIRES that adjusted eDNA should reflect the other variable – you should always present BOTH relationships
March 12, 2025 at 3:08 PM
10) This is what most studies have done – previous researchers typically presented the ‘best-fit’ relationship for either eDNA/biomass or eDNA/abundance. We show that either assumption REQUIRES that adjusted eDNA should reflect the other variable – you should always present BOTH relationships
9) Key take-home: Assuming eDNA correlates with numerical abundance assumes an allometric scaling coefficient of 0, and assuming eDNA correlates with biomass assumes an allometric scaling coefficient of 1.
March 12, 2025 at 3:08 PM
9) Key take-home: Assuming eDNA correlates with numerical abundance assumes an allometric scaling coefficient of 0, and assuming eDNA correlates with biomass assumes an allometric scaling coefficient of 1.