Henrik Karlstrøm
@karlstrom.bsky.social
Bibliometrician, data scientist at Nordic Institute for Studies of innovation, research and education, nifu.no
Also not sure that other ways of evaluation necessarily pose a problem to smaller institutions or non-polytechnicals. I don't think the UK is uniquely great at protecting such institutions.
February 10, 2026 at 12:14 PM
Also not sure that other ways of evaluation necessarily pose a problem to smaller institutions or non-polytechnicals. I don't think the UK is uniquely great at protecting such institutions.
I think it is interesting to think of the REF in relation to other systems of evaluation around the world - it is pretty singular in scope and amount of resources spent. One could ask "would I recommend other countries change to a REF-like system?" and I'm not sure I would.
February 10, 2026 at 12:13 PM
I think it is interesting to think of the REF in relation to other systems of evaluation around the world - it is pretty singular in scope and amount of resources spent. One could ask "would I recommend other countries change to a REF-like system?" and I'm not sure I would.
New preprint out, about the feasibility of using LLMs for internal reviews (for example in pre-REF institutional exercises). The results are a mixed bag: LLMs match the score well, but it seems to be mostly guesswork, and the LLM reports are more generic and less informative than human written ones.
Large Language Models for Departmental Expert Review Quality Scores
Presumably, peer reviewers and Large Language Models (LLMs) do very different things when asked to assess research. Still, recent evidence has shown that LLMs have a moderate ability to predict qualit...
arxiv.org
February 4, 2026 at 3:31 PM
New preprint out, about the feasibility of using LLMs for internal reviews (for example in pre-REF institutional exercises). The results are a mixed bag: LLMs match the score well, but it seems to be mostly guesswork, and the LLM reports are more generic and less informative than human written ones.
Mine nyanserende sitater kommer nederst i denne saken, bak hun som ikke har lest rapporten vi har skrevet. Det må være fordi jeg mangler et bra "se strengt inn i kamera"-profilbilde.
Norge samarbeider mest med Kina om KI: — Ikke greit i det hele tatt
Norske KI-forskeres fremste samarbeidsland er Kina. Slik hjelper Norge indirekte Russland i Ukraina-krigen, mener Ståle Ulriksen ved Sjøkrigsskolen.
www.khrono.no
January 29, 2026 at 11:39 AM
Mine nyanserende sitater kommer nederst i denne saken, bak hun som ikke har lest rapporten vi har skrevet. Det må være fordi jeg mangler et bra "se strengt inn i kamera"-profilbilde.
Reposted by Henrik Karlstrøm
RE: https://social.cwts.nl/@vtraag/111917750301769426
Happy to say that our introduction to structural causal models in science studies was just published in QSS:
https://doi.org/10.1162/QSS.a.405.
Happy to say that our introduction to structural causal models in science studies was just published in QSS:
https://doi.org/10.1162/QSS.a.405.
social.cwts.nl
January 15, 2026 at 5:30 PM
RE: https://social.cwts.nl/@vtraag/111917750301769426
Happy to say that our introduction to structural causal models in science studies was just published in QSS:
https://doi.org/10.1162/QSS.a.405.
Happy to say that our introduction to structural causal models in science studies was just published in QSS:
https://doi.org/10.1162/QSS.a.405.
We have an analogous problem in bibliometrics when constructing citation networks - approving or supporting citations are treated the same way as references to work being criticised, all flattened into a "flow of influence"-like graph.
July 10, 2025 at 8:46 AM
We have an analogous problem in bibliometrics when constructing citation networks - approving or supporting citations are treated the same way as references to work being criticised, all flattened into a "flow of influence"-like graph.
Reposted by Henrik Karlstrøm
Registration for this year's Nordic Workshop on Bibliometrics and Research Policy is now OPEN: www.nifu.no/nwb2025/regi....
Hope to see you there in October!
#nwb2025
Hope to see you there in October!
#nwb2025
Registration
NWB2025Practical informationProgramRegistrationAbout NWB
Registration for NWB 2025 is now open.
Please register for the workshop using the registration form: https://form.app.uib.no/NWB2025. Not...
www.nifu.no
July 1, 2025 at 8:50 AM
Registration for this year's Nordic Workshop on Bibliometrics and Research Policy is now OPEN: www.nifu.no/nwb2025/regi....
Hope to see you there in October!
#nwb2025
Hope to see you there in October!
#nwb2025
Reposted by Henrik Karlstrøm
The 30th Nordic Workshop on Bibliometrics and Research Policy takes place in Bergen, Norway October 29-31 this year. The Call for Abstracts is now out, and submissions open March 17th. Presentations and posters alike are welcome. Hope to see you there?
Welcome to Bergen in 2025!
Welcome to the 30th Nordic Workshop on Bibliometrics and Research Policy (NWB 2025). Nordic Institute for Studies of Innovation, Research and Education are hosts together with the University Library o...
www.nifu.no
March 13, 2025 at 11:25 AM
The 30th Nordic Workshop on Bibliometrics and Research Policy takes place in Bergen, Norway October 29-31 this year. The Call for Abstracts is now out, and submissions open March 17th. Presentations and posters alike are welcome. Hope to see you there?
New article out in London Review of Education. It's sort of a conceptual piece about uses and non-uses of machine learning / AI in systematic reviews using the field of education as a lense. It's diamond open access, so give it a read if you're interested.
Uses of artificial intelligence and machine learning in systematic reviews of education research
The speed and volume of scientific publishing is accelerating, both in terms of number of authors and in terms of the number of publications by each author. At the same time, the demand for knowledge ...
doi.org
December 4, 2024 at 8:19 AM
New article out in London Review of Education. It's sort of a conceptual piece about uses and non-uses of machine learning / AI in systematic reviews using the field of education as a lense. It's diamond open access, so give it a read if you're interested.
Kan han bli det første mennesket som tweeter tjuefem timer i døgnet?
November 22, 2024 at 9:07 AM
Kan han bli det første mennesket som tweeter tjuefem timer i døgnet?
Ceremony to welcome new users to Bluesky
November 22, 2024 at 12:54 AM
Ceremony to welcome new users to Bluesky
Wonder about the experience of being interviewed by an AI - have they transcended the uncanny valley for this sort of stuff? Does it lead to different answers when you know you're not being judged by the person on the other end?
Really interesting work. I would not have predicted that interview text + RAG was sufficient to simulate survey responses.
Also fun: how do you run 2hr interviews for 1000 people? They created an AI interviewer.
Also fun: how do you run 2hr interviews for 1000 people? They created an AI interviewer.
Simulating human behavior with AI agents promises a testbed for policy and the social sciences. We interviewed 1,000 people for two hours each to create generative agents of them. These agents replicate their source individuals’ attitudes and behaviors. 🧵
arxiv.org/abs/2411.10109
arxiv.org/abs/2411.10109
November 19, 2024 at 10:15 AM
Wonder about the experience of being interviewed by an AI - have they transcended the uncanny valley for this sort of stuff? Does it lead to different answers when you know you're not being judged by the person on the other end?
They're very common? Here's a chart all sports fans can understand
November 14, 2024 at 3:56 PM
They're very common? Here's a chart all sports fans can understand
Reposted by Henrik Karlstrøm
TIL bees get drunk (but wasps don't): "Bees also show pronounced dose-dependent ethanol-induced behavioral alterations such as reduced walking time, decreased flying frequency, and loss of postural control as indicated by an increased amount of time inebriated bees spent upside down" 🧪
How to survive enormous amounts of alcohol | PNAS
How to survive enormous amounts of alcohol
doi.org
November 13, 2024 at 10:33 AM
TIL bees get drunk (but wasps don't): "Bees also show pronounced dose-dependent ethanol-induced behavioral alterations such as reduced walking time, decreased flying frequency, and loss of postural control as indicated by an increased amount of time inebriated bees spent upside down" 🧪
I guess "lax reviewing standards" only applies to Scientific Reports. Nature the journal and the other spin-offs aren't bad, they just suffer from over-generality in their disciplinary orientation - good for summative work, but not where anyone goes to get the bleeding edge work from a given field.
November 12, 2024 at 11:17 AM
I guess "lax reviewing standards" only applies to Scientific Reports. Nature the journal and the other spin-offs aren't bad, they just suffer from over-generality in their disciplinary orientation - good for summative work, but not where anyone goes to get the bleeding edge work from a given field.
Well, lax reviewing standards are only a problem when they fail to address seriously flawed papers. When the submitted paper is near perfect as is, like I'm sure your commentary was, it doesn't really factor in.
November 12, 2024 at 10:48 AM
Well, lax reviewing standards are only a problem when they fail to address seriously flawed papers. When the submitted paper is near perfect as is, like I'm sure your commentary was, it doesn't really factor in.
Er vel det som ligger bak rebrandingen til Fjotolf
November 10, 2024 at 3:11 PM
Er vel det som ligger bak rebrandingen til Fjotolf
For example, the authors fail to correct for the fact that the length of reference lists in papers is steadily increasing, so citation signals will become watered down over time unless you account for this. These things would be caught by reviewers in a disciplinary journal, but not in Nature!
The disruption index is biased by citation inflation
Abstract. A recent analysis of scientific publication and patent citation networks by Park et al. (Nature, 2023) suggests that publications and patents are becoming less disruptive over time. Here we ...
doi.org
November 9, 2024 at 10:03 AM
For example, the authors fail to correct for the fact that the length of reference lists in papers is steadily increasing, so citation signals will become watered down over time unless you account for this. These things would be caught by reviewers in a disciplinary journal, but not in Nature!
...and if I feel that way regarding my own field, my suspicion is the same goes for other fields as well. An example, the much-discussed paper about how science is becoming less innovative and disruptive over time. An interesting analysis, but the methodology raises questions.
Papers and patents are becoming less disruptive over time - Nature
A decline in disruptive science and technology over time is reported, representing a substantive shift in science and technology, which is attributed in part to the reliance on a narrower set of exist...
doi.org
November 9, 2024 at 9:58 AM
...and if I feel that way regarding my own field, my suspicion is the same goes for other fields as well. An example, the much-discussed paper about how science is becoming less innovative and disruptive over time. An interesting analysis, but the methodology raises questions.
I'm always fascinated by the high status of journals like Nature. One thing is that Scientific Reports keeps pumping out pure gibberish papers at MDPI-level rates, but the flagship journal too. I seldom feel that their bibliometric analyses would pass muster in one of our disciplinary journals.
November 9, 2024 at 9:55 AM
I'm always fascinated by the high status of journals like Nature. One thing is that Scientific Reports keeps pumping out pure gibberish papers at MDPI-level rates, but the flagship journal too. I seldom feel that their bibliometric analyses would pass muster in one of our disciplinary journals.
I could probably find out for you, I'm a bibliometrician. I'm not sure what you mean by a bifurcation in the citation patterns, though.
November 8, 2024 at 8:38 PM
I could probably find out for you, I'm a bibliometrician. I'm not sure what you mean by a bifurcation in the citation patterns, though.
This looks really interesting! But is the website down? Can't open the link in any browser.
November 5, 2024 at 8:06 AM
This looks really interesting! But is the website down? Can't open the link in any browser.
Reposted by Henrik Karlstrøm
New study by @karlstrom.bsky.social, Aksnes & Piro suggests that "the emergence of OA publishing has been particularly advantageous" in lower-income countries.
Read the article to learn more about this important topic!
#scholarlypublishing #OA
Read the article to learn more about this important topic!
#scholarlypublishing #OA
Benefits of open access to researchers from lower-income countries: A global analysis of reference patterns in 1980–2020 - Henrik Karlstrøm, Dag W Aksnes, Fredrik N Piro, 2024
The main objective of the open access (OA) movement is to make scientific literature freely available to everyone. This may be of particular importance to resea...
journals.sagepub.com
April 25, 2024 at 9:11 AM
New study by @karlstrom.bsky.social, Aksnes & Piro suggests that "the emergence of OA publishing has been particularly advantageous" in lower-income countries.
Read the article to learn more about this important topic!
#scholarlypublishing #OA
Read the article to learn more about this important topic!
#scholarlypublishing #OA
New paper! Increased access to literature changes the reference lists of academic papers, particularly for research from low- and middle-income countries. There are more references, they are newer, and they come from more established sources. doi.org/10.1177/0165...
#metascience
#metascience
April 25, 2024 at 8:14 AM
New paper! Increased access to literature changes the reference lists of academic papers, particularly for research from low- and middle-income countries. There are more references, they are newer, and they come from more established sources. doi.org/10.1177/0165...
#metascience
#metascience
How long until the language models stop replying with these super-generic stock phrases? How will we even know who is using them in their research practice?
March 13, 2024 at 1:11 PM
How long until the language models stop replying with these super-generic stock phrases? How will we even know who is using them in their research practice?