@iroldie.bsky.social
"From Foundations to GPT in Text Classification: A Comprehensive Survey on Current Approaches and Future Trends", the latest review article from FnTIR www.nowpublishers.com/article/Deta...
now publishers - From Foundations to GPT in Text Classification: A Comprehensive Survey on Current Approaches and Future Trends
Publishers of Foundations and Trends, making research accessible
www.nowpublishers.com
June 10, 2025 at 8:40 AM
"From Foundations to GPT in Text Classification: A Comprehensive Survey on Current Approaches and Future Trends", the latest review article from FnTIR www.nowpublishers.com/article/Deta...
Reposted
Are search engines getting worse—or is it time to rethink how we search? ADM+S researchers Oleg Zendel, Ashwin Nagappa & Johanne Trippas share insights on search quality, AI & what it means for how we find trustworthy information online @olegzendel.bsky.social @ashwinnag.bsky.social bit.ly/3F8sx44
Is Google search getting worse? Maybe what we want from it has changed
Has the simple search become less useful in a world of low-information, SEO-optimised sites, and often faulty AI summaries? Are there better ways to navigate the web?
bit.ly
May 14, 2025 at 4:04 AM
Are search engines getting worse—or is it time to rethink how we search? ADM+S researchers Oleg Zendel, Ashwin Nagappa & Johanne Trippas share insights on search quality, AI & what it means for how we find trustworthy information online @olegzendel.bsky.social @ashwinnag.bsky.social bit.ly/3F8sx44
Reposted
Excellent presentation by @admscentre.org.au @rmitcomputing.bsky.social master’s student Kun Ran m at #chiir2025
@marwahalaofi.com @iroldie.bsky.social
www.damianospina.com/publication/...
@marwahalaofi.com @iroldie.bsky.social
www.damianospina.com/publication/...
March 26, 2025 at 1:03 AM
Excellent presentation by @admscentre.org.au @rmitcomputing.bsky.social master’s student Kun Ran m at #chiir2025
@marwahalaofi.com @iroldie.bsky.social
www.damianospina.com/publication/...
@marwahalaofi.com @iroldie.bsky.social
www.damianospina.com/publication/...
Reposted
Our @theconversation.com article discussing 4 ethical models of search engines is out!
theconversation.com/what-makes-a...
@admscentre.org.au
@rmitcomputing.bsky.social
@umbrellacorpn.bsky.social
theconversation.com/what-makes-a...
@admscentre.org.au
@rmitcomputing.bsky.social
@umbrellacorpn.bsky.social
What makes a good search engine? These 4 models can help you use search in the age of AI
To make sure search engines serve us well, it’s helpful to imagine these tools having different roles – whether it’s a “librarian” or a “teacher”.
theconversation.com
March 26, 2025 at 2:32 AM
Our @theconversation.com article discussing 4 ethical models of search engines is out!
theconversation.com/what-makes-a...
@admscentre.org.au
@rmitcomputing.bsky.social
@umbrellacorpn.bsky.social
theconversation.com/what-makes-a...
@admscentre.org.au
@rmitcomputing.bsky.social
@umbrellacorpn.bsky.social
"Two Heads Are Better Than One: Improving Search Effectiveness Through LLM Generated Query Variants", preprint is up marksanderson.org/publications... Awesome work by @rmitcomputing.bsky.social Masters student @rankun203.bsky.social with Marwah Alaofi and @damianospina.com Presented ACM CHIIR.
marksanderson.org
March 1, 2025 at 7:51 AM
"Two Heads Are Better Than One: Improving Search Effectiveness Through LLM Generated Query Variants", preprint is up marksanderson.org/publications... Awesome work by @rmitcomputing.bsky.social Masters student @rankun203.bsky.social with Marwah Alaofi and @damianospina.com Presented ACM CHIIR.
For the first time, Nature publishes a list of "retraction hotspots": academic institutions where a great many papers have been withdrawn post publication. www.nature.com/articles/d41...
Exclusive: These universities have the most retracted scientific articles
A first-of-its-kind analysis by Nature reveals which institutions are retraction hotspots.
www.nature.com
February 23, 2025 at 10:23 PM
For the first time, Nature publishes a list of "retraction hotspots": academic institutions where a great many papers have been withdrawn post publication. www.nature.com/articles/d41...
The TREC website has been updated for the first time in... decades? Looking good. trec.nist.gov/index.html
February 20, 2025 at 11:55 PM
The TREC website has been updated for the first time in... decades? Looking good. trec.nist.gov/index.html
"Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges". Inject unrelated toxic content into a relevant passage, an LLM judge still says the passage is relevant arxiv.org/abs/2501.18536 from Manveer Tamber & Jimmy Lin.
Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges
Consider a scenario in which a user searches for information, only to encounter texts flooded with misleading or non-relevant content. This scenario exemplifies a simple yet potent vulnerability in ne...
arxiv.org
February 3, 2025 at 12:31 AM
"Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges". Inject unrelated toxic content into a relevant passage, an LLM judge still says the passage is relevant arxiv.org/abs/2501.18536 from Manveer Tamber & Jimmy Lin.
Reposted
"Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessment" w/@Julian S. @danulah.bsky.social @umbrellacorpn.bsky.social accepted at #webconf2025! 😱🌟
We present an LLM-based pipeline that boosts relevance assessment accuracy through modular classification.
#SIGIR2025
We present an LLM-based pipeline that boosts relevance assessment accuracy through modular classification.
#SIGIR2025
January 22, 2025 at 6:19 AM
"Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessment" w/@Julian S. @danulah.bsky.social @umbrellacorpn.bsky.social accepted at #webconf2025! 😱🌟
We present an LLM-based pipeline that boosts relevance assessment accuracy through modular classification.
#SIGIR2025
We present an LLM-based pipeline that boosts relevance assessment accuracy through modular classification.
#SIGIR2025
Two SIGIR Information Retrieval greats were made ACM Fellows this year: Maarten de Rijke and Justin Zobel. Many congratulations to both! www.acm.org/media-center...
2024 ACM Fellows Celebrated for transformative contributions to computing science and technology.
ACM has named 55 of its members ACM for transformative contributions to computing science and technology. All the 2024 inductees are longstanding ACM Members whose accomplishments were selected by the...
www.acm.org
January 22, 2025 at 11:50 PM
Two SIGIR Information Retrieval greats were made ACM Fellows this year: Maarten de Rijke and Justin Zobel. Many congratulations to both! www.acm.org/media-center...
Building on four successful workshops in the last few years, it was decided to create the Inaugural SEASON conference of the Search Engines and Society Network, starting in 2025. Submissions, April 30th. Held in Hamburg. easychair.org/cfp/SEASON2025
CFP
easychair.org
January 17, 2025 at 6:51 AM
Building on four successful workshops in the last few years, it was decided to create the Inaugural SEASON conference of the Search Engines and Society Network, starting in 2025. Submissions, April 30th. Held in Hamburg. easychair.org/cfp/SEASON2025
Thanks @hscells.bsky.social for pointing me to this
My colleagues published about pairwise relevance labeling this last year (and even earlier on arXiv):
aclanthology.org/2024.finding...
Why I personally think it works better, mainly because it's hard to calibrate a pointwise relevance prediction, but a pairwise prediction hardly needs calibration.
aclanthology.org/2024.finding...
Why I personally think it works better, mainly because it's hard to calibrate a pointwise relevance prediction, but a pairwise prediction hardly needs calibration.
Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting
Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, Michael Bendersky. Findings of the Association for Computational ...
aclanthology.org
January 14, 2025 at 3:15 PM
Thanks @hscells.bsky.social for pointing me to this
Using LLMs for pairwise relevance decisions, I haven't seen that tried before, but it makes perfect sense to try them here. My impression is that pairwise relevance hasn't been used much in the past because of the cost of lablelling.
Local LLMs might unlock a new age of democratized search relevance.
Here I talk about using Qwen 2.5 locally as a local pairwise search evaluator
softwaredoug.com/blog/2025/01...
Here I talk about using Qwen 2.5 locally as a local pairwise search evaluator
softwaredoug.com/blog/2025/01...
Local LLMs as search relevance judge
Local LLMs can evaluate 100s of result pairs a minute, enabling a new age of rapid search relevance improvements
softwaredoug.com
January 14, 2025 at 5:48 AM
Using LLMs for pairwise relevance decisions, I haven't seen that tried before, but it makes perfect sense to try them here. My impression is that pairwise relevance hasn't been used much in the past because of the cost of lablelling.
There have been a number of investigations on whether LLMs can replace humans for relevance assessment. I think the evidence is showing LLMs have a strong role, but won't replace. This recent paper from @claclarke.bsky.social & Laura Dietz supports this view arxiv.org/abs/2412.17156
LLM-based relevance assessment still can't replace human relevance assessment
The use of large language models (LLMs) for relevance assessment in information retrieval has gained significant attention, with recent studies suggesting that LLM-based judgments provide comparable e...
arxiv.org
January 1, 2025 at 10:05 PM
There have been a number of investigations on whether LLMs can replace humans for relevance assessment. I think the evidence is showing LLMs have a strong role, but won't replace. This recent paper from @claclarke.bsky.social & Laura Dietz supports this view arxiv.org/abs/2412.17156
What a team of keynote speakers. I must confess seeing that Steve Robertson will be there is a thrill. One of the legends of information retrieval reflecting on the field. #sigir2025
sigir2025.dei.unipd.it/keynote-spea...
sigir2025.dei.unipd.it/keynote-spea...
SIGIR 2025, Padua, 13-18 July | Keynotes
The SIGIR 2025 keynotes are held by esteemed speakers: Robertson S., Gurevych I. and Frieder O., who will cover topics that range from AI in medical search and ecommendation to BM25 and probabilistic ...
sigir2025.dei.unipd.it
December 24, 2024 at 6:11 AM
What a team of keynote speakers. I must confess seeing that Steve Robertson will be there is a thrill. One of the legends of information retrieval reflecting on the field. #sigir2025
sigir2025.dei.unipd.it/keynote-spea...
sigir2025.dei.unipd.it/keynote-spea...
"Two Heads Are Better Than One: Improving Search Effectiveness Through LLM Generated Query Variants", short paper accepted #chiir2025. Led by our Masters student Kun Ran with Marwah Alaofi, myself, and @damianospina.com. Awesome result Kun! @admscentre.org.au @rmitcomputing.bsky.social
December 17, 2024 at 8:58 AM
"Two Heads Are Better Than One: Improving Search Effectiveness Through LLM Generated Query Variants", short paper accepted #chiir2025. Led by our Masters student Kun Ran with Marwah Alaofi, myself, and @damianospina.com. Awesome result Kun! @admscentre.org.au @rmitcomputing.bsky.social
We invite PhD students to submit your work and join us at PhD Symposium @TheWebConf 2025, (www2025.thewebconf.org/phd-symposium). The submission due date is 18 Dec, 2024 (AOE)! We will see you in the beautiful and amazing Sydney down under! #WWW2025 #WebConf2025
December 14, 2024 at 10:59 PM
We invite PhD students to submit your work and join us at PhD Symposium @TheWebConf 2025, (www2025.thewebconf.org/phd-symposium). The submission due date is 18 Dec, 2024 (AOE)! We will see you in the beautiful and amazing Sydney down under! #WWW2025 #WebConf2025
Thanks to the #sigirap2024 people for giving us a mention on the conference bag, brought a smile to our faces.
December 10, 2024 at 11:03 AM
Thanks to the #sigirap2024 people for giving us a mention on the conference bag, brought a smile to our faces.
So many aspects of evaluation including fully synthetic test collections, loved it.
Excellent keynote by Peter Bailey from Canva about how to build a more cost-effective online & offline experimentation framework #sigirap2024 #melbourne #naarm @admscentre.org.au @rmitcomputing.bsky.social
December 10, 2024 at 12:48 AM
So many aspects of evaluation including fully synthetic test collections, loved it.
I enjoyed attending the thought provoking two day gathering that helped drive the creation of this document. I look forward to reading it. "Future of Information Retrieval Research in the Age of Generative AI" arxiv.org/pdf/2412.02043
arxiv.org
December 9, 2024 at 9:28 AM
I enjoyed attending the thought provoking two day gathering that helped drive the creation of this document. I look forward to reading it. "Future of Information Retrieval Research in the Age of Generative AI" arxiv.org/pdf/2412.02043
In her talk, @katecrawford.bsky.social delved into the history of Large Language Models, she rightly highlighted the pioneering work of Fred Jelinek who kicked off language models in 1972, however, I wouldn't be faithful to my handle (IR Oldie) if I didn't highlight some IR history...
December 8, 2024 at 1:23 AM
In her talk, @katecrawford.bsky.social delved into the history of Large Language Models, she rightly highlighted the pioneering work of Fred Jelinek who kicked off language models in 1972, however, I wouldn't be faithful to my handle (IR Oldie) if I didn't highlight some IR history...
Just in case you didn't see this shared elsewhere. Maarten writes "I am asking for donations because our daughter Emma passed away". epilepsie.digicollect.nl/maarten-de-r...
Maarten de Rijke kan het niet alleen. Help jij mee?
Epilepsie verstoort levens! Steun je mij met een donatie in mijn collectebus? Met de opbrengst kunnen we samen op zoek naar de oplossing van de verstorende aanvallen. Ik ben je eeuwig dankbaar!
epilepsie.digicollect.nl
December 7, 2024 at 9:01 PM
Just in case you didn't see this shared elsewhere. Maarten writes "I am asking for donations because our daughter Emma passed away". epilepsie.digicollect.nl/maarten-de-r...
What a magnificent communicator @katecrawford.bsky.social is. Loved her talk at the State Library of Victoria on Thursday broadening the perspectives we have about AI. The non-techy who accompanied me was buzzing about this event.
December 6, 2024 at 8:38 PM
What a magnificent communicator @katecrawford.bsky.social is. Loved her talk at the State Library of Victoria on Thursday broadening the perspectives we have about AI. The non-techy who accompanied me was buzzing about this event.
Reposted
And we have a #Dagstuhl Report!
Evaluation Perspectives of #RecSys. Edited by @christinebauer.bsky.social, @evazangerle.bsky.social, and myself. Written by a whole host of fantastic #recsys people. Too many to mention (pic here www.dagstuhl.de/en/seminars/...)
drops.dagstuhl.de/entities/doc...
Evaluation Perspectives of #RecSys. Edited by @christinebauer.bsky.social, @evazangerle.bsky.social, and myself. Written by a whole host of fantastic #recsys people. Too many to mention (pic here www.dagstuhl.de/en/seminars/...)
drops.dagstuhl.de/entities/doc...
Evaluation Perspectives of Recommender Systems: Driving Research and Education (Dagstuhl Seminar 24211)
drops.dagstuhl.de
November 26, 2024 at 8:57 PM
And we have a #Dagstuhl Report!
Evaluation Perspectives of #RecSys. Edited by @christinebauer.bsky.social, @evazangerle.bsky.social, and myself. Written by a whole host of fantastic #recsys people. Too many to mention (pic here www.dagstuhl.de/en/seminars/...)
drops.dagstuhl.de/entities/doc...
Evaluation Perspectives of #RecSys. Edited by @christinebauer.bsky.social, @evazangerle.bsky.social, and myself. Written by a whole host of fantastic #recsys people. Too many to mention (pic here www.dagstuhl.de/en/seminars/...)
drops.dagstuhl.de/entities/doc...
A cool blog post from Ellese Cotterill on her work at Canva to spin up a known item test collection from scratch generated entirely from an LLM. www.canva.dev/blog/enginee...
How to improve search without looking at queries or results - Canva Engineering Blog
How we improved Canva’s private design search while respecting the privacy of our community.
www.canva.dev
November 25, 2024 at 7:32 AM
A cool blog post from Ellese Cotterill on her work at Canva to spin up a known item test collection from scratch generated entirely from an LLM. www.canva.dev/blog/enginee...