CertivizeAI
@certivizeai.bsky.social
Agentic software testing for GenAI applications for risk identification.
https://www.certivize.me/
https://www.certivize.me/
Pinned
CertivizeAI
@certivizeai.bsky.social
· Apr 17
Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective
Agentic systems powered by large language models (LLMs) are becoming progressively more complex and capable. Their increasing agency and expanding deployment settings attract growing attention over ef...
arxiv.org
Agentic systems based on large language models are prone to various liability issues due to their vulnerabilities. We analyzed different aspects of agent liability by considering existing agent failure modes in a principal-agent perspective.
arxiv.org/abs/2504.03255
arxiv.org/abs/2504.03255
Reposted by CertivizeAI
"More than 100 top artificial intelligence researchers have signed an open letter calling on generative AI companies to allow investigators access to their systems, arguing that opaque company rules are preventing them from safety-testing tools"
#Philtech
www.washingtonpost.com/technology/2...
#Philtech
www.washingtonpost.com/technology/2...
Top AI researchers say OpenAI, Meta and more hinder independent evaluations
Firms like OpenAI and Meta use strict protocols to keep bad actors from abusing AI systems. But researchers argue these rules are chilling independent evaluations.
www.washingtonpost.com
March 11, 2024 at 12:15 AM
"More than 100 top artificial intelligence researchers have signed an open letter calling on generative AI companies to allow investigators access to their systems, arguing that opaque company rules are preventing them from safety-testing tools"
#Philtech
www.washingtonpost.com/technology/2...
#Philtech
www.washingtonpost.com/technology/2...
Reposted by CertivizeAI
OpenAI slashes AI model safety testing time
https://www.ft.com/content/8253b66e-ade7-4d1f-993b-2d0779c7e7d8
https://www.ft.com/content/8253b66e-ade7-4d1f-993b-2d0779c7e7d8
OpenAI slashes AI model safety testing time
Testers have raised concerns that its technology is being rushed out without sufficient safeguards
www.ft.com
April 11, 2025 at 4:08 AM
OpenAI slashes AI model safety testing time
https://www.ft.com/content/8253b66e-ade7-4d1f-993b-2d0779c7e7d8
https://www.ft.com/content/8253b66e-ade7-4d1f-993b-2d0779c7e7d8
Agentic systems based on large language models are prone to various liability issues due to their vulnerabilities. We analyzed different aspects of agent liability by considering existing agent failure modes in a principal-agent perspective.
arxiv.org/abs/2504.03255
arxiv.org/abs/2504.03255
Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective
Agentic systems powered by large language models (LLMs) are becoming progressively more complex and capable. Their increasing agency and expanding deployment settings attract growing attention over ef...
arxiv.org
April 17, 2025 at 10:16 PM
Agentic systems based on large language models are prone to various liability issues due to their vulnerabilities. We analyzed different aspects of agent liability by considering existing agent failure modes in a principal-agent perspective.
arxiv.org/abs/2504.03255
arxiv.org/abs/2504.03255
Reposted by CertivizeAI
Well said!
It'd be nice to replace animal testing with AI simulation but
if AI was really able to do that
they would not have needed to first fire all the smart, experienced, and honest people at the FDA.
if AI was really able to do that
they would not have needed to first fire all the smart, experienced, and honest people at the FDA.
FDA plans to phase out animal testing requirement for drug testing and replace it with “AI-based computational models of toxicity” and organoid toxicity testing www.fda.gov/news-events/...
April 17, 2025 at 10:21 AM
Well said!
Reposted by CertivizeAI
The massive staff cuts at #HHS hit many of the folks who kept HHS running, leaving #NIH labs hoarding dwindling supplies & trying to decide whether further demanded cuts should affect critical people or critical equipment. Destruction, not restructuring. www.statnews.com/2025/04/11/h...
Inside U.S. health agencies, workers confront chaos and questions as operations become unglued
The Trump administration’s remaking of HHS — in a matter of weeks — is sparking basic questions about how parts of the agency and those it oversees can continue to function.
www.statnews.com
April 11, 2025 at 6:24 PM
The massive staff cuts at #HHS hit many of the folks who kept HHS running, leaving #NIH labs hoarding dwindling supplies & trying to decide whether further demanded cuts should affect critical people or critical equipment. Destruction, not restructuring. www.statnews.com/2025/04/11/h...
We definitely need to stipulate third-party evals and testing for this...including in areas not traditionally covered by FDA.
"Most of the medical artificial intelligence currently being used in the United States does not fall under FDA’s jurisdiction..."
Even if you knew nothing else about AI in medicine, that should be enough to make you want to scream.
hls.harvard.edu/today/ai-is-...
Even if you knew nothing else about AI in medicine, that should be enough to make you want to scream.
hls.harvard.edu/today/ai-is-...
AI is transforming health care — and the law could help safeguard innovation and patients alike - Harvard Law School
Harvard Law Professor Glenn Cohen shares how artificial intelligence is changing medicine — and how the law can adapt.
hls.harvard.edu
April 17, 2025 at 9:39 AM
We definitely need to stipulate third-party evals and testing for this...including in areas not traditionally covered by FDA.
Reposted by CertivizeAI
Forgot to share this work here as well:
In medical settings, researchers & institutional users often cite medical examination benchmark results as a proxy for AI model performance.
This is a woefully misguided practice - as with actual doctors, exam performance does NOT equal clinical competence!
In medical settings, researchers & institutional users often cite medical examination benchmark results as a proxy for AI model performance.
This is a woefully misguided practice - as with actual doctors, exam performance does NOT equal clinical competence!
January 28, 2025 at 5:47 PM
Forgot to share this work here as well:
In medical settings, researchers & institutional users often cite medical examination benchmark results as a proxy for AI model performance.
This is a woefully misguided practice - as with actual doctors, exam performance does NOT equal clinical competence!
In medical settings, researchers & institutional users often cite medical examination benchmark results as a proxy for AI model performance.
This is a woefully misguided practice - as with actual doctors, exam performance does NOT equal clinical competence!
FDA should definitely speed up on the genAI regulation and evaluation standards for medical softwares (aka. SaMDs).
April 17, 2025 at 9:32 AM
FDA should definitely speed up on the genAI regulation and evaluation standards for medical softwares (aka. SaMDs).
Finally set this up!!
April 17, 2025 at 9:31 AM
Finally set this up!!