Retinal Scanning Startup DeepEyeVision, Inc. is First in the World to Receive Validation from BSI's New AI Algorithm Audit and Dataset Testing Services
Verification of safety and reliability for AI applications in high-risk fields such as healthcare
20 November 2024 – DeepEyeVision, Inc., a pioneering med tech startup and Jichi Medical University venture that uses AI to improve diagnosis of eye conditions and enable faster treatment and diagnosis has become the first in the world to be verified by BSI that its AI algorithms and models, and the datasets used to train them, are performing as reported.
When utilizing AI-based systems in high-risk fields such as healthcare, it is essential to ensure the safety and reliability of the AI models and datasets. This is to verify that the systems are not influenced by 'AI bias' and that they possess robustness against 'unknown data.'
In light of this situation, an independent evaluation of DeepEyeVision’s tool by BSI’s new AI Algorithm Auditing & Dataset Testing service (AA&DT) has affirmed that the start-up accurately reports its AI performance metrics and values. This means ophthalmologists, customers, regulators, and other stakeholders can be confident that the models and datasets used in DeepEyeVision’s computer-aided diagnosis product have been verified, providing trust in the organization’s AI performance claims.
The BSI Algorithm Auditing & Dataset Testing (AA&DT) service is a voluntary, independent and impartial assessment of AI algorithms, models and the datasets used to train them by experienced technical and regulatory AI experts[1]. The service verifies the accurate reporting of AI performance metrics describing robustness, bias, classification performance and semantic segmentation.
This service is substantiated by international AI standards including Bias in AI systems and AI aided decision making (ISO 24027), Robustness for Neural Networks ISO (ISO 24029-1), Assessment of machine learning Part 1 (ISO 4213), Information technology. Medical image-based modelling for 3D printing – Segmentation (ISO 3532-2).
In DeepEyeVision’s case, its AI algorithms and models make predictions based on robust datasets of retinal scan images, and particularly the fundus (the back part of the inside of the eye), to detect potential eye conditions and enable faster diagnosis and treatment.
Manuela Gazzard, President, Regulatory Services, BSI said: “AI has the potential to be a transformative force for good for healthcare and the treatment of myriad conditions, including those affecting eyesight. But to realize this potential it is vital to build societal trust that AI is being used safely and ethically.
“BSI’s new AI Algorithm Auditing & Dataset Testing service is designed to support this process and establish confidence in AI-enabled products, services, or systems. AA&DT services can help organizations that deploy AI in critical areas where biases or inaccuracies can have serious consequences, such as healthcare, finance, or law enforcement, and ensure that they manage risks whilst not stifling innovation. It is fantastic to see DeepEyeVision become the first company to be verified by BSI and we look forward to seeing how their pioneering work benefits clinicians and patients around the world.”
Dr. Hidenori Takahashi, CEO of DeepEyeVision, Professor at the University of Tsukuba, and Associate Professor at Jichi Medical University, said:
"DeepEyeVision's AI has been designed to assess the consistency of outputs in response to input data variations and to investigate the risks of abnormal outputs. The results of these assessments are fed back into development to implement improvements that minimize output fluctuations and prevent abnormal outputs. For example, adjustments have been made to ensure that the same analytical results are obtained, even when fundus images—photos of the back of the eye—are either too bright or too dark. DeepEyeVision continues to advance its development to ensure that AI can be used safely and ethically in the medical workflow.
However, for society to recognize that medical AI is being inspected and used in a safe and ethical manner, third-party audits are essential. We are pleased that BSI has established this audit system, the first of its kind globally, and that DeepEyeVision was selected as its inaugural verification target. I expect BSI's pioneering efforts will contribute to the spread of reliable AI, not only in healthcare but also in fields such as finance and law enforcement, driving societal progress."
For further information on the BSI Algorithm Auditing & Dataset Testing service (AA&DT), visit https://www.bsigroup.com/en-GB/products-and-services/assessment-and-certification/system-certification/ai-algorithm-auditing-dataset-testing/
About BSI:
BSI is a business improvement and standards company that partners with more than 77,500 clients globally across multiple industry sectors. BSI provides organizations with the confidence to grow by working with them to tackle society’s critical issues – from climate change to building trust in AI and everything in between - to accelerate progress towards a fair society and a sustainable world.
For over a century BSI has been recognized for having a positive impact on organizations and society, building trust and enhancing lives. Today BSI engages with a 15,000 strong global community of experts, industry and consumer groups, organizations and governments to deliver on its purpose by helping its clients fulfil theirs.
About DeepEyeVision:
DeepEyeVision Inc. is a venture company originating from Jichi Medical University, established by Dr. Hidenori Takahashi, M.D., Ph.D., who is a Professor at the Cybermedicine Research Center of the University of Tsukuba and an Associate Professor in the Department of Ophthalmology at Jichi Medical University. The company was founded with the mission of bringing medical AI technologies developed in academic research to the broader society. Leveraging AI-based imaging diagnostics, DeepEyeVision is committed to advancing not only ophthalmology but also various other medical fields through its research and development efforts.
[1] An AI model is a program that has been trained on a set of data to recognize certain patterns or make certain decisions without further human intervention.
