Why Pilot an AI Certification Scheme?
AI laws and regulations will rely significantly upon soft law mechanisms, like standards, certification programs and industry practices to fulfill their regulatory objectives. The EU Commission has requested that European standardization organizations draft or adopt standards in support of the European approach to AI regulation. Policymakers in the US and UK have similarly acknowledged the role of soft law mechanisms, like conformity assessment, in helping extend the objectives of AI regulation to numerous contexts and to respond rapidly to technological and research developments.
In 2023, to better understand how such mechanisms might support regulatory objectives, the Standards Council of Canada conducted a pilot of the ISO/IEC 42001 standard for AI management systems as well as an assessment of RAI Institute’s Certification Scheme for AI systems.
RAI Institute participated in a section of the pilot which assessed ATB Financial, an Alberta-based financial institution and member of the RAI Institute. The pilot assessed a recommender system that it has developed to prompt ATB Client Facing Team Members about the ‘Next Best Conversation’ to have with customers. ATB Financial was an ideal candidate for the pilot due to their responsible AI readiness and maturity. ATB invested in and built out their responsible AI practice, with support from the RAI Institute. FAIRLY AI, which operates safety and compliance platforms for enterprises, supported the pilot by helping RAI Institute test and evaluate controls that required additional expertise.
Pilot Objectives and Outcomes
RAI Institute had two key objectives for the pilot, both of which were achieved:
Real-World Validation of the RAI Institute Certification Scheme: RAI Institute’s Certification Scheme is focused on the use case level and aligns with principles, laws, guidelines, research and best practices relevant to the industry, function and jurisdiction in which an AI system is developed and deployed.
Knowledge Sharing and Collaboration: The pilot participants have been collaborating with one another, and more broadly, to share learnings from the pilot in ways that can inform policy, practice and research.
Phases of the Pilot
This section of the accreditation pilot focused on ATB Financials’ Next Best Conversation (NBC) system, developed by ATB to support its customer service employees as in interactions with ATB retail-product customers.
Calibration: The first step in the pilot, undertaken by RAI Institute, was to calibrate its certification to the use case based on documentation provided by ATB Financial. The steps to do so included a high-level landscape mapping of the domain, an impact identification research and consulting exercise, validation with subject matter experts, configuration of the certification scheme and alignment of controls.
Evidence Review: The second step in the pilot was for RAI Institute to provide its calibrated questions and controls to ATB Financial and for ATB Financial to provide responses and supporting documentation. The RAI Institute team reviewed documents and checked in with the ATB Financial team to request further documents or clarify any documentation provided. During this step, FAIRLY helped RAI Institute test and evaluate controls that required additional expertise.
Scoring: The third step in the pilot, which commenced after a final document submission date, was for the RAI Institute team to score ATB Financials’ responses and document submissions according to its configured certification scheme.
Outcomes and Next Steps
The pilot allowed RAI Institute to validate, test and improve its Certification Scheme for AI systems and improve its responsible AI guidance for member organizations. Based on evidence provided during the pilot, ATB Financials’ NBC system received a provisional RAI Institute certification. FAIRLY AI gained further expertise and expanded its client offerings. SCC learned more about how RAI Institute’s system-level Certification Scheme and can link to the ISO/IEC 42001 standard for management systems and how this might inform a Canadian approach to AI standardization.
SCC has presented the pilot at a number of global events attended by lawmakers, regulators, standards organizations and industry organizations. These events have been organized by the British Standards Institution (BSI), the Asia Pacific Economic Cooperation (APEC), the United Nations Economic Cooperation of Europe and others.
Next steps for RAI Institute’s Certification Scheme will include further testing and the formalization of community feedback mechanisms. The RAI Institute’s Certification Scheme at the system level will play an important role in AI assurance, as organizations respond to the increasing pace of technology change.
About Responsible AI Institute (RAI Institute)
Founded in 2016, Responsible AI Institute (RAI Institute) is a global and member-driven non-profit dedicated to enabling successful responsible AI efforts in organizations. We accelerate and simplify responsible AI adoption by providing our members with AI conformity assessments, benchmarks and certifications that are closely aligned with global standards and emerging regulations.
Members include leading companies such as Amazon Web Services, Boston Consulting Group, ATB Financial and many others dedicated to bringing responsible AI to all industry sectors.
About ATB Financial
“About ATB”: With $58.3 billion in assets, ATB Financial is a leading financial institution that started in Alberta with the focus of putting people first. Our success comes from more than 5,000 team members who love to deliver exceptional experiences to nearly 800,000 clients across our Personal and Business Banking, ATB Wealth Management and ATB Capital Markets businesses. ATB provides expert advice, services and products through our many branches and agencies, our 24-hour Client Care Centre, four entrepreneur centres and our digital banking options. ATB powers possibilities for our clients, communities and beyond. More information about ATB can be found at atb.com
About FAIRLY AI
Fairly AI is an award-winning pioneer in AI Governance. Our innovative platform streamlines the adoption of AI technologies, ensuring they adhere to the highest standards of compliance and risk management. The Fairly approach has proven crucial in addressing key areas such as fairness, security, and privacy risks for both predictive and generative AI.
About Standards Council of Canada
The Standards Council of Canada (SCC) represents Canada at the International Organization for Standardization and International Electrotechnical Commission, helping shape tomorrow’s standards and conformity assessment.
SCC accredits standards development organizations and conformity assessment bodies, and our multilateral arrangements with international accreditation organizations help ensure certifications issued in different countries are consistent and recognized across the globe. Visit scc.ca to find out how to get involved.
Media Contact
For all media inquiries please refer to Head of Marketing & Engagement, Nicole McCaffrey, [email protected].
+1 440.785.3588.
Social Media
