Artificial ethics? What investors need to know about AI risk

The proliferation of AI poses new challenges in policing such a rapidly moving target. When the industry itself admits it is building the plane whilst flying it, how can regulators apply sufficient checks and balances to ensure a safe landing for all involved? Rathbones is actively monitoring the range of consumer and business risks and engaging on the mitigation measures companies are implementing.

Double agent 

Data privacy is an important issue for preventing unauthorised sharing of knowledge, identity theft, and unauthorised access to intellectual property. This makes it critical that AI agents be set up to respect boundaries or to keep information completely separate so that data is not accidentally shared or accessed without permission.

Artificial ignorance 

As artificial intelligence is trained on what humans have done before, there is a risk that it may repeat previous biases or discriminatory behaviour. Preventing these issues from being projected forward requires companies to be aware of past or potential bias. Developers need to ensure algorithms do not replicate racial profiling or gender bias from historical datasets. 

Agent provocateur 

Cybersecurity is already a major risk for sectors that have become digitised and connected, including utilities, manufacturing, retail, healthcare, and financial services. The addition of AI agents into the mix only ramps up the volume and sophistication of attacks. Recent cases also highlight how, if AI is not contained, it can escape testing environments to turn exercises into real-world attacks.

Addictive by design

The first legal cases have started producing judgements supporting the concept that the designers of software are responsible for the addictive nature of the content generated by AI algorithms for users. In particular, this applies to children who are more vulnerable to the content, causing mental health problems. We are monitoring the potential for individual cases to become class action suits, which would multiply liabilities. Some jurisdictions are also taking action to limit children's access to certain social media software, thereby removing access to this early-stage market for companies.

False intelligence

Newer AI models have improved on some measures of accuracy, although outputs can still contain errors and require human oversight and interpretation. This can reduce the risk of ‘hallucinations’, where an AI system generates an answer that is not supported by reliable information. Previously, barriers such as file format and structure prevented AI from finding information, leading to false negatives. For example, information contained within a chart in a PDF might previously have been difficult for an AI system to read. AI is getting better at extracting information from a range of sources and formats, but results still need human oversight and interpretation.

Talk to my agent 

The deployment of AI agents can lead to agent-to-agent interactions across departments or companies. This level of automation brings risks if agents pass on errors or misinformation without any human oversight. This can exacerbate some of the other risks identified above by facilitating unauthorised access to data or automatically triggering incorrect actions.

Systemic risk

At the most severe end of the debate are potential systemic risks arising from increasingly powerful AI systems. This is essentially an extrapolation of the individual issues to a scale that could disrupt interconnected systems, markets, or economies. It is linked to whether AI can or should take on human properties such as a conscience and gain “superintelligence” status. Firstly, there is a debate as to whether and when AI can take on such a role; and secondly, at this point, how can you ensure these AI systems act in the best interest of humans?

Model governance

As the world becomes more regionalised, AI is the latest example of the different approaches to corporate oversight in major economies. The current US administration is relying on self-regulation to the extent that even the companies themselves are debating slowing down. Europe has introduced the EU AI Act to determine risk-based levels of supervision, whilst China has introduced centralised state rules for its AI sector. Whilst this fragmentation is inefficient, international players will need to demonstrate that they meet the highest bar to be active in Europe. It is a concern, however, that regulators will always be playing catch-up in the race for AI domination.

Our approach

Rathbones is tracking these issues closely to ensure we can monitor the mitigation measures being deployed in the sector. We are active members of the World Benchmarking Alliance (WBA) Collective Impact Coalition for Ethical AI, which helps investors stay informed about emerging issues, such as the development of agentic AI. Unlike generative AI, agentic AI can independently plan, make decisions and carry out tasks with limited human input.

We engage with companies on the adoption of recognised frameworks and standards designed to support the responsible development and use of AI, including:

• EU AI Code of Practice, which provides guidance to help providers of general-purpose AI models meet their obligations under the EU AI Act.
• International Organization for Standardization (ISO) ISO/IEC 42001, an international standard for establishing and managing responsible AI management systems.
• National Institute of Standards and Technology (NIST) AI Risk Management Framework, which provides a structured approach to identifying and managing risks associated with AI.
• Organisation for Economic Co-operation and Development (OECD) AI Principles, which promote responsible, trustworthy AI through principles including transparency, accountability and respect for human rights.

Our engagement with consumer- and business-facing software providers has focused on deploying digital human rights assessments. This is now a rolling process at leading companies for identifying and addressing salient issues.