Grounding your research and advisory work is essential, but it can be challenging to do. It’s all too easy to get caught up in the virtual echo chamber of technology hype. Which the main reason that for many years I have led or participated in industry communities of practice CoP’s). These CoP’s each differ a little in format, but the basic gist of them is that public and private sector technology users and buyers can come together in private. Whilst there, they can share best practices and in equal measure horror stories as you may imagine conversations at these CoP’s in no way aligned with the excitement and hype of technology vendors. For example, if there is a topic that technology vendors don’t want to talk about, but the members of CoP’s can’t speak enough about it, is the concern of bias in AI. And they have a point, as it is a real concern that is not adequately addressed.
To recap a previous Analyst Note (which you can read in full here)
For those that don’t know, AI bias is the phenomena of an AI system giving prejudiced results due to misassumptions in the process. It’s easy to label biases as mistakes, but frequently they are not. They are answers that we do not agree with.
In practical terms, we need to understand that there are times when AI should not be used at all, no matter how sophisticated, efficient or cost-effective it may be. The risks are simply too high. Similarly, we need to understand where we can and arguably should use AI as the risks are negligible, and the benefits are high. Today there is no universally codified method or approach available to make these decisions. So all too often, enterprises either stumble headlong into costly and messy AI projects. Or, even more often, they don’t do anything at all and sit back to wait and see where the market heads.
Very early in my career, I tested a system that decided if applicants could get a phone connected or not. In its wisdom, it had decided that anyone living in a particular North London area could not have a phone without paying a prohibitive deposit upfront. It had also decided that no phones (without hefty upfront deposits) could be installed in homes where a crime had previously been committed. It was a terrible system that unfairly discriminated against regularly due to huge biases in its data set. Today’s AI systems are far more sophisticated, analyze much more data, much faster and more efficiently, but are often no less clumsy or error-ridden in their decision making. This raises many questions about the ethical use of AI. In can be argued that decisions that directly impact human lives should typically not be left solely to technology. On the other hand, using technology to decide if a document is an invoice or if it is a purchase order, whether it needs to be retained for ten years or disposed of, is the type of situation that AI is ideal for.
Hiding our heads in the sand and pretending there is not a problem, is not going to cut it anymore. The concerns over bias and the use of Deep Learning are going mainstream. A recent article in the UK’s Guardian newspaper goes as far as to say that some innovation is worth stifling. Sadly, that is correct, shinier, faster, and more efficient isn’t always better.
To be clear, there is no single technology called AI, which is simply an umbrella term for a broad set of tools that can be used with a broad set of data. Before you embark on building or deploying an AI system, you need to get expert advice. You can get that advice from us or somebody else, but one way or another, make sure you do get expert advice before you move too far forward. IT projects can be very hard to stop or pivot in a new direction once they gain momentum. With AI the result may not be simply an unloved or unused new IT system; it could deliver much graver repercussions for your business. On the flip side, merely sitting back and avoiding AI altogether isn’t a good plan either, as its a great fit for many front and back-office tasks and processes, deciding which ones can be tricky and at times take us into ethical mindfields.
Get trusted advice and technology insights for your business from the experts at Deep Analysis. [email protected]