AI Project Failures

Today a friend of mine, who is developing an AI consultancy, asked me the following question:

Hi Peter,

I wanted to ask you something. I'm trying to find a report that shows the failure rate of AI projects but haven't been successful yet.  Do you have any studies or articles in mind? 

Kind Regards,

Name Withheld

My answer was unequivocal.  If you don't agree, please tell me why.

What is 'success'?

--------------------------------

Hi Name Withheld,

I would be 99.8% certain that such a report or statistic does not exist, for the following reasons:

1) Definition of an "AI project"

• Does this include projects for academic research, public sector, private sector, pilot studies and 'proof of concept' studies?
• Does this include projects in which AI is a minor feature, a significant feature or a major feature?

2) Definition of a 'failed' project

• If a project completes, but it does not meet expectations, is that a failure?
• If a project completes successfully, but costs 3 times the original estimate, is that a failure?
• If an AI project fails to complete for reasons unrelated to AI (e.g. organisational difficulties, withdrawal of funding, poorly implemented technologies) is that a failure?

3) Confidentiality

Most projects take place within private organisations, or in public bodies protected by non-disclosure agreements, or 'official secrecy'.  Therefore, the amount of reporting, especially failure reporting, is very low.  Perhaps less than 10% of failures will enter the public domain.

Conversely, when there are successes, they are more likely to be reported.

However, many successful commercial and state-sponsored projects will never be reported, due to their need for secrecy.

Conclusion:

If anyone offers statistics on AI project failures, they must be inventing the numbers for the reasons explained above.  However, the major project management organisations, such as the Project Management Institute (PMI), Axelos (for PRINCE2) and the APM, often promote their membership schemes and training courses by talking about the historical failure rates of projects.

My guess is that around 50% of digital projects fail to deliver on their intended goals.  Some projects fail completely, e.g. are terminated, while others are completed, but the outcomes do not meet original expectations.

Given that AI projects are implementing new technologies and novel solutions, they are unable to follow long-established, proven methodologies.  Hence, the failure rates for AI projects are likely to be significantly higher than the failure rates for conventional IT projects.  My estimate is therefore a failure rate of 50% or more.

Best regards,

Peter

--------------------------------------

Author: PJ Moar of Moar Partnerships

Email: p.moar@moar.com

Twitter: @MoarPart