Learning to design machines that learn

Reading time: 2 mins


The lifecycle of any AI or machine learning product starts with its conception, followed by requirements analysis and then design.  This may sound conventional.  However, many of the thought processes which accompany these phases are relatively new.  So let’s consider some of these new ways of thinking.

This article will focus on products based solely on machine learning, because this technology is visibly transforming many of our goods, services and business sectors.  It does not attempt to generalise for the whole AI product landscape.  More research would be needed for that. 

Most professional people in the digital sphere, both developers and clients, have built their careers around the implementation and operation of 'traditional' algorithmic software solutions.  Machine learning is different, of course, because it uses unfamiliar architectures offering novel capabilities.  It is this unfamiliarity and novelty which makes it so compelling, but which also requires a different product design methodology.

Product designers for machine learning are discovering, through trial and error, how best to refine their early-stage development techniques.  One team making tangible progress are Futurice, a software company with offices across northern and central Europe.

During 2017 Futurice ran workshops exploring design methods for machine learning, generously sharing their tools and techniques with attendees.

They presented on the technical and ethical dimensions of building “IA” products, where IA is the acronym for intelligence augmentation, meaning people working more effectively while assisted by smart, machine learning products.  Their goal is to help designers “become conversant with the opportunities and threats machine learning services present.”

Futurice graphic representing IA, or "intelligence augmentation"

They define machine learning as “a toolbox that gives you, the designer, new ways to address human needs”, which of course is the fundamental purpose of most good product design.

Futurice describe four categories of use cases amenable to an AI-type solutions:

• Predicting
• Personalising
• Recognising
• Uncovering structures

The design process must also consider some novel aspects of machine learning tools:

• Design for learning – AI solutions will make mistakes, so they must be configured to learn and improve over time.
• Design for failure - 100% accuracy or effectiveness cannot be guaranteed, so assumptions of failure must be intrinsic to the design solution.
• Design for the worst – A combination of feedback loops and novel environments can result in wholly inappropriate outputs from products, even after massive investment and extensive training.  Designers must make contingencies for these worst-case scenarios.

To support their product design workshops, Futurice utilised a Smart Service Storyline canvas on which stakeholders could apply machine learning concepts to their new product ideas.  The canvas directs clients to consider the use cases listed earlier, such as prediction and personalisation.  They are also encouraged to think about any opportunities to incorporate learning loops.

Machine learning’s inherent weaknesses, notably its inability to deliver correct outputs at all times, is explored via a Confusion Matrix (see below) which highlights the challenge for users of receiving, in varying quantities, false negatives, false positives, true positives and true negatives.

Designing for false positives and false negatives is paramount.

A confusion matrix which could be applied to a health diagnostics tool

Futurice were successful in conveying some of the differences between design processes for traditional software development and those for machine learning products.  For workshop participants it was both absorbing and, at times, conceptually challenging.  Five years from now, however, these methods will surely be familiar to all product designers.

To learn more, visit the Futurice IA Design Kit.



Writer: PJ Moar of Moar Partnerships
Email: p.moar@moar.com
Twitter: @MoarPart