How does a machine recognise plankton? In much the same way as people recognise the faces of other people. Two processes come together here. First, certain regions of the human brain are specialised in recognising faces. This is why even babies respond to faces or even face-like patterns. However, the ability is still only vaguely developed at birth. Only gradually do children learn to recognise and distinguish people better and better by looking at many different faces.

The automatic classification of plankton works in a very similar way. First, researchers develop a machine learning architecture that is specialised in classifying images. This architecture is still very vague and flexible. In a second step, the machine learns to identify important features and patterns on its own with the help of training examples, i.e. thousands of plankton photos identified by specialists, and to deduce the plankton species from them. The machine learns with each additional annotated photo, independently adapts its architecture and thereby refines its ability to classify plankton. 

Machines learn similarly to humans

Similar to humans, who usually cannot say how they recognised their peers, in machine learning it also remains unknown which features the machine uses to identify the plankton. But when it comes to classifying millions of plankton photos, this is not of great interest in any case. “Our goal is to be able to correctly identify as many plankton species as possible in a short period of time,” says physicist Marco Baity-Jesi, head of the Eawag Data Science Group. “That’s where machine learning offers great advantages.” While a specialist can perform dozens of determinations a day and will be tired at the end of the day, the machine accurately determines about a million plankton every day, and without any signs of fatigue. 

A group of Eawag researchers around Sreenath Kyathanahally, postdoc in the Data Science Group, and Baity-Jesi has now applied advanced models of machine learning, or more precisely deep learning, to the classification of zooplankton in Lake Greifen. The best models were based on transfer learning and ensembling. “We were able to show that these models classify plankton photos better than previously used models,” says Baity-Jesi. “We reached an accuracy of 98 percent.” The plankton photos were taken by the Aquascope, an underwater microscope from the Eawag Phytoplankton Dynamics Group led by Francesco Pomati, which takes several thousand photos of plankton in Lake Greifen every day.

Deep learning – further developing the “brain” of the machine

So what exactly is deep learning, transfer learning and ensembling? Let us return to the analogy of facial recognition: What evolution has achieved over millions of years, namely the evolution of the human brain with all its finesse and highly specialised functions, is what researchers are trying to recreate, at least approximately, in deep learning. Inspired by the human brain, they are deepening the inner structures of the “machine brain” and building artificial neural networks with many intermediate layers into the algorithms. In this way, the machine learns to link what it has already learned with new content and thus to continuously learn new things – without human intervention. With each additional plankton photo, the machine learns to recognise even inconspicuous differences between the small creatures and to classify them correctly.

Another further development is ensembling. The principle here is similar to the conviction of a criminal: one eyewitness is good; several eyewitnesses are better. With ensembling, the researchers therefore combine several deep learning methods – the machine’s eyewitnesses – to increase accuracy. This ensures as few plankton as possible can sneak away undetected.

Transfer learning – transferring acquired knowledge to a new task

Parallel to the development of the “machine brain”, the machine learning process is being further refined. Transfer learning is about using knowledge already gained from other applications. For example, if the machine can already classify faces, it can use this knowledge to learn to classify lake plankton more quickly. This kind of learning also has a human analogy. People who grow up in Europe first learn to distinguish European faces. Asians look rather similar to Europeans. But once you have understood the typical European facial features, you also learn to distinguish Asian faces more quickly, for example during a longer stay in Asia.

Machines also learn to predict flooding and toxicity of substances

The methods of artificial intelligence, such as machine learning, can support water research in many areas. “We are also supporting other Eawag projects,” says Baity-Jesi. For instance, a project to study flood forecast models led by Peter Reichert, group leader of Systems Analysis and Water Management, and Chaopeng Shen, a guest professor from the USA currently working at Eawag. “In another project we collaborate with the group Animal Cells and Organisms led by Kristin Schirmer to predict how chemical substances affect fish.” The latter is likely to reduce the number of tests on live animals to determine the toxicity of substances.

Cover picture: Eawag