Scientists develop algorithms that detect people with depression by simply scanning Instagram photos

The researchers say their proof-of-concept artificial intelligence can identify depressed people ‘better than doctors’.
The researchers say their proof-of-concept artificial intelligence can identify depressed people ‘better than doctors’.

Scientists in the US say they have created machine learning algorithms that can identify depressed people by scanning for “clues” in Instagram photos.

The researchers from the University of Vermont and Harvard University claim their algorithm’s detection rate is 70%, adding that it is “more reliable than the 42% success rate of general-practice doctors diagnosing depression in person”.

“This points toward a new method for early screening of depression and other emerging mental illnesses,” says Chris Danforth, of the University of Vermont.

A young girl taking a selfie.
(jax973/Getty Images/iStockphoto)
Researchers claim their AI algorithm can detect those living with depression (jax973/Getty Images)

The research was done in two stages: the first was about identifying the clues on Instagram photos that suggests the user might be depressed, while the second stage involved teaching the computer to detect those people using machine learning algorithms.

The scientists asked for access to the Instagram feed and mental health history of 166 participants, half of whom were reported to have been clinically depressed in the last three years.

They analysed around 44,000 photos and used findings from well-established psychology research to look for clues in Instagram images that could reveal the user’s state of mind.

“Pixel analysis of the photos in our dataset revealed that depressed individuals in our sample tended to post photos that were, on average, bluer, darker and greyer than those posted by healthy individuals,” the researchers wrote in a blog post.

Pair of deckchairs.
(Lizzie_Lamont/Getty Images/iStockphoto)
Previous studies have shown those who live with depression are more likely to use black-and-white filters in their photos (Lizzie Lamont/Getty Images)

They also found that people without depression opted for warmer and brighter photo filters, like Valencia, while black-and-white filters, such as Inkwell, were more popular among depressed people.

“In other words, people suffering from depression were more likely to favour a filter that literally drained all the colour out of the images they wanted to share,” the scientists continued.

In addition, the researchers also looked at photos that included people and found that depressed individuals were more likely to post selfies when compared with non-depressed people.

However, these photos had fewer faces on average than the healthy people’s Instagram feeds which, the scientists say, corresponds to existing research linking depression to reduced social interaction.

Young people taking a selfie while eating pizza.
(jacoblund/Getty Images/iStockphoto)
Scientists say those who are depressed are less likely to have large group photos on their Instagram feed (Jacoblund/Getty Images)

The researchers next developed a machine learning algorithm based on established studies and pitted it against volunteers to see who was better at identifying Instagram photos posted by depressed people.

The scientists claim that artificial intelligence performed better, with Danforth saying: “Obviously you know your friends better than a computer.

“But you might not, as a person casually flipping through Instagram, be as good at detecting depression as you think.”

The scientists say their new study also showed that the algorithm was able to detect signs of depression before a person’s date of diagnosis.

People using their mobile phones.
(Lauren Hurley/PA Wire/PA Images)
According to the researchers, the algorithm’s detection rate for identifying depressed people is 70% (Lauren Hurley/PA)

Danforth points out that while their research holds promise, the technology is still far from perfect.

“This study is not yet a diagnostic test, not by a long shot,” he said. “But it is a proof of concept of a new way to help people.”

Figures from the mental health charity Mind show one in six people reports experiencing a common mental health problem (such as anxiety and depression) in any given week, while one in four in the UK will experience a mental health problem each year.

The research is published in PJ Data Science.