Saturday, April 13, 2024
HomeSECURITYArtificial intelligence can recognize materials by one pixel

Artificial intelligence can recognize materials by one pixel


Artificial intelligence can recognize materials by one pixel

Scientists at MIT and Adobe Research have developed a method that can determine what objects in photos and videos are made of by selecting a single pixel.

It is not difficult for a person to distinguish wood from metal, leather from fabric, glass from plastic, but for a machine it is a difficult task. After all, the same material can look different depending on the shape, size, lighting or shadow of the object.

Scientists from MIT and Adobe Research found solution to this problem. They created a method that allows artificial intelligence to identify all objects in an image or video that are made from the same material. To do this, it is enough to select one pixel representing the material of interest.

This method can be useful for robots that work with different objects in different conditions. For example, a robot chef will be able to choose the right force to lift or cut food from different materials. It can also help with image editing, defining content parameters, or web content recommendations.

The method works even with different lighting or the shape of objects, which can change the appearance of the material. The machine learning model is not confused by shadows or reflections.

The scientists trained the model on synthetic data consisting of 50,000 images and more than 16,000 materials. However, the model also does well with real scenes that she has not seen before. The method also works for video.

“Knowing what material you are interacting with is often very important. Our method can make it easier to select all the other pixels in the image that are made of the same material,” says Prafull Sharma, lead author of the paper.

The study will be presented at the SIGGRAPH 2023 conference.

How does the method work?

The basis of the method is a machine learning model that transforms ordinary visual features into material-specific ones. To do this, she uses a pre-trained computer vision model that has already seen millions of real images.

The model calculates a material similarity ratio for each pixel in the image. When the user selects a pixel, the model determines how close each other pixel looks to the request. A map is then created where each pixel is scored from 0 to 1 in similarity.

The user can customize the results by setting a threshold, such as 90 percent similarity, and get an image map with highlighted areas. The method also works for selecting similar materials in another image.

During the experiments, the scientists found that the model can predict areas of the image that are made up of the same material more accurately than other methods. When comparing the prediction with reality, that is, real image areas consisting of the same material, the accuracy of the model was about 92 percent.

In the future, scientists want to improve the model so that it can better capture the fine details of objects in the image, which will increase the accuracy of the approach.

Source link


Please enter your comment!
Please enter your name here

Most Popular