Finding clusters in an image. Go beyond finding clusters in tabular… | by Pranay Dave | Nov, 2022
Go beyond finding clusters in tabular data
We are also comfortable in finding clusters in rows and columns of data. But how about finding clusters within an image? Let me illustrate the subject using an example of an image from recent world cup football 2022 in Qatar.
Shown here is a photo which I had taken during Brazil vs Serbia match. The photo has been taken just before the stunning goal from Richarlison.
Can you observe any clusters in the photo? Visually, you can locate a cluster of persons as shown below.
Now let me give you the technical steps on how to identify the cluster automatically.
The first step is to detect all objects in the image. What objects to detect will be based on the objective of clustering. Here our objective is to detect clusters of persons. Detecting persons in a photo can be done YOLOv5. The results of object detection using YOLOv5 are shown below.
YOLOv5 also gives additional statistics such as the number of object detection. In this photo, there are 17 objects (persons). Here is a snippet of the code
import yolov5
from sklearn.cluster import DBSCAN
from sklearn.preprocessing import StandardScaler
import numpy as np
import pandas as pd# load pretrained model
model = yolov5.load('yolov5s.pt')
# set model parameters
model.conf = 0.25 # NMS confidence threshold
model.iou = 0.45 # NMS IoU threshold
model.agnostic = False # NMS class-agnostic
model.multi_label = False # NMS multiple labels per box
model.max_det = 1000 # maximum number of detections per image
# set image
img = 'brazil_vs_serbian.png'
# inference with test time augmentation
results = model(img, augment=True)
# parse results
predictions = results.pred[0]
boxes = predictions[:, :4] # x1, y1, x2, y2
scores = predictions[:, 4]
categories = predictions[:, 5]
The next step is to identify the X and Y location of objects. The YOLOv5 results stores the location of each object in the image. The locations are stored as xmin, ymin, xmax, ymax. We can take the average of xmin and xmax, as well as ymin and ymax, to find the x, y location of the object.
One can convert the coordinates into a data frame. There is one tweak that is important to make, which is related to inverting the Y location. Generally, the image is stored with Y-axis inverted. This means that the Y=0 is top of the image. During the calculation, we can revert the location by subtracting from the height of the image (here it is 2253). The height can be determined automatically also, but I have directly put the height in order to simplify the code snippet.
df = results.pandas().xyxy[0]
df['x'] = (df['xmin'] + df['xmax'])/2.0
df['y'] = (2253-(df['ymin'] + df['ymax'])/2.0) #2253 is image height
The result is shown here as dataframe.
Now we are all set to find the clusters. Finding clusters in the image means finding areas where there are more objects compared to other parts of the image. Essentially, this means finding clusters based on the density of the objects in the image. One of the efficient algorithms is density-based clustering (DBSCAN). Shown below are the results plotted in a scatter plot.
The green dots are identified as part of a dense area by the DBSCAN algorithm. Here is the code snippet for DBSCAN algorithm.
#Density clustering
X = StandardScaler().fit_transform(df[['x','y']])
clustering = DBSCAN(eps=0.5, min_samples=5).fit(X)
df['c'] = clustering.labels_
Let us now analyze the results on the scatter plot compared to the photo.
Voila! scatterplot looks like a visually digital representation of the football match!
Finding clusters in an image is an interesting technique, as well as many use cases. In the blog, you saw a use case of sports analytics. In addition, there are various other use cases such as identifying over-crowding in a public space or identifying areas where shoppers spend more time in a shopping complex.
Hope you enjoyed the blog!
Please join Medium with my referral link.
Please subscribe to stay informed whenever I release a new story.
Website
You can visit my website which is a no-code platform to learn data science. https://experiencedatascience.com
Youtube channel
Here is a link to my YouTube channel
https://www.youtube.com/c/DataScienceDemonstrated
Go beyond finding clusters in tabular data
We are also comfortable in finding clusters in rows and columns of data. But how about finding clusters within an image? Let me illustrate the subject using an example of an image from recent world cup football 2022 in Qatar.
Shown here is a photo which I had taken during Brazil vs Serbia match. The photo has been taken just before the stunning goal from Richarlison.
Can you observe any clusters in the photo? Visually, you can locate a cluster of persons as shown below.
Now let me give you the technical steps on how to identify the cluster automatically.
The first step is to detect all objects in the image. What objects to detect will be based on the objective of clustering. Here our objective is to detect clusters of persons. Detecting persons in a photo can be done YOLOv5. The results of object detection using YOLOv5 are shown below.
YOLOv5 also gives additional statistics such as the number of object detection. In this photo, there are 17 objects (persons). Here is a snippet of the code
import yolov5
from sklearn.cluster import DBSCAN
from sklearn.preprocessing import StandardScaler
import numpy as np
import pandas as pd# load pretrained model
model = yolov5.load('yolov5s.pt')
# set model parameters
model.conf = 0.25 # NMS confidence threshold
model.iou = 0.45 # NMS IoU threshold
model.agnostic = False # NMS class-agnostic
model.multi_label = False # NMS multiple labels per box
model.max_det = 1000 # maximum number of detections per image
# set image
img = 'brazil_vs_serbian.png'
# inference with test time augmentation
results = model(img, augment=True)
# parse results
predictions = results.pred[0]
boxes = predictions[:, :4] # x1, y1, x2, y2
scores = predictions[:, 4]
categories = predictions[:, 5]
The next step is to identify the X and Y location of objects. The YOLOv5 results stores the location of each object in the image. The locations are stored as xmin, ymin, xmax, ymax. We can take the average of xmin and xmax, as well as ymin and ymax, to find the x, y location of the object.
One can convert the coordinates into a data frame. There is one tweak that is important to make, which is related to inverting the Y location. Generally, the image is stored with Y-axis inverted. This means that the Y=0 is top of the image. During the calculation, we can revert the location by subtracting from the height of the image (here it is 2253). The height can be determined automatically also, but I have directly put the height in order to simplify the code snippet.
df = results.pandas().xyxy[0]
df['x'] = (df['xmin'] + df['xmax'])/2.0
df['y'] = (2253-(df['ymin'] + df['ymax'])/2.0) #2253 is image height
The result is shown here as dataframe.
Now we are all set to find the clusters. Finding clusters in the image means finding areas where there are more objects compared to other parts of the image. Essentially, this means finding clusters based on the density of the objects in the image. One of the efficient algorithms is density-based clustering (DBSCAN). Shown below are the results plotted in a scatter plot.
The green dots are identified as part of a dense area by the DBSCAN algorithm. Here is the code snippet for DBSCAN algorithm.
#Density clustering
X = StandardScaler().fit_transform(df[['x','y']])
clustering = DBSCAN(eps=0.5, min_samples=5).fit(X)
df['c'] = clustering.labels_
Let us now analyze the results on the scatter plot compared to the photo.
Voila! scatterplot looks like a visually digital representation of the football match!
Finding clusters in an image is an interesting technique, as well as many use cases. In the blog, you saw a use case of sports analytics. In addition, there are various other use cases such as identifying over-crowding in a public space or identifying areas where shoppers spend more time in a shopping complex.
Hope you enjoyed the blog!
Please join Medium with my referral link.
Please subscribe to stay informed whenever I release a new story.
Website
You can visit my website which is a no-code platform to learn data science. https://experiencedatascience.com
Youtube channel
Here is a link to my YouTube channel
https://www.youtube.com/c/DataScienceDemonstrated
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.