Imagine yourself pointing a light on the right side of the car model. All the points that fall on the right outer surface of the 3D model would be illuminated, while all the other points in the point cloud would not.
Let us now label these illuminated points as “visible” and all the non-illuminated points as “hidden”. These “hidden” points would also include all the points that belong to the interior of the car.
This operation is known as Hidden Point Removal in Open3D. In order to perform this operation on the point cloud using Open3D, run the following lines of code:
# Defining the camera and radius parameters for the hidden point removal operation.
diameter = np.linalg.norm(np.asarray(pcd.get_min_bound()) - np.asarray(pcd.get_max_bound()))
camera = [0, 0, diameter]
radius = diameter * 100# Performing the hidden point removal operation on the point cloud using the
# camera and radius parameters defined above.
# The output is a list of indexes of points that are visible.
_, pt_map = pcd.hidden_point_removal(camera, radius)
Using the above output list of indexes of points that are visible, we can colour the visible and hidden points in different colours before visualizing the point cloud.
# Painting all the visible points in the point cloud in blue, and all the hidden points in red.pcd_visible = pcd.select_by_index(pt_map)
pcd_visible.paint_uniform_color([0, 0, 1]) # Blue points are visible points (to be kept).
print("No. of visible points : ", pcd_visible)
pcd_hidden = pcd.select_by_index(pt_map, invert=True)
pcd_hidden.paint_uniform_color([1, 0, 0]) # Red points are hidden points (to be removed).
print("No. of hidden points : ", pcd_hidden)
# Visualizing the visible (blue) and hidden (red) points in the point cloud.
draw_geoms_list = [mesh_coord_frame, pcd_visible, pcd_hidden]
o3d.visualization.draw_geometries(draw_geoms_list)
From the visualization above, we see how the hidden point removal operation works from a given camera viewpoint. The operation eliminates all the points in the background that are occluded by the points in the foreground from a given camera viewpoint.
To understand this better, we could repeat the same operation again, but this time after rotating the point cloud slightly. Effectively, we’re trying to change the viewpoint here. But instead of changing it by re-defining the camera parameters, we will be rotating the point cloud itself.
# Defining a function to convert degrees to radians.
def deg2rad(deg):
return deg * np.pi/180# Rotating the point cloud about the X-axis by 90 degrees.
x_theta = deg2rad(90)
y_theta = deg2rad(0)
z_theta = deg2rad(0)
tmp_pcd_r = copy.deepcopy(pcd)
R = tmp_pcd_r.get_rotation_matrix_from_axis_angle([x_theta, y_theta, z_theta])
tmp_pcd_r.rotate(R, center=(0, 0, 0))
# Visualizing the rotated point cloud.
draw_geoms_list = [mesh_coord_frame, tmp_pcd_r]
o3d.visualization.draw_geometries(draw_geoms_list)
By repeating the same process again with the rotated car model, we would see that this time all the points that fall on the upper outer surface of the 3D model (roof of the car) would get illuminated, while all the other points in the point cloud would not.
We can repeat the hidden point removal operation with the rotated point cloud by running the following lines of code:
# Performing the hidden point removal operation on the rotated point cloud
# using the same camera and radius parameters defined above.
# The output is a list of indexes of points that are visible.
_, pt_map = tmp_pcd_r.hidden_point_removal(camera, radius)# Painting all the visible points in the rotated point cloud in blue,
# and all the hidden points in red.
pcd_visible = tmp_pcd_r.select_by_index(pt_map)
pcd_visible.paint_uniform_color([0, 0, 1]) # Blue points are visible points (to be kept).
print("No. of visible points : ", pcd_visible)
pcd_hidden = tmp_pcd_r.select_by_index(pt_map, invert=True)
pcd_hidden.paint_uniform_color([1, 0, 0]) # Red points are hidden points (to be removed).
print("No. of hidden points : ", pcd_hidden)
# Visualizing the visible (blue) and hidden (red) points in the rotated point cloud.
draw_geoms_list = [mesh_coord_frame, pcd_visible, pcd_hidden]
o3d.visualization.draw_geometries(draw_geoms_list)
The above visualization of the rotated point cloud clearly illustrates how the hidden point removal operation works. So now, in order to remove all the “hidden” points from this car point cloud, we can perform this hidden point removal operation sequentially by rotating the point cloud slightly about all the three axes from -90 to +90 degrees. After each hidden point removal operation, we can aggregate the output list of indexes of points. After all the hidden point removal operations, the aggregated list of indexes of points will contain all the points that are not hidden (ie., points that are on the outer surface of the point cloud). The following code performs this sequential hidden point removal operation:
# Defining a function to rotate a point cloud in X, Y and Z-axis.
def get_rotated_pcd(pcd, x_theta, y_theta, z_theta):
pcd_rotated = copy.deepcopy(pcd)
R = pcd_rotated.get_rotation_matrix_from_axis_angle([x_theta, y_theta, z_theta])
pcd_rotated.rotate(R, center=(0, 0, 0))
return pcd_rotated# Defining a function to get the camera and radius parameters for the point cloud
# for the hidden point removal operation.
def get_hpr_camera_radius(pcd):
diameter = np.linalg.norm(np.asarray(pcd.get_min_bound()) - np.asarray(pcd.get_max_bound()))
camera = [0, 0, diameter]
radius = diameter * 100
return camera, radius
# Defining a function to perform the hidden point removal operation on the
# point cloud using the camera and radius parameters defined earlier.
# The output is a list of indexes of points that are not hidden.
def get_hpr_pt_map(pcd, camera, radius):
_, pt_map = pcd.hidden_point_removal(camera, radius)
return pt_map
# Performing the hidden point removal operation sequentially by rotating the
# point cloud slightly in each of the three axes from -90 to +90 degrees,
# and aggregating the list of indexes of points that are not hidden after
# each operation.# Defining a list to store the aggregated output lists from each hidden
# point removal operation.
pt_map_aggregated = []
# Defining the steps and range of angle values by which to rotate the point cloud.
theta_range = np.linspace(-90, 90, 7)
# Counting the number of sequential operations.
view_counter = 1
total_views = theta_range.shape[0] ** 3
# Obtaining the camera and radius parameters for the hidden point removal operation.
camera, radius = get_hpr_camera_radius(pcd)
# Looping through the angle values defined above for each axis.
for x_theta_deg in theta_range:
for y_theta_deg in theta_range:
for z_theta_deg in theta_range:
print(f"Removing hidden points - processing view {view_counter} of {total_views}.")
# Rotating the point cloud by the given angle values.
x_theta = deg2rad(x_theta_deg)
y_theta = deg2rad(y_theta_deg)
z_theta = deg2rad(z_theta_deg)
pcd_rotated = get_rotated_pcd(pcd, x_theta, y_theta, z_theta)
# Performing the hidden point removal operation on the rotated
# point cloud using the camera and radius parameters defined above.
pt_map = get_hpr_pt_map(pcd_rotated, camera, radius)
# Aggregating the output list of indexes of points that are not hidden.
pt_map_aggregated += pt_map
view_counter += 1
# Removing all the duplicated points from the aggregated list by converting it to a set.
pt_map_aggregated = list(set(pt_map_aggregated))
# Painting all the visible points in the point cloud in blue, and all the hidden points in red.pcd_visible = pcd.select_by_index(pt_map_aggregated)
pcd_visible.paint_uniform_color([0, 0, 1]) # Blue points are visible points (to be kept).
print("No. of visible points : ", pcd_visible)
pcd_hidden = pcd.select_by_index(pt_map_aggregated, invert=True)
pcd_hidden.paint_uniform_color([1, 0, 0]) # Red points are hidden points (to be removed).
print("No. of hidden points : ", pcd_hidden)
# Visualizing the visible (blue) and hidden (red) points in the point cloud.
draw_geoms_list = [mesh_coord_frame, pcd_visible, pcd_hidden]
# draw_geoms_list = [mesh_coord_frame, pcd_visible]
# draw_geoms_list = [mesh_coord_frame, pcd_hidden]
o3d.visualization.draw_geometries(draw_geoms_list)
Let’s crop the point cloud again to have a look at the points which belong to the interior of the car.
# Cropping the point cloud of visible points using bounding box defined
# earlier to remove its right half (positive Z-axis).
pcd_visible_cropped = pcd_visible.crop(bbox_cropped)# Cropping the point cloud of hidden points using bounding box defined
# earlier to remove its right half (positive Z-axis).
pcd_hidden_cropped = pcd_hidden.crop(bbox_cropped)
# Visualizing the cropped point clouds.
draw_geoms_list = [mesh_coord_frame, pcd_visible_cropped, pcd_hidden_cropped]
o3d.visualization.draw_geometries(draw_geoms_list)
From the above visualization of the cropped point cloud after the hidden point removal operation, we see that all the “hidden” points which belong to the interior of the car model (red) are now separated from the “visible” points which are on the outer surface of the point cloud (blue).
Imagine yourself pointing a light on the right side of the car model. All the points that fall on the right outer surface of the 3D model would be illuminated, while all the other points in the point cloud would not.
Let us now label these illuminated points as “visible” and all the non-illuminated points as “hidden”. These “hidden” points would also include all the points that belong to the interior of the car.
This operation is known as Hidden Point Removal in Open3D. In order to perform this operation on the point cloud using Open3D, run the following lines of code:
# Defining the camera and radius parameters for the hidden point removal operation.
diameter = np.linalg.norm(np.asarray(pcd.get_min_bound()) - np.asarray(pcd.get_max_bound()))
camera = [0, 0, diameter]
radius = diameter * 100# Performing the hidden point removal operation on the point cloud using the
# camera and radius parameters defined above.
# The output is a list of indexes of points that are visible.
_, pt_map = pcd.hidden_point_removal(camera, radius)
Using the above output list of indexes of points that are visible, we can colour the visible and hidden points in different colours before visualizing the point cloud.
# Painting all the visible points in the point cloud in blue, and all the hidden points in red.pcd_visible = pcd.select_by_index(pt_map)
pcd_visible.paint_uniform_color([0, 0, 1]) # Blue points are visible points (to be kept).
print("No. of visible points : ", pcd_visible)
pcd_hidden = pcd.select_by_index(pt_map, invert=True)
pcd_hidden.paint_uniform_color([1, 0, 0]) # Red points are hidden points (to be removed).
print("No. of hidden points : ", pcd_hidden)
# Visualizing the visible (blue) and hidden (red) points in the point cloud.
draw_geoms_list = [mesh_coord_frame, pcd_visible, pcd_hidden]
o3d.visualization.draw_geometries(draw_geoms_list)
From the visualization above, we see how the hidden point removal operation works from a given camera viewpoint. The operation eliminates all the points in the background that are occluded by the points in the foreground from a given camera viewpoint.
To understand this better, we could repeat the same operation again, but this time after rotating the point cloud slightly. Effectively, we’re trying to change the viewpoint here. But instead of changing it by re-defining the camera parameters, we will be rotating the point cloud itself.
# Defining a function to convert degrees to radians.
def deg2rad(deg):
return deg * np.pi/180# Rotating the point cloud about the X-axis by 90 degrees.
x_theta = deg2rad(90)
y_theta = deg2rad(0)
z_theta = deg2rad(0)
tmp_pcd_r = copy.deepcopy(pcd)
R = tmp_pcd_r.get_rotation_matrix_from_axis_angle([x_theta, y_theta, z_theta])
tmp_pcd_r.rotate(R, center=(0, 0, 0))
# Visualizing the rotated point cloud.
draw_geoms_list = [mesh_coord_frame, tmp_pcd_r]
o3d.visualization.draw_geometries(draw_geoms_list)
By repeating the same process again with the rotated car model, we would see that this time all the points that fall on the upper outer surface of the 3D model (roof of the car) would get illuminated, while all the other points in the point cloud would not.
We can repeat the hidden point removal operation with the rotated point cloud by running the following lines of code:
# Performing the hidden point removal operation on the rotated point cloud
# using the same camera and radius parameters defined above.
# The output is a list of indexes of points that are visible.
_, pt_map = tmp_pcd_r.hidden_point_removal(camera, radius)# Painting all the visible points in the rotated point cloud in blue,
# and all the hidden points in red.
pcd_visible = tmp_pcd_r.select_by_index(pt_map)
pcd_visible.paint_uniform_color([0, 0, 1]) # Blue points are visible points (to be kept).
print("No. of visible points : ", pcd_visible)
pcd_hidden = tmp_pcd_r.select_by_index(pt_map, invert=True)
pcd_hidden.paint_uniform_color([1, 0, 0]) # Red points are hidden points (to be removed).
print("No. of hidden points : ", pcd_hidden)
# Visualizing the visible (blue) and hidden (red) points in the rotated point cloud.
draw_geoms_list = [mesh_coord_frame, pcd_visible, pcd_hidden]
o3d.visualization.draw_geometries(draw_geoms_list)
The above visualization of the rotated point cloud clearly illustrates how the hidden point removal operation works. So now, in order to remove all the “hidden” points from this car point cloud, we can perform this hidden point removal operation sequentially by rotating the point cloud slightly about all the three axes from -90 to +90 degrees. After each hidden point removal operation, we can aggregate the output list of indexes of points. After all the hidden point removal operations, the aggregated list of indexes of points will contain all the points that are not hidden (ie., points that are on the outer surface of the point cloud). The following code performs this sequential hidden point removal operation:
# Defining a function to rotate a point cloud in X, Y and Z-axis.
def get_rotated_pcd(pcd, x_theta, y_theta, z_theta):
pcd_rotated = copy.deepcopy(pcd)
R = pcd_rotated.get_rotation_matrix_from_axis_angle([x_theta, y_theta, z_theta])
pcd_rotated.rotate(R, center=(0, 0, 0))
return pcd_rotated# Defining a function to get the camera and radius parameters for the point cloud
# for the hidden point removal operation.
def get_hpr_camera_radius(pcd):
diameter = np.linalg.norm(np.asarray(pcd.get_min_bound()) - np.asarray(pcd.get_max_bound()))
camera = [0, 0, diameter]
radius = diameter * 100
return camera, radius
# Defining a function to perform the hidden point removal operation on the
# point cloud using the camera and radius parameters defined earlier.
# The output is a list of indexes of points that are not hidden.
def get_hpr_pt_map(pcd, camera, radius):
_, pt_map = pcd.hidden_point_removal(camera, radius)
return pt_map
# Performing the hidden point removal operation sequentially by rotating the
# point cloud slightly in each of the three axes from -90 to +90 degrees,
# and aggregating the list of indexes of points that are not hidden after
# each operation.# Defining a list to store the aggregated output lists from each hidden
# point removal operation.
pt_map_aggregated = []
# Defining the steps and range of angle values by which to rotate the point cloud.
theta_range = np.linspace(-90, 90, 7)
# Counting the number of sequential operations.
view_counter = 1
total_views = theta_range.shape[0] ** 3
# Obtaining the camera and radius parameters for the hidden point removal operation.
camera, radius = get_hpr_camera_radius(pcd)
# Looping through the angle values defined above for each axis.
for x_theta_deg in theta_range:
for y_theta_deg in theta_range:
for z_theta_deg in theta_range:
print(f"Removing hidden points - processing view {view_counter} of {total_views}.")
# Rotating the point cloud by the given angle values.
x_theta = deg2rad(x_theta_deg)
y_theta = deg2rad(y_theta_deg)
z_theta = deg2rad(z_theta_deg)
pcd_rotated = get_rotated_pcd(pcd, x_theta, y_theta, z_theta)
# Performing the hidden point removal operation on the rotated
# point cloud using the camera and radius parameters defined above.
pt_map = get_hpr_pt_map(pcd_rotated, camera, radius)
# Aggregating the output list of indexes of points that are not hidden.
pt_map_aggregated += pt_map
view_counter += 1
# Removing all the duplicated points from the aggregated list by converting it to a set.
pt_map_aggregated = list(set(pt_map_aggregated))
# Painting all the visible points in the point cloud in blue, and all the hidden points in red.pcd_visible = pcd.select_by_index(pt_map_aggregated)
pcd_visible.paint_uniform_color([0, 0, 1]) # Blue points are visible points (to be kept).
print("No. of visible points : ", pcd_visible)
pcd_hidden = pcd.select_by_index(pt_map_aggregated, invert=True)
pcd_hidden.paint_uniform_color([1, 0, 0]) # Red points are hidden points (to be removed).
print("No. of hidden points : ", pcd_hidden)
# Visualizing the visible (blue) and hidden (red) points in the point cloud.
draw_geoms_list = [mesh_coord_frame, pcd_visible, pcd_hidden]
# draw_geoms_list = [mesh_coord_frame, pcd_visible]
# draw_geoms_list = [mesh_coord_frame, pcd_hidden]
o3d.visualization.draw_geometries(draw_geoms_list)
Let’s crop the point cloud again to have a look at the points which belong to the interior of the car.
# Cropping the point cloud of visible points using bounding box defined
# earlier to remove its right half (positive Z-axis).
pcd_visible_cropped = pcd_visible.crop(bbox_cropped)# Cropping the point cloud of hidden points using bounding box defined
# earlier to remove its right half (positive Z-axis).
pcd_hidden_cropped = pcd_hidden.crop(bbox_cropped)
# Visualizing the cropped point clouds.
draw_geoms_list = [mesh_coord_frame, pcd_visible_cropped, pcd_hidden_cropped]
o3d.visualization.draw_geometries(draw_geoms_list)
From the above visualization of the cropped point cloud after the hidden point removal operation, we see that all the “hidden” points which belong to the interior of the car model (red) are now separated from the “visible” points which are on the outer surface of the point cloud (blue).