Utils¶

Load¶

Load an image from a url¶

load_image_from_url(url, save_filename=None, s3_client=None, gcp_client=None)[source]¶

Load an image from url.

Parameters:

url (str) – The image url.
save_filename (Optional[str]) – The filename to save to.
s3_client (Optional[Any]) – A boto3 S3 client, e.g. s3_client = boto3.client("s3"). Needs to be provided if your images are in a private S3 bucket. Defaults to None.
gcp_client (Optional[Any]) – A Google Gloud Storage client, e.g. gcp_client = storage.Client(). Needs to be provided if your images are in a private GCP bucket. Defaults to None.

Return type:

Image

Returns:

A PIL image.

Load a point cloud from a url¶

load_pointcloud_from_url(url, save_filename=None, s3_client=None, gcp_client=None)[source]¶

Load a point cloud from a url.

Parameters:

url (str) – The point cloud url.
save_filename (Optional[str]) – The filename to save to.
s3_client (Optional[Any]) – A boto3 S3 client, e.g. s3_client = boto3.client("s3"). Needs to be provided if your point clouds are in a private S3 bucket. Defaults to None.
gcp_client (Optional[Any]) – A Google Gloud Storage client, e.g. gcp_client = storage.Client(). Needs to be provided if your images are in a private GCP bucket. Defaults to None.

Return type:

o3d.geometry.PointCloud

Returns:

A point cloud.

Raises:

ImportError – If open3d is not installed (to install run pip install open3d).

Load a label bitmap from a url¶

load_label_bitmap_from_url(url, save_filename=None)[source]¶

Load a label bitmap from url.

Parameters:

url (str) – The label bitmap url.
save_filename (Optional[str]) – The filename to save to.

Return type:

ndarray[tuple[Any, ...], dtype[uintc]]

Returns:

A numpy.ndarray with numpy.uint32 dtype where each unique value represents an instance id.

Load a release¶

load_release(release)[source]¶

Load JSON from Segments release.

Parameters:: release (Release) – A Segments release.
Return type:: Any
Returns:: A JSON with the release labels.

Transform¶

Turn a bitmap into a file¶

bitmap2file(bitmap, is_segmentation_bitmap=True)[source]¶

Convert a label bitmap to a file with the proper format.

Parameters:

bitmap (ndarray[tuple[Any, ...], dtype[uintc]]) – A numpy.ndarray with numpy.uint32 dtype where each unique value represents an instance id.
is_segmentation_bitmap (bool) – If this is a segmentation bitmap. Defaults to True.

Return type:

BytesIO

Returns:

A file object.

Raises:

ValueError – If the dtype is not np.uint32 or np.uint8.
ValueError – If the bitmap is not a segmentation bitmap.

Turn an instance bitmap and annotations dict into a semantic bitmap¶

get_semantic_bitmap(instance_bitmap=None, annotations=None, id_increment=0)[source]¶

Convert an instance bitmap and annotations dict into a segmentation bitmap.

Parameters:

instance_bitmap – A numpy.ndarray with numpy.uint32 dtype where each unique value represents an instance id. Defaults to None.
annotations – An annotations dictionary. Defaults to None.
id_increment – Increment the category ids with this number. Defaults to 0.

Returns:

An array here each unique value represents a category id.

Return type:

Optional[npt.NDArray[np.uint32]]

Fix an image rotation¶

handle_exif_rotation(image)[source]¶

Handle the exif rotation of a PIL image.

Parameters:: image (Image) – A PIL image.
Return type:: Image
Returns:: A rotated PIL image.

Turn a cuboid label into a segmentation label¶

cuboid_to_segmentation(pointcloud, label_attributes, ego_pose=None)[source]¶

Convert a cuboid label to an instance segmentation label.

Parameters:

pointcloud (ndarray[tuple[Any, ...], dtype[single]]) – A point cloud of size Nx3.
label_attributes (PointcloudCuboidLabelAttributes) – A cuboid label from a single frame interface or one frame from a sequence interface.

Return type:

ndarray[tuple[Any, ...], dtype[uintc]]

Returns:

An instance segmentation label of size Nx1 mapping each point cloud point to a cuboid instance.

Raises:

ImportError – If pyquaternion is not installed (to install run pip install pyquaternion).
ImportError – If open3d is not installed (to install run pip install open3d).

Turn a numpy array into a point cloud¶

array_to_pcd(positions, output_path, intensity=None, rgb=None, compressed=False, write_ascii=True)[source]¶

Convert a numpy array to a pcd file.

Parameters:

positions (ndarray[tuple[Any, ...], dtype[single]]) – Array of xyz points (Nx3 shape).
output_path (str) – Path to write the pcd.
intensity (Optional[ndarray[tuple[Any, ...], dtype[single]]]) – Optional array of intensity values (Nx1 shape).
rgb (Optional[ndarray[tuple[Any, ...], dtype[single]]]) – Optional array of rgb values (Nx3 shape) where red, green and blue are values between 0 and 255 or 0 and 1.
compressed (bool) – If the pcd should be compressed. Defaults to False.
write_ascii (bool) – If the pcd should be written in ascii format. Defaults to True.

Return type:

None

Returns:

None

Raises:

ImportError – If open3d is not installed (to install run pip install open3d).
AssertionError – If the positions array does not have shape (N, 3).
AssertionError – If the intensity array does not have shape (N, 1) or (N,).
AssertionError – If the rgb array does not have shape (N, 3).
AssertionError – If the intensity array does not have the same length as the positions array.
AssertionError – If the rgb array does not have the same length as the positions array.

Turn a ply file into a pcd file¶

ply_to_pcd(ply_file, compressed=False, write_ascii=True)[source]¶

Convert a .ply file to a .pcd file.

Parameters:

ply_file (str) – The path to the .ply file.
compressed (bool) – If the pcd should be compressed. Defaults to False.
write_ascii (bool) – If the pcd should be written in ascii format. Defaults to True.

Return type:

None

Returns:

None

Raises:

ImportError – If plyfile is not installed (to install run pip install plyfile).
KeyError – If the positions are not found in the ply file (expected colum names are x, y and z).

Turn a point cloud into a sampled point cloud¶

sample_pcd(pcd_path, points=500000, output_path=None)[source]¶

Sample a point cloud to a given number of points.

Parameters:

pcd_path (str) – The path to the point cloud.
points (int) – The number of points to sample. Defaults to 500_000.
output_path (Optional[str]) – The path to save the sampled point cloud to. Defaults to None.

Return type:

None

Returns:

None

Raises:

ImportError – If open3d is not installed (to install run pip install open3d).

Export¶

Export a dataset to a different format¶

export_dataset(dataset, export_folder='.', export_format=ExportFormat.COCO_PANOPTIC, id_increment=0, **kwargs)[source]¶

Export a dataset to a different format.

Export format	Supported dataset type
COCO panoptic	`segmentation-bitmap` and `segmentation-bitmap-highres`
COCO instance	`segmentation-bitmap` and `segmentation-bitmap-highres`
YOLO	`vector`, `bboxes` and `image-vector-sequence`
Instance	`segmentation-bitmap` and `segmentation-bitmap-highres`
Colored instance	`segmentation-bitmap` and `segmentation-bitmap-highres`
Semantic	`segmentation-bitmap` and `segmentation-bitmap-highres`
Colored semantic	`segmentation-bitmap` and `segmentation-bitmap-highres`
Polygon	`segmentation-bitmap` and `segmentation-bitmap-highres`

Example:

# pip install segments-ai
from segments import SegmentsClient, SegmentsDataset
from segments.utils import export_dataset

# Initialize a SegmentsDataset from the release file
client = SegmentsClient('YOUR_API_KEY')
release = client.get_release('jane/flowers', 'v1.0') # Alternatively: release = 'flowers-v1.0.json'
dataset = SegmentsDataset(release, labelset='ground-truth', filter_by=['labeled', 'reviewed'])

# Export to COCO panoptic format
export_dataset(dataset, export_format='coco-panoptic')

Alternatively, you can use the initialized SegmentsDataset to loop through the samples and labels, and visualize or process them in any way you please:

import matplotlib.pyplot as plt
from segments.utils import get_semantic_bitmap

for sample in dataset:
    # Print the sample name and list of labeled objects
    print(sample['name'])
    print(sample['annotations'])

    # Show the image
    plt.imshow(sample['image'])
    plt.show()

    # Show the instance segmentation label
    plt.imshow(sample['segmentation_bitmap'])
    plt.show()

    # Show the semantic segmentation label
    semantic_bitmap = get_semantic_bitmap(sample['segmentation_bitmap'], sample['annotations'])
    plt.imshow(semantic_bitmap)
    plt.show()

Parameters:

dataset (SegmentsDataset) – A SegmentsDataset.
export_folder (str) – The folder to export the dataset to. Defaults to ..
export_format (ExportFormat) – The destination format. Defaults to coco-panoptic.
id_increment (int) – Increment the category ids with this number. Defaults to 0. Ignored unless export_format is semantic or semantic-color.

Return type:

Optional[Union[Tuple[str, Optional[str]], Optional[str]]]

Returns:

Returns the file name and the image directory name (for COCO panoptic, COCO instance, YOLO and polygon), or returns the export folder name (for (colored) instance and (colored) panoptic).

Raises:

ImportError – If scikit image is not installed (to install run pip install scikit-image).
ValueError – If an unvalid export_format is used.

Show¶

Show the exported contours of a segmented image¶

show_polygons(image_directory_path, image_id, exported_polygons_path, seed=0, output_path=None)[source]¶

Show the exported contours of a segmented image (i.e., resulting from export_dataset() with polygon export format).

Parameters:

image_directory_path (str) – The image directory path.
image_id (int) – The image id (this can be found in the exported polygons JSON file).
exported_polygons_path (str) – The exported polygons path.
seed (int) – The seed used to generate random colors. Defaults to 0.
output_path (Optional[str]) – The directory to save the plot to. Defaults to None.

Return type:

None

Returns:

None

Raises:

ImportError – If matplotlib is not installed.

Try all camera rotations to find the best one¶

find_camera_rotation(client, dataset_identifier)[source]¶

Find the correct camera rotation by trying all possibilities.

Parameters:

client (SegmentsClient) – A Segments client.
dataset_identifier (str) – The dataset identifier.

Return type:

Quaternion

Returns:

A pyquaternion.Quaternion representing the correct rotation.

Raises:

ImportError – If pyquaternion is not installed (to install run pip install pyquaternion).
ValueError – If the dataset is not a point cloud sequence dataset.
ValueError – If the user answers neither y(es) nor n(o) (case insensitive).
ValueError – If the correct rotation is not found.
AlreadyExistsError – If the cloned dataset already exists.

Kinematics¶

Add velocity and acceleration calculations to sequence labels¶

add_kinematics_to_label(sample, label, timestamp_unit=1000000000.0, fps=None)[source]¶

Add velocity and acceleration calculations to point cloud sequence cuboid labels.

This function calculates velocity and acceleration (relative and absolute) for annotations with position, ego_position and timestamp data across frames in point cloud sequence datasets. If timestamp data is missing, it estimates a frame per 0.5 seconds. If ego-position data is missing, only absolute kinematics are calculated.

Parameters:

sample (Sample) – A Sample object containing the point cloud sequence sample data.
label (Label) – A Label object containing point cloud sequence data.
timestamp_unit (Optional[int]) – Conversion factor for timestamps to seconds. Defaults to 1_000_000_000.0 (i.e., timestamps in nanoseconds).
fps (Optional[float]) – Frames per second to use when timestamps are missing.

Return type:

Label

Returns:

A copy of the label with velocity and acceleration attributes added to annotations.

Raises:

ValueError – If the sample is not a multisensor or point cloud sequence cuboid label.
ValueError – If the label sample UUID does not match the sample UUID.
ValueError – If fps is not provided when timestamps are missing.

Example

# pip install segments-ai
from segments import SegmentsClient
from segments.utils import add_kinematics_to_label

client = SegmentsClient('YOUR_API_KEY')
sample = client.get_sample(sample_uuid)
label = client.get_label(sample_uuid)
updated_label = add_kinematics_to_label(sample, label)