See also

Please visit our docs for more information on Segments.ai and visit the setup page to learn how to install and setup the Segments.ai Python SDK.

Utils#

Load#

Load an image from a url#

load_image_from_url(url, save_filename=None, s3_client=None)[source]#

Load an image from url.

Parameters

  • url (str) – The image url.

  • save_filename (Optional[str]) – The filename to save to.

  • s3_client (Optional[Any]) – A boto3 S3 client, e.g. s3_client = boto3.client("s3"). Needs to be provided if your images are in a private S3 bucket. Defaults to None.

Return type

Image

Returns

A PIL image.

Load a point cloud from a url#

load_pointcloud_from_url(url, save_filename=None, s3_client=None)[source]#

Load a point cloud from a url.

Parameters

  • url (str) – The point cloud url.

  • save_filename (Optional[str]) – The filename to save to.

  • s3_client (Optional[Any]) – A boto3 S3 client, e.g. s3_client = boto3.client("s3"). Needs to be provided if your point clouds are in a private S3 bucket. Defaults to None.

Return type

o3d.geometry.PointCloud

Returns

A point cloud.

Raises

ImportError – If open3d is not installed (to install run pip install open3d).

Load a label bitmap from a url#

load_label_bitmap_from_url(url, save_filename=None)[source]#

Load a label bitmap from url.

Parameters

  • url (str) – The label bitmap url.

  • save_filename (Optional[str]) – The filename to save to.

Return type

ndarray[Any, dtype[uint32]]

Returns

A numpy.ndarray with numpy.uint32 dtype where each unique value represents an instance id.

Load a release#

load_release(release)[source]#

Load JSON from Segments release.

Parameters

release (Release) – A Segments release.

Return type

Any

Returns

A JSON with the release labels.

Transform#

Turn a bitmap into a file#

bitmap2file(bitmap, is_segmentation_bitmap=True)[source]#

Convert a label bitmap to a file with the proper format.

Parameters

  • bitmap (ndarray[Any, dtype[uint32]]) – A numpy.ndarray with numpy.uint32 dtype where each unique value represents an instance id.

  • is_segmentation_bitmap (bool) – If this is a segmentation bitmap. Defaults to True.

Return type

BytesIO

Returns

A file object.

Raises

  • ValueError – If the dtype is not np.uint32 or np.uint8.

  • ValueError – If the bitmap is not a segmentation bitmap.

Turn an instance bitmap and annotations dict into a semantic bitmap#

get_semantic_bitmap(instance_bitmap=None, annotations=None, id_increment=0)[source]#

Convert an instance bitmap and annotations dict into a segmentation bitmap.

Parameters

  • instance_bitmap (Optional[ndarray[Any, dtype[uint32]]]) – A numpy.ndarray with numpy.uint32 dtype where each unique value represents an instance id. Defaults to None.

  • annotations (Optional[Dict[str, Any]]) – An annotations dictionary. Defaults to None.

  • id_increment (int) – Increment the category ids with this number. Defaults to 0.

Return type

Optional[ndarray[Any, dtype[uint32]]]

Returns

An array here each unique value represents a category id.

Fix an image rotation#

handle_exif_rotation(image)[source]#

Handle the exif rotation of a PIL image.

Parameters

image (Image) – A PIL image.

Return type

Image

Returns

A rotated PIL image.

Turn a cuboid label into a segmentation label#

cuboid_to_segmentation(pointcloud, label_attributes, ego_pose=None)[source]#

Convert a cuboid label to an instance segmentation label.

Parameters
Return type

ndarray[Any, dtype[uint32]]

Returns

An instance segmentation label of size Nx1 mapping each point cloud point to a cuboid instance.

Raises

  • ImportError – If pyquaternion is not installed (to install run pip install pyquaternion).

  • ImportError – If open3d is not installed (to install run pip install open3d).

Turn a numpy array into a point cloud#

array_to_pcd(positions, output_path, intensity=None, rgb=None, compressed=False, write_ascii=True)[source]#

Convert a numpy array to a pcd file.

Parameters

  • positions (ndarray[Any, dtype[float32]]) – Array of xyz points (Nx3 shape).

  • output_path (str) – Path to write the pcd.

  • intensity (Optional[ndarray[Any, dtype[float32]]]) – Optional array of intensity values (Nx1 shape).

  • rgb (Optional[ndarray[Any, dtype[float32]]]) – Optional array of rgb values (Nx3 shape) where red, green and blue are values between 0 and 255 or 0 and 1.

  • compressed (bool) – If the pcd should be compressed. Defaults to False.

  • write_ascii (bool) – If the pcd should be written in ascii format. Defaults to True.

Return type

None

Returns

None

Raises

  • ImportError – If open3d is not installed (to install run pip install open3d).

  • AssertionError – If the positions array does not have shape (N, 3).

  • AssertionError – If the intensity array does not have shape (N, 1) or (N,).

  • AssertionError – If the rgb array does not have shape (N, 3).

  • AssertionError – If the intensity array does not have the same length as the positions array.

  • AssertionError – If the rgb array does not have the same length as the positions array.

Turn a ply file into a pcd file#

ply_to_pcd(ply_file, compressed=False, write_ascii=True)[source]#

Convert a .ply file to a .pcd file.

Parameters

  • ply_file (str) – The path to the .ply file.

  • compressed (bool) – If the pcd should be compressed. Defaults to False.

  • write_ascii (bool) – If the pcd should be written in ascii format. Defaults to True.

Return type

None

Returns

None

Raises

  • ImportError – If plyfile is not installed (to install run pip install plyfile).

  • KeyError – If the positions are not found in the ply file (expected colum names are x, y and z).

Turn a point cloud into a sampled point cloud#

sample_pcd(pcd_path, points=500000, output_path=None)[source]#

Sample a point cloud to a given number of points.

Parameters

  • pcd_path (str) – The path to the point cloud.

  • points (int) – The number of points to sample. Defaults to 500_000.

  • output_path (Optional[str]) – The path to save the sampled point cloud to. Defaults to None.

Return type

None

Returns

None

Raises

ImportError – If open3d is not installed (to install run pip install open3d).

Export#

Export a dataset to a different format#

export_dataset(dataset, export_folder='.', export_format=ExportFormat.COCO_PANOPTIC, id_increment=0, **kwargs)[source]#

Export a dataset to a different format.

Export format

Supported dataset type

COCO panoptic

segmentation-bitmap and segmentation-bitmap-highres

COCO instance

segmentation-bitmap and segmentation-bitmap-highres

YOLO

vector, bboxes and image-vector-sequence

Instance

segmentation-bitmap and segmentation-bitmap-highres

Colored instance

segmentation-bitmap and segmentation-bitmap-highres

Semantic

segmentation-bitmap and segmentation-bitmap-highres

Colored semantic

segmentation-bitmap and segmentation-bitmap-highres

Polygon

segmentation-bitmap and segmentation-bitmap-highres

Example:

# pip install segments-ai
from segments import SegmentsClient, SegmentsDataset
from segments.utils import export_dataset

# Initialize a SegmentsDataset from the release file
client = SegmentsClient('YOUR_API_KEY')
release = client.get_release('jane/flowers', 'v1.0') # Alternatively: release = 'flowers-v1.0.json'
dataset = SegmentsDataset(release, labelset='ground-truth', filter_by=['labeled', 'reviewed'])

# Export to COCO panoptic format
export_dataset(dataset, export_format='coco-panoptic')

Alternatively, you can use the initialized SegmentsDataset to loop through the samples and labels, and visualize or process them in any way you please:

import matplotlib.pyplot as plt
from segments.utils import get_semantic_bitmap

for sample in dataset:
    # Print the sample name and list of labeled objects
    print(sample['name'])
    print(sample['annotations'])

    # Show the image
    plt.imshow(sample['image'])
    plt.show()

    # Show the instance segmentation label
    plt.imshow(sample['segmentation_bitmap'])
    plt.show()

    # Show the semantic segmentation label
    semantic_bitmap = get_semantic_bitmap(sample['segmentation_bitmap'], sample['annotations'])
    plt.imshow(semantic_bitmap)
    plt.show()
Parameters

  • dataset (SegmentsDataset) – A SegmentsDataset.

  • export_folder (str) – The folder to export the dataset to. Defaults to ..

  • export_format (ExportFormat) – The destination format. Defaults to coco-panoptic.

  • id_increment (int) – Increment the category ids with this number. Defaults to 0. Ignored unless export_format is semantic or semantic-color.

Return type

Optional[Union[Tuple[str, Optional[str]], Optional[str]]]

Returns

Returns the file name and the image directory name (for COCO panoptic, COCO instance, YOLO and polygon), or returns the export folder name (for (colored) instance and (colored) panoptic).

Raises

  • ImportError – If scikit image is not installed (to install run pip install scikit-image).

  • ValueError – If an unvalid export_format is used.

Show#

Show the exported contours of a segmented image#

show_polygons(image_directory_path, image_id, exported_polygons_path, seed=0, output_path=None)[source]#

Show the exported contours of a segmented image (i.e., resulting from export_dataset() with polygon export format).

Parameters

  • image_directory_path (str) – The image directory path.

  • image_id (int) – The image id (this can be found in the exported polygons JSON file).

  • exported_polygons_path (str) – The exported polygons path.

  • seed (int) – The seed used to generate random colors. Defaults to 0.

  • output_path (Optional[str]) – The directory to save the plot to. Defaults to None.

Return type

None

Returns

None

Raises

ImportError – If matplotlib is not installed.

Try all camera rotations to find the best one#

find_camera_rotation(client, dataset_identifier)[source]#

Find the correct camera rotation by trying all possibilities.

Parameters

  • client (SegmentsClient) – A Segments client.

  • dataset_identifier (str) – The dataset identifier.

Return type

Quaternion

Returns

A pyquaternion.Quaternion representing the correct rotation.

Raises

  • ImportError – If pyquaternion is not installed (to install run pip install pyquaternion).

  • ValueError – If the dataset is not a point cloud sequence dataset.

  • ValueError – If the user answers neither ‘y(es)’ nor ‘n(o)’ (case insensitive).

  • ValueError – If the correct rotation is not found.

  • AlreadyExistsError – If the cloned dataset already exists.