Overview
Stardust-SDK is a python SDK provided by Stardust.ai for artificial intelligence and data processing, it is designed for bridging the gap between raw annotated data and available data for machine learning. All data is sourced from Rosetta, a platform for data labeling, and MorningStar, a platform for data closed-loop, two powerful platforms released by Stardust.ai
Main Features
Export labeled data from Rosetta and MorningStar in Stardust data format, convert it to COCO, KITTI, and PandaSet formats in few lines
Visualize your model’s prediction of mainstream computer vision tasks, including Object Detection, Object Tracking, Lane Detection, etc.
Calculate mainstream metrics, including Precision, Recall, f1
Verify the accuracy of your calibration matrices
Installation
Create a Python>=3.9 environment and activate it.
conda create -n stardust_sdk python=3.9 -y
conda activate stardust_sdk
Clone this repository:
git clone [email protected]:stardustai/Stardust-SDK.git
Install necessary dependencies:
pip install .
Quick start
1. export data
Export data from Rosetta in Stardust data structure, save to json format
import json
from stardust.utils.convert import read_rosetta
gen_data = read_rosetta(project_id=1694, # int, rosetta project id
input_path='your local path', # path in your machine
pool_lst=[48904, ], # optional, List[int], default to be all pool list in your project
export_type="json" # optional
)
with open(f"your_file.json", 'w', encoding='utf-8') as sf:
for json_data in gen_data:
json.dump(json_data, sf, ensure_ascii=False)
break
<img src="docs/statics/demo_1507.png"><br>
2. to kitti
Save exported data to kitti format
import os
from stardust.utils.convert import read_rosetta
from stardust.convertion.to_kitti.to_kitti import kitti_export
gen_json_data = read_rosetta(project_id=1507, # int, rosetta project id
input_path='your local save path',
)
kitti_export(gen_json_data,
os.path.join('your local save path to save kitti file', 'kitti')) # save to path: input_path/kitti
output:
text
car None None None 987.26 369.54 1105.18 1105.18 4.065910603036291 1.9499999999999689 1.4590691860892584 10.058481575778755 -1.963636319056108 3.8287414127142725 0.5734304778127632 None
car None None None 359.95 318.03 474.96 474.96 4.065910603036291 1.9499999999999689 1.4590691860892584 -7.192361113079511 -4.548575686259395 8.079420351383028 0.5734304778127632 None
car None None None 1083.58 392.09 1182.13 1182.13 4.425148860728962 1.8987388788114303 1.4755051470238796 9.616270864324097 -0.8420100314670216 1.362204038371135 0.598822700903493 None
3. compute metrics
With pre labeled data annotated by your model and human annotated ground truth, you can evaluate your model’s performance using stardust-sdk with few lines
from stardust.utils.convert import read_rosetta
from pprint import pprint
from stardust.metric.object_detection import compute_metric
project_id = 1507
json_datas = read_rosetta(project_id=project_id,
input_path='your local path',
) # export data containing model's prediction and ground truth
metric = compute_metric(json_datas, 0.5, 'IoU', 'results')
pprint(metric)
'output'
[{
0: {
'f1_2d': 1.0,
'f1_3d': 0.875,
'gt_2d': 21,
'gt_3d': 16,
'pd_2d': 21,
'pd_3d': 16,
'precision_2d': 1.0,
'precision_3d': 0.875,
'recall_2d': 1.0,
'recall_3d': 0.875,
'tp_2d': 21,
'tp_3d': 14}}
]
4. project 3D points to image
Verify the accuracy of your calibration matrices
import numpy as np
import open3d as o3d
from stardust.geometry.ops3d import pointcloud2img
from stardust.utils.convert import read_rosetta
pcd_path = 'your_pcd_paths'
frames_jsons = read_rosetta(project_id='your project id',
input_path='rosetta data path',
export_type="json"
)
pcd = o3d.io.read_point_cloud(pcd_path)
pcd_pts = np.asarray(pcd.points)
for js in frames_jsons:
frame_info = js['media']
img_pixel = pointcloud2img(frame_info, cam_idx=1, points_3d=pcd_pts)
"""
img_pixel is the projection result of pcd_pts on image 1
"""
Data Structure
Main Structure
{
"media": "Media object, annotated file",
"task_info": "TaskInfo object, project information",
"annotation": "Annotation object, annotation result",
"prediction": "Prediction object, algorithmic pre-annotation"
}
Media Example
{
"point_cloud": {
"uri": "https://*/1693195350100.pcd",
"type": "POINTCLOUD",
"file_path": "1693195350100.pcd",
"timestamp": 1693195350100
},
"image": [
{
"uri": "https://*/1693195350051_A.png",
"type": "IMAGE",
"width": 1280,
"height": 960,
"name": "rear",
"camera_param": {
"camera_type": "Fisheye",
"heading": [
0.5939518483100418,
0.5981129564055775,
-0.3857232869482262,
-0.37509950569778316
],
"position": [
-4.890000000219184,
-1.4723984531617162e-09,
0.500000000836241
],
"intrinsic": [
292.1987593886603,
292.1987593886603,
639.5,
479.5
],
"radial": [
0.11454359348812884,
0.04270471361327884,
-0.03476042570306784,
0.004770354254106963
],
"tangential": [
0,
0
],
"fov": 206,
"projection": {},
"skew": 0
}
}
]
}
Annotation Example
{
"box2d_lst": {
"19bf6e71-2773-4a8b-8e2d-7ef860c32f14": {
"p1": [
987.26,
369.54
],
"p2": [
1105.18,
441.48
],
"center": [
1046.22,
405.51
],
"size": [
117.92000000000007,
71.94
],
"source": "rear",
"label": "label",
"children": {
"2d632ebc-5eb1-43ac-8751-1ede4ded7736": "input",
"40aa6bff-6a7d-4326-89f3-dd85f29d4897": "input"
},
"parent": "022f67d8-2f31-4c18-821d-00853799b36a"
}
},
"box3d_lst": {},
"input_lst": {
"92b58f8d-af74-445f-a680-fcde1a7f294e": {
"input_type": "select",
"name": "Delete3DBox",
"parent": "022f67d8-2f31-4c18-821d-00853799b36a",
"required": false,
"value": "NO"
}
}
}
TaskInfo Example
{
"frame_num": 0,
"pool_id": 37906,
"project_id": 1507,
"task_id": 271829271554051024
}