Pretrained RT-DETR inference

RTDetrWorkflow provides an object detection API based on Ultralytics RT-DETR. The default model is rtdetr-l.pt and the default configuration requests one NVIDIA RTX A5000 GPU.

Create the workflow

from cgc.sdk import Image
from cgc.sdk.recipes.rtdetr import RTDetrConfig, RTDetrWorkflow

workflow = RTDetrWorkflow(
    name="sdk-rtdetr-inference-demo",
    image=Image.from_registry("ultralytics/ultralytics:latest"),
    config=RTDetrConfig(
        model_name="rtdetr-l.pt",
        inference_cpu=4,
        inference_memory=16,
        inference_gpu=1,
        inference_gpu_type="nvidia-rtx-a5000",
    ),
)

Arguments used above:

Argument	Meaning
name="sdk-rtdetr-inference-demo"	CGC workflow name. It is also used as the inference resource name unless inference_name is provided.
image=...	Container image containing the environment in which Ultralytics will run.
"ultralytics/ultralytics:latest"	Public Ultralytics container image. Pin a versioned tag when reproducibility is required.
config=...	RT-DETR model and compute configuration.
model_name="rtdetr-l.pt"	Pretrained model weights downloaded and loaded by Ultralytics.
inference_cpu=4	CPU cores reserved for the inference resource.
inference_memory=16	Memory reserved for inference, in GB.
inference_gpu=1	Number of GPUs reserved for inference.
inference_gpu_type="nvidia-rtx-a5000"	CGC identifier of the required GPU model.

Deploy

endpoint = workflow.deploy_pretrained_inference(
    wait=True,
    reuse_existing=True,
)
endpoint.wait_http("/health", timeout_seconds=300)
print(endpoint.url)

Arguments used above:

Argument	Meaning
wait=True	Waits until the CGC resource is running.
reuse_existing=True	Reuses an inference resource with the same name instead of failing when it already exists.
"/health"	Health endpoint exposed by the generated RT-DETR service.
timeout_seconds=300	Maximum wait for the HTTP service, in seconds.

Detect objects

prediction = endpoint.post_json(
    "/predict",
    {"image_url": "https://ultralytics.com/images/bus.jpg"},
    timeout=120,
)

for detection in prediction["detections"]:
    print(detection["class_name"], detection["confidence"])

Arguments used above:

Argument	Meaning
"/predict"	RT-DETR HTTP route that performs object detection.
{"image_url": "..."}	JSON request body. image_url must point to an image reachable from the inference container. The API also accepts image_base64.
timeout=120	Maximum number of seconds allowed for the HTTP request.

The response also contains annotated_image_base64, which can be decoded and saved as an image.

endpoint.delete()

To run this example, create a Python file in your own project, for example rtdetr_inference.py, and combine the code blocks from this page:

from cgc.sdk import Image
from cgc.sdk.recipes.rtdetr import RTDetrConfig, RTDetrWorkflow

workflow = RTDetrWorkflow(
    name="sdk-rtdetr-inference-demo",
    image=Image.from_registry("ultralytics/ultralytics:latest"),
    config=RTDetrConfig(
        model_name="rtdetr-l.pt",
        inference_cpu=4,
        inference_memory=16,
        inference_gpu=1,
        inference_gpu_type="nvidia-rtx-a5000",
    ),
)

endpoint = workflow.deploy_pretrained_inference(
    wait=True,
    reuse_existing=True,
)
endpoint.wait_http("/health", timeout_seconds=300)

prediction = endpoint.post_json(
    "/predict",
    {"image_url": "https://ultralytics.com/images/bus.jpg"},
    timeout=120,
)

for detection in prediction["detections"]:
    print(detection["class_name"], detection["confidence"])

Run the file from your own project:

python rtdetr_inference.py

The filename is not part of the SDK. The functionality comes from the installed cgcsdk package and these imports:

from cgc.sdk import Image
from cgc.sdk.recipes.rtdetr import RTDetrConfig, RTDetrWorkflow

See the complete rtdetr_inference.py script on the Workflow examples page.

note

This guide covers inference with pretrained RT-DETR weights. See Datasets for the shared dataset workflow used by Ultralytics training workflows.

note

reuse_existing=True does not update the model, image or environment of an existing resource. The example leaves the service running until explicit deletion.