Job Automation - SDK

The CGC SDK allows users to construct pipelines and incorporate logic into jobs management.

Information

A valid context is required for connection to the platform and telemetry permissions must be set to avoid process interruption.

Basic Usage

Creating and deleting an jobs can be done as follows:

Job create

Simple job creation:

import cgc.sdk as cgc
# Create a job
response_created = cgc.jobs.job_create("name", "image-name:tag", cpu = 12, memory = 128,
                                            gpu = 1, gpu_type="A100")

Job creation with mounted volume, at specific path:

import cgc.sdk as cgc
# Create a job with mounted volume
response_created = cgc.jobs.job_create("name", "image-name:tag", cpu = 12, memory = 128,
                                            gpu = 1, gpu_type="A100", volumes=["training-data"], volume_full_path="/tmp/data")

Data will be mounted in /tmp/data directory.

If you want to utilize multiple volumes for the job, you can pass a list of volumes:

Information

Volumes will be mounted in: /usr/'volume_name'

import cgc.sdk as cgc
# Create a job with multiple mounted volumes
response_created = cgc.jobs.job_create("name", "image-name:tag", cpu = 12, memory = 128,
                                            gpu = 1, gpu_type="A100", volumes=["vol-1","vol-2"])

Whenever you do not need to check the status of the job, you can set the ttl_seconds_after_finished parameter to automatically delete the job after a specified time:

import cgc.sdk as cgc
# Create a job with TTL
response_created = cgc.jobs.job_create("name", "image-name:tag", cpu = 12, memory = 128,
                                            gpu = 1, gpu_type="A100", ttl_seconds_after_finished=3600)

Function arguments

Arg name	Value	Default	Description	Required
name	string	None	Name of the job	*
image_name	string	None	Public or private repo with image name and tag	*
repository_secret	string	None	Secret name provided by CGC Team for private repo
cpu	int	1	vCPU count
memory	int [GB]	2	RAM in GBs
shm_size	int [GB]	0	Shared memory cut off from resource RAM
gpu	int	0	GPU count
gpu_type	string [A100 | A5000]	A5000	Type of GPU
volumes	list	[]	List of volumes to attach
volume_full_path	string	None	Way to specify where volume will be mounted
startup_command	string	None	If not specified in Docker Image
resource_data	list	[]	List of key=value parameters
ttl_seconds_after_finished	int	None	Time to live in seconds after the job is finished
active_deadline_seconds	int	None	Time to live in seconds after the job is started

active_deadline_seconds can me set manually per user, by the administrator

Compute delete

import cgc.sdk as cgc

# Delete a custom app
response_deleted = cgc.jobs.job_delete("name")

Function arguments

Arg name	Value	Default	Description	Required
name	string	None	Name of the job	*

Listing running and pending jobs

import cgc.sdk as cgc
# Get the status of apps
response_list = cgc.jobs.job_list()