Tasks on Jobset

Example Workloads

Wed, 23 Jul 2025 00:00:00 +0000

PyTorch Example

Distributed Training of a CNN on the MNIST dataset using PyTorch and JobSet

Note: Machine learning container images can be quite large so it may take some time to pull the images.

TensorFlow Example

Distributed Training of a Handwritten Digit Classifier on the MNIST dataset using TensorFlow and JobSet

This example runs an example job for a single epoch. You can view the progress of your jobs via kubectl logs jobs/tensorflow-tensorflow-0.

Simple Examples

Wed, 23 Jul 2025 00:00:00 +0000

Here we have some simple examples demonstrating core JobSet features.

Success Policy demonstrates an example of utilizing successPolicy. Success Policy allows one to specify when to mark a JobSet as completed successfully. This example showcases how to use success policy to mark the JobSet as successful if the worker replicated job completes.
Exclusive Job Placement demonstrates how to configure a JobSet to have a 1:1 mapping between each child Job and a particular topology domain, such as a datacenter rack or zone. This means that all the pods belonging to a child job will be colocated in the same topology domain, while pods from other jobs will not be allowed to run within this domain. This gives the child job exclusive access to computer resources in this domain.

Failure Policy

Wed, 23 Jul 2025 00:00:00 +0000

JobSet provides failure policy API to control how your workload behaves in response to child Job failures.

The failurePolicy is defined by a set of rules. For any job failure, the rules are evaluated in order, and the first matching rule’s action is executed. If no rule matches, the default action is RestartJobSet, which counts towards the maxRestarts limit.

Failure Policy Actions

`FailJobSet`

This action immediately marks the entire JobSet as failed.

Volume Claim Policies

Tue, 13 Jan 2026 00:00:00 +0000

JobSet provides the VolumeClaimPolicies API to automatically create and manage shared PersistentVolumeClaims (PVCs) across multiple ReplicatedJobs within a JobSet. This enables stateful JobSets that require persistent storage for datasets, models, checkpoints, or intermediate results.

Basic Usage

To use VolumeClaimPolicies, define them in the volumeClaimPolicies field of your JobSet spec. Each policy can contain one or more PVC templates.

This example demonstrates creating shared PVCs with different retention policies:

In this example: