AWS has announced quota management for AWS Batch scheduling of Amazon SageMaker Training jobs. The capability is designed for organizations sharing GPU capacity across multiple teams, projects, or workloads.

According to AWS, administrators can create quota shares within a Batch job queue and assign each share a capacity limit and resource-sharing strategy. The available strategies are RESERVE, which keeps idle capacity available for the share; LEND, which allows idle capacity to be used by other shares; and LEND_AND_BORROW, which supports both lending and borrowing.

AWS says that when work arrives in a quota share that has lent capacity, AWS Batch can reclaim that capacity through cross-share preemption. Administrators can also enable in-share preemption, allowing higher-priority jobs to preempt lower-priority jobs in the SCHEDULED, STARTING, or RUNNING state.

The AWS example configures three quota shares, each with a one-instance limit using the ml.g5.xlarge instance type. One share uses LEND_AND_BORROW with in-share preemption enabled, another uses LEND without in-share preemption, and the third uses RESERVE with preemption disabled.

Using the SageMaker Python SDK v3, customers can configure the related scheduling policy, service environment, job queue, and quota shares, then submit SageMaker Training jobs to the queue. AWS also provides AWS Command Line Interface and console examples, along with a sample notebook.