[Tsinghua University] JCS: Breaking the Bottleneck of Shared DML Clusters via Job-Aware Scheduling

TL;DR

Distributed Machine Learning (DML) in shared clusters often suffers from massive performance degradation due to network contention. While computation resources (GPUs) are often isolated, the network is not. This paper proposes JCS, a scheduler that looks into the "Application Layer" to prioritize communications based on both data size and computation time. It achieves up to a 46% reduction in Job Completion Time (JCT) compared to traditional coflow scheduling.

The "Shared Cluster" Crisis

In the modern industry, DML jobs rarely run in isolation. Multiple jobs (e.g., training ResNet, VGG, or Transformers) share the same underlying data center fabric. While we have achieved significant milestones in overlapping computation and communication within a single job, these methods break down when "Network Contention" hits.

The authors observe a critical Inconsistency of Optimization Goals:

• Network Layer Schedulers (like SCF or LCoF) aim to minimize CCT (Coflow Completion Time).
• DML Applications care about JCT (Job Completion Time).

Because an iteration only finishes after both communication and the dependent computation are done, a fast coflow finish doesn't guarantee a fast iteration if it delays a "straggler" worker.

Methodology: The JCS Design

JCS introduces two layers of intelligence to solve the contention problem.

1. Inter-job Scheduling: Relative Contention

Instead of just looking at who has the smallest flow (Shortest Flow First), JCS calculates a Relative Contention priority. It asks: If I let Job A go now, how much does it hurt the other jobs competing for the same ports?

It also factors in Computation Priority. Jobs with longer computation times are prioritized for communication so they can start their exclusive GPU work sooner, effectively "getting out of the way" of the network.

2. Intra-job Scheduling: Rescuing the Stragglers

Within a single job, JCS abandons "Fair Sharing" or "Round-Robin." Instead, it identifies the straggler (the slowest worker) and gives it the full available bandwidth.

Figure: The JCS architecture showing the interaction between the Central Scheduler and Job Managers.

By accelerating the straggler's parameter pull, JCS ensures that the "synchronization barrier" at the end of the iteration is reached faster by all workers.

Experimental Validation

Using SimGrid to model a 128-server GPU cluster, the authors tested JCS against synthetic workloads derived from Microsoft's production traces.

Key Findings:

• General Speedup: JCS outperformed SCF (a state-of-the-art coflow scheduler) by 39% on average.
• Model Sensitivity:
  • For ResNet50 (compute-intensive), JCS provides high gains by clearing the network quickly.
  • For VGG-16 (communication-intensive), JCS provides stable acceleration regardless of bandwidth.
• Bandwidth Scaling: Even on 100 Gbps high-speed networks, JCS provides a 15% to 37% speedup, proving its relevance in ultra-high-speed InfiniBand environments.

Figure: CDF of speedup showing JCS significantly shifting JCT distribution toward faster completion compared to RRSP.

Critical Insight & Future Outlook

The primary takeaway is that Network Contention is an Application Problem. JCS proves that by exposing "Computation Time" and "Straggler Status" to the network scheduler, we can achieve massive efficiency gains that purely network-centric approaches miss.

Limitations: JCS currently relies on a central scheduler, which might become a bottleneck in massive-scale clusters (thousands of nodes). Future iterations might explore decentralized scheduling or tighter integration with Collective Communication libraries (NCCL/HCCL) rather than just the Parameter Server (PS) architecture.

Final takeaway: If you are running a multi-tenant GPU cluster, ignoring the interdependence of compute and network is leaving nearly 50% of your performance on the table.