YellowDog, the leading provider of cloud native workload management software, has announced the concept of Meta Scheduling in the latest release of its Platform. Meta Scheduling provides a single workload submission system that can coordinate multiple third-party heterogeneous clusters. This is achieved by allowing users to create and manage dynamically scaling compute clusters, run by a variety of technologies and schedulers.
Meta Scheduling enables the YellowDog Workload Manager to control an overall queue of work and through templating, define the configuration and type of nodes required to execute this work. Once defined, it then orchestrates and pushes workloads to these nodes.
By applying configuration to the worker nodes at particular points in the lifecycle, YellowDog offers flexible event-driven configuration management. Templated configuration can be deployed using scripts or tools, such as Ansible, on specific nodes. The general use case involves the creation of loosely or tightly coupled compute clusters, but sidecars, such as file and DNS servers, can also be deployed for the cluster to use. When the workload has been executed, the cluster can either be dynamically scaled up or down for other tasks or be rapidly dismantled and the instances terminated.
With Meta Scheduling, businesses can benefit from a single view on cost and department consumption across applications. Users can build economically viable compute clusters at scale, in both hybrid and multi-cloud environments, and multiple clusters can be created concurrently to help customers overcome on-premise capacity constraints. Fundamentally, Meta Scheduling enables businesses to utilise a wide range of familiar technologies and processes, whilst taking advantage of the intelligent provisioning strategies provided by the YellowDog Platform.