登壇
「KubeCon＋CloudNativeCon Europe 2024」に岩井佑樹が登壇いたします
昨年CNCFのSandboxプロジェクトに採択された「PipeCD」はワークショップを開催
2024年3月19日（火）～22日（金）にかけてフランス（パリ）で開催される「KubeCon + CloudNativeCon Europe 2024」に、 CyberAgent group Infrastructure Unit（CIU）の岩井佑樹が登壇いたします。また Developer Productivity室（DP室）のカーン・チャンは、昨年Cloud Native Computing Foundation（CNCF）のSandboxプロジェクトに採択された、継続的デリバリーシステム「PipeCD」に関するワークショップを行います。なお「PipeCD」チームのカーン・チャンがライトニングトークにて「Effective centric CD pipeline: Toward PipeCD v1.0」を発表いたします。
■KubeCon + CloudNativeConとは
KubeCon + CloudNativeConは、CloudNative技術全般やその中心技術であるKubernetesに関する世界最大規模のカンファレンスです。
■登壇について
・日時
Tuesday, March 19 • 16:25 - 17:00（France)
・発表者
岩井佑樹（CIU）、Andrey Velichkevich（Apple シニア ソフトウェア エンジニア）、Johnu George (Nutanix スタッフ ソフトウェア エンジニア）、Amber Graner（HPE）
・セッションタイトル
AutoML and Training Working Group Updates
・概要
AutoML working group (WG) is responsible for all aspects of AutoML features on Kubeflow with Katib as the sub-project. Katib is a Kubernetes-native project with rich support for HyperParameter tuning, Neural Architecture Search, and Early Stopping algorithms. The speakers will share Katib's 2024 ROADMAP, which includes Katib V1 APIs graduation, support for advanced parameter distribution (e.g. uniform or log-uniform), and large language model (LLM) parameters tuning. Training WG is responsible for operating scalable distributed training jobs on Kubeflow with Training Operator and MPI-Operator as sub-projects. Training Operator is a unified service for model training on various ML frameworks like PyTorch, Tensorflow, XGBoost, and PaddlePaddle. MPI Operator makes it easy to run HPC and all-reduce training tasks on Kubernetes. The speakers will share Training Operators' 2024 ROADMAP, which includes support of LLM API via Python SDK, advanced suspend semantics, and indexed job support.
https://sched.co/1YFjd
—-------------------------------------------
・日時
Wednesday, March 20 • 11:15 - 11:50（France)
・発表者
岩井佑樹（CIU）、Michał Woźniak（Google ソフトウェア エンジニア）
・セッションタイトル
Advanced Resource Management for Running AI/ML Workloads with Kueue
・概要
Kueue is a Job-level queueing manager which stands up to the challenges of managing computational resources to run batch workloads on Kubernetes. We walk you through its architecture, demonstrating how it can be used to set up quota- and priority-based sharing of resources between multiple teams. We describe how the Kueue’s scheduler decides when to start or stop (preempt) a job. We showcase Kueue by its production use at CyberAgent, where it is a building block of the multi-tenant system, supporting multiple engineers and ML research teams; using multiple types of CPUs and GPUs. Here, Kueue manages various types of Jobs (batch Job, MPIJob, or in-house Jobs), using various ML frameworks (TensorFlow, PyTorch or DeepSpeed). Finally, we discuss the challenge of running ML training jobs which require all pods to be scheduled. We show how it is solved by using Kueue at CyberAgent, and how it can be solved using Kueue in the autoscaling environments with the new ProvisioningRequest API.
https://sched.co/1YeLj
■ワークショップについて
・日時
Thursday, March 21 • 11:00 - 12:30（France)
・参加者
カーン・チャン（DP室）
・概要
In the latest release of PipeCD, we have introduced a new feature that allows for a pluggable architecture in Piped. This exciting enhancement means that PipeCD is no longer limited to supporting only five types of applications. Users now have the flexibility to develop stages and plugins for application types specific to the platforms they are utilizing. During this session, we will focus on reviewing the interface of Piped for the pluggable architecture and demonstrate how to create a simple plugin. Attendees can build a simple plugin that only includes the QuickSync stage for their application's platform use. In addition, we have prepared many good-first-issues so that participants can get acquainted with the PipeCD project and understand how PipeCD is used to achieve progressive delivery quickly and effectively.
https://sched.co/1Yheb
■ライトニングトークについて
・日時
Tuesday, March 19 • 11:00 - 11:07（France)
・発表者
カーン・チャン（DP室）
・セッションタイトル
Effective centric CD pipeline: Toward PipeCD v1.0
・概要
PipeCD provides a unified and easy-to-understand interface that helps you achieve progressive delivery easily for various application platforms at all levels, from a single project to a large organization. Moving towards v1.0, many new features have been added, the most important of which is the ability to write plugin providers to support pipeline deployment for whatever platform you use. In this section, we will talk about PipeCD's progress so far and its future prospects.
https://sched.co/1aQX4
■KubeCon + CloudNativeConとは
KubeCon + CloudNativeConは、CloudNative技術全般やその中心技術であるKubernetesに関する世界最大規模のカンファレンスです。
■登壇について
・日時
Tuesday, March 19 • 16:25 - 17:00（France)
・発表者
岩井佑樹（CIU）、Andrey Velichkevich（Apple シニア ソフトウェア エンジニア）、Johnu George (Nutanix スタッフ ソフトウェア エンジニア）、Amber Graner（HPE）
・セッションタイトル
AutoML and Training Working Group Updates
・概要
AutoML working group (WG) is responsible for all aspects of AutoML features on Kubeflow with Katib as the sub-project. Katib is a Kubernetes-native project with rich support for HyperParameter tuning, Neural Architecture Search, and Early Stopping algorithms. The speakers will share Katib's 2024 ROADMAP, which includes Katib V1 APIs graduation, support for advanced parameter distribution (e.g. uniform or log-uniform), and large language model (LLM) parameters tuning. Training WG is responsible for operating scalable distributed training jobs on Kubeflow with Training Operator and MPI-Operator as sub-projects. Training Operator is a unified service for model training on various ML frameworks like PyTorch, Tensorflow, XGBoost, and PaddlePaddle. MPI Operator makes it easy to run HPC and all-reduce training tasks on Kubernetes. The speakers will share Training Operators' 2024 ROADMAP, which includes support of LLM API via Python SDK, advanced suspend semantics, and indexed job support.
https://sched.co/1YFjd
—-------------------------------------------
・日時
Wednesday, March 20 • 11:15 - 11:50（France)
・発表者
岩井佑樹（CIU）、Michał Woźniak（Google ソフトウェア エンジニア）
・セッションタイトル
Advanced Resource Management for Running AI/ML Workloads with Kueue
・概要
Kueue is a Job-level queueing manager which stands up to the challenges of managing computational resources to run batch workloads on Kubernetes. We walk you through its architecture, demonstrating how it can be used to set up quota- and priority-based sharing of resources between multiple teams. We describe how the Kueue’s scheduler decides when to start or stop (preempt) a job. We showcase Kueue by its production use at CyberAgent, where it is a building block of the multi-tenant system, supporting multiple engineers and ML research teams; using multiple types of CPUs and GPUs. Here, Kueue manages various types of Jobs (batch Job, MPIJob, or in-house Jobs), using various ML frameworks (TensorFlow, PyTorch or DeepSpeed). Finally, we discuss the challenge of running ML training jobs which require all pods to be scheduled. We show how it is solved by using Kueue at CyberAgent, and how it can be solved using Kueue in the autoscaling environments with the new ProvisioningRequest API.
https://sched.co/1YeLj
■ワークショップについて
・日時
Thursday, March 21 • 11:00 - 12:30（France)
・参加者
カーン・チャン（DP室）
・概要
In the latest release of PipeCD, we have introduced a new feature that allows for a pluggable architecture in Piped. This exciting enhancement means that PipeCD is no longer limited to supporting only five types of applications. Users now have the flexibility to develop stages and plugins for application types specific to the platforms they are utilizing. During this session, we will focus on reviewing the interface of Piped for the pluggable architecture and demonstrate how to create a simple plugin. Attendees can build a simple plugin that only includes the QuickSync stage for their application's platform use. In addition, we have prepared many good-first-issues so that participants can get acquainted with the PipeCD project and understand how PipeCD is used to achieve progressive delivery quickly and effectively.
https://sched.co/1Yheb
■ライトニングトークについて
・日時
Tuesday, March 19 • 11:00 - 11:07（France)
・発表者
カーン・チャン（DP室）
・セッションタイトル
Effective centric CD pipeline: Toward PipeCD v1.0
・概要
PipeCD provides a unified and easy-to-understand interface that helps you achieve progressive delivery easily for various application platforms at all levels, from a single project to a large organization. Moving towards v1.0, many new features have been added, the most important of which is the ability to write plugin providers to support pipeline deployment for whatever platform you use. In this section, we will talk about PipeCD's progress so far and its future prospects.
https://sched.co/1aQX4