SHARCNETHelp - Recent changes [en]

Online Seminars

Jhqin — Mon, 30 Mar 2026 14:51:18 GMT

2026

← Older revision		Revision as of 14:51, 30 March 2026
Line 6:		Line 6:
	===2026===		===2026===
	* 2026/04/08 - Mixing diverse workloads within a single job: a heterogeneous scheduling primer, Sergey Mashchenko, [[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Abstract]]		* 2026/04/08 - Mixing diverse workloads within a single job: a heterogeneous scheduling primer, Sergey Mashchenko, [[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Abstract]]
	* 2026/03/25 - [https://~~training~~.~~computeontario.ca/courses/course~~/~~view.php~~?id=~~203~~ Getting started with MLflow for machine learning and AI development], Jinhui Qin, [[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Abstract]]		* 2026/03/25 - [https://youtu.be/Nk08qeTO9qY?si=mt9B4xZVevDFWFno Getting started with MLflow for machine learning and AI development], Jinhui Qin, [[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Abstract]]
	* 2026/03/11 - [https://youtu.be/tfxeN2kOKI8 Know your cluster: ways that national systems vary in performance], Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]		* 2026/03/11 - [https://youtu.be/tfxeN2kOKI8 Know your cluster: ways that national systems vary in performance], Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]
	* 2026/02/25 - [https://training.computeontario.ca/courses/course/view.php?id=202 Launching a Virtual Machine Using Nibi OpenStack Cloud], Ed Armstrong, [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Abstract]]		* 2026/02/25 - [https://training.computeontario.ca/courses/course/view.php?id=202 Launching a Virtual Machine Using Nibi OpenStack Cloud], Ed Armstrong, [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Abstract]]

Online Seminars

Syam — Thu, 26 Mar 2026 16:11:01 GMT

2026

← Older revision		Revision as of 16:11, 26 March 2026
Line 5:		Line 5:
	==General Interest Webinars==		==General Interest Webinars==
	===2026===		===2026===
			* 2026/04/08 - Mixing diverse workloads within a single job: a heterogeneous scheduling primer, Sergey Mashchenko, [[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Abstract]]
	* 2026/03/25 - [https://training.computeontario.ca/courses/course/view.php?id=203 Getting started with MLflow for machine learning and AI development], Jinhui Qin, [[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Abstract]]		* 2026/03/25 - [https://training.computeontario.ca/courses/course/view.php?id=203 Getting started with MLflow for machine learning and AI development], Jinhui Qin, [[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Abstract]]
	* 2026/03/11 - [https://youtu.be/tfxeN2kOKI8 Know your cluster: ways that national systems vary in performance], Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]		* 2026/03/11 - [https://youtu.be/tfxeN2kOKI8 Know your cluster: ways that national systems vary in performance], Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]

General Interest Seminars

Bge — Wed, 25 Mar 2026 16:59:54 GMT

Round 13 (Starting October 2025)

← Older revision		Revision as of 16:59, 25 March 2026
(One intermediate revision by the same user not shown)
Line 704:		Line 704:
	\|Tyson* \|\| \|\| 2026/4/22 \|\| \|\|		\|Tyson* \|\| \|\| 2026/4/22 \|\| \|\|
	\|-		\|-
	\|Baolai \|\| \|\| 2026/5/6 \|\| \|\|		\|Baolai \|\| A walk though of Alliance's supercomputing resources and an overview of the training courses SN offers \|\| 2026/5/6 \|\| \|\|
	\|-		\|-
	\|Pawel \|\| \|\| 2026/7/15 \|\| \|\|		\|Pawel \|\| \|\| 2026/7/15 \|\| \|\|

General Interest Seminars

Syam — Wed, 25 Mar 2026 16:14:55 GMT

Round 13 (Starting October 2025)

← Older revision		Revision as of 16:14, 25 March 2026
Line 698:		Line 698:
	\|Mark \|\| [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Know your cluster: ways that national systems vary in performance]]\|\| 2026/3/11 \|\| 15 \|\|		\|Mark \|\| [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Know your cluster: ways that national systems vary in performance]]\|\| 2026/3/11 \|\| 15 \|\|
	\|-		\|-
	\|Jinhui* \|\|[[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| \|\|		\|Jinhui* \|\|[[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| 13 \|\|
	\|-		\|-
	\|Sergey \|\|[[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Mixing diverse workloads within a single job: a heterogeneous scheduling primer]] \|\| 2026/4/8 \|\| \|\|		\|Sergey \|\|[[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Mixing diverse workloads within a single job: a heterogeneous scheduling primer]] \|\| 2026/4/8 \|\| \|\|

Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer

Syam — Tue, 24 Mar 2026 14:39:22 GMT

← Older revision		Revision as of 14:39, 24 March 2026
(6 intermediate revisions by the same user not shown)
Line 1:		Line 1:
	Most of the National systems' users are perfectly happy with our "vanilla" job scheduling setup: one requests a particular type (CPU cores, GPUs or MIGs, large memory) and amount of resources, and then lets the job consume those resources over the specified job's runtime. In most cases, this results in reasonably good job efficiency, and reasonably short queue wait time. A ~~sizable~~ minority of jobs do not fit this homogeneous job model well, and end up wasting resources, and staying long in the job queue - because of the diverse workloads within the job. In more extreme cases, some of these jobs cannot even run on our clusters. ~~A typical~~ example: a large (~~a few hundreds of~~ CPU cores) MPI job, where the typical memory per rank requirement is modest (say 4GB ~~or less~~), but the rank 0 frequently needs much larger memory.		Most of the National systems' users are perfectly happy with our "vanilla" job scheduling setup: one requests a particular type (CPU cores, GPUs or MIGs, large memory) and amount of resources, and then lets the job consume those resources over the specified job's runtime. In most cases, this results in reasonably good job efficiency, and reasonably short queue wait time. A sizeable minority of the jobs do not fit this homogeneous job model well, and end up wasting resources and staying long in the job queue - because of the diverse workloads within the job. In more extreme cases, some of these jobs cannot even run on our clusters. An example: a large (say 1000 CPU cores) MPI job, where the typical memory per rank requirement is modest (say 4GB), but the rank 0 frequently needs much larger memory (say, 12GB). If using the simple (homogeneous) job model, one would need to request the large amount of memory per CPU core (12GB) for all the ranks, which will result in a long queue wait time, and a waste of the memory or CPU cores. Another example: one component in the job needs GPUs, while the other component doesn't (it needs lots of CPU cores instead). Again, trying to fit such a workload into a homogeneous job will result in longer wait time, and a significant waste of the resources (most likely GPUs).

			The proper way to handle diverse workloads within a single job is by using the heterogeneous job scheduling mode of SLURM (our scheduler). This scheduler feature has been available for some time now, but it wasn't widely advertised for our users as we were testing it out. In this webinar I will walk you through the whys and hows of heterogeneous job scheduling, and will provide some specific job script examples. The webinar has no prerequisites, but some experience with job submission on our clusters would be a plus.

Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer

Syam — Tue, 24 Mar 2026 13:49:23 GMT

Created page with "Most of the National systems' users are perfectly happy with our "vanilla" job scheduling setup: one requests a particular type (CPU cores, GPUs or MIGs, large memory) and amo..."

New page

Most of the National systems' users are perfectly happy with our "vanilla" job scheduling setup: one requests a particular type (CPU cores, GPUs or MIGs, large memory) and amount of resources, and then lets the job consume those resources over the specified job's runtime. In most cases, this results in reasonably good job efficiency, and reasonably short queue wait time. A sizable minority of jobs do not fit this homogeneous job model well, and end up wasting resources, and staying long in the job queue - because of the diverse workloads within the job. In more extreme cases, some of these jobs cannot even run on our clusters. A typical example: a large (a few hundreds of CPU cores) MPI job, where the typical memory per rank requirement is modest (say 4GB or less), but the rank 0 frequently needs much larger memory.

General Interest Seminars

Syam — Tue, 24 Mar 2026 13:31:58 GMT

Round 13 (Starting October 2025)

← Older revision		Revision as of 13:31, 24 March 2026
Line 700:		Line 700:
	\|Jinhui* \|\|[[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| \|\|		\|Jinhui* \|\|[[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| \|\|
	\|-		\|-
	\|Sergey \|\| \|\| 2026/4/8 \|\| \|\|		\|Sergey \|\|[[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Mixing diverse workloads within a single job: a heterogeneous scheduling primer]] \|\| 2026/4/8 \|\| \|\|
	\|-		\|-
	\|Tyson* \|\| \|\| 2026/4/22 \|\| \|\|		\|Tyson* \|\| \|\| 2026/4/22 \|\| \|\|

Webinar 2026 Getting started with MLflow for machine learning and AI development

Syam — Wed, 11 Mar 2026 20:56:55 GMT

Created page with "Reproducibility and experiment tracking are essential in machine learning workflows. MLflow is an open-source platform for experiment tracking and model management in machine..."

New page

Reproducibility and experiment tracking are essential in machine learning workflows. MLflow is an open-source platform for experiment tracking and model management in machine learning and AI development. This webinar introduces MLflow with quickstart examples running on the clusters, focusing on a lightweight setup with local storage. The examples will be demonstrated in Jupyter notebooks and in batch jobs.

General Interest Seminars

Syam — Wed, 11 Mar 2026 20:56:44 GMT

Round 13 (Starting October 2025)

← Older revision		Revision as of 20:56, 11 March 2026
Line 698:		Line 698:
	\|Mark \|\| [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Know your cluster: ways that national systems vary in performance]]\|\| 2026/3/11 \|\| 15 \|\|		\|Mark \|\| [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Know your cluster: ways that national systems vary in performance]]\|\| 2026/3/11 \|\| 15 \|\|
	\|-		\|-
	\|Jinhui* \|\|[[Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| \|\|		\|Jinhui* \|\|[[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| \|\|
	\|-		\|-
	\|Sergey \|\| \|\| 2026/4/8 \|\| \|\|		\|Sergey \|\| \|\| 2026/4/8 \|\| \|\|

Online Seminars

Syam — Wed, 11 Mar 2026 20:55:34 GMT

2026

← Older revision		Revision as of 20:55, 11 March 2026
Line 5:		Line 5:
	==General Interest Webinars==		==General Interest Webinars==
	===2026===		===2026===
	* 2026/03/11 - Know your cluster: ways that national systems vary in performance, Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]		* 2026/03/25 - [https://training.computeontario.ca/courses/course/view.php?id=203 Getting started with MLflow for machine learning and AI development], Jinhui Qin, [[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Abstract]]
			* 2026/03/11 - [https://youtu.be/tfxeN2kOKI8 Know your cluster: ways that national systems vary in performance], Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]
	* 2026/02/25 - [https://training.computeontario.ca/courses/course/view.php?id=202 Launching a Virtual Machine Using Nibi OpenStack Cloud], Ed Armstrong, [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Abstract]]		* 2026/02/25 - [https://training.computeontario.ca/courses/course/view.php?id=202 Launching a Virtual Machine Using Nibi OpenStack Cloud], Ed Armstrong, [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Abstract]]
	* 2026/02/11 - [https://youtu.be/-1trpCl2GOs Running Gaussian16 and NBO7 effectively on Nibi and Fir: Performance Issues], Jemmy Hu, [[Webinar 2026 Running Gaussian16 and NBO7 effectively on Nibi and Fir: Performance Issues\|Abstract]], [[Media:Gauss_NBO_2026_02.pdf\|slides]]		* 2026/02/11 - [https://youtu.be/-1trpCl2GOs Running Gaussian16 and NBO7 effectively on Nibi and Fir: Performance Issues], Jemmy Hu, [[Webinar 2026 Running Gaussian16 and NBO7 effectively on Nibi and Fir: Performance Issues\|Abstract]], [[Media:Gauss_NBO_2026_02.pdf\|slides]]

General Interest Seminars

Syam — Wed, 11 Mar 2026 17:25:21 GMT

Round 13 (Starting October 2025)

← Older revision		Revision as of 17:25, 11 March 2026
Line 696:		Line 696:
	\|Ed* \|\| [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Launching a Virtual Machine Using Nibi OpenStack Cloud]]\|\| 2026/2/25 \|\| 7 \|\|		\|Ed* \|\| [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Launching a Virtual Machine Using Nibi OpenStack Cloud]]\|\| 2026/2/25 \|\| 7 \|\|
	\|-		\|-
	\|Mark \|\| [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Know your cluster: ways that national systems vary in performance]]\|\| 2026/3/11 \|\| \|\|		\|Mark \|\| [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Know your cluster: ways that national systems vary in performance]]\|\| 2026/3/11 \|\| 15 \|\|
	\|-		\|-
	\|Jinhui* \|\|[[Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| \|\|		\|Jinhui* \|\|[[Getting started with MLflow for machine learning and AI development\|Getting started with MLflow for machine learning and AI development]] \|\| 2026/3/25 \|\| \|\|

← Older revision		Revision as of 14:51, 30 March 2026
Line 6:		Line 6:
	===2026===		===2026===
	* 2026/04/08 - Mixing diverse workloads within a single job: a heterogeneous scheduling primer, Sergey Mashchenko, [[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Abstract]]		* 2026/04/08 - Mixing diverse workloads within a single job: a heterogeneous scheduling primer, Sergey Mashchenko, [[Webinar 2026 Mixing diverse workloads within a single job: a heterogeneous scheduling primer\|Abstract]]
	* 2026/03/25 - [https://~~training~~.~~computeontario.ca/courses/course~~/~~view.php~~?id=~~203~~ Getting started with MLflow for machine learning and AI development], Jinhui Qin, [[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Abstract]]		* 2026/03/25 - [https://youtu.be/Nk08qeTO9qY?si=mt9B4xZVevDFWFno Getting started with MLflow for machine learning and AI development], Jinhui Qin, [[Webinar 2026 Getting started with MLflow for machine learning and AI development\|Abstract]]
	* 2026/03/11 - [https://youtu.be/tfxeN2kOKI8 Know your cluster: ways that national systems vary in performance], Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]		* 2026/03/11 - [https://youtu.be/tfxeN2kOKI8 Know your cluster: ways that national systems vary in performance], Mark Hahn, [[Webinar 2026 Know your cluster: ways that national systems vary in performance\|Abstract]]
	* 2026/02/25 - [https://training.computeontario.ca/courses/course/view.php?id=202 Launching a Virtual Machine Using Nibi OpenStack Cloud], Ed Armstrong, [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Abstract]]		* 2026/02/25 - [https://training.computeontario.ca/courses/course/view.php?id=202 Launching a Virtual Machine Using Nibi OpenStack Cloud], Ed Armstrong, [[Webinar 2026 Launching a Virtual Machine Using Nibi OpenStack Cloud\|Abstract]]