FAQ - Revision history

C7wilson: /* Getting an Account with SHARCNET and Related Issues */

2025-09-19T15:44:12Z

Getting an Account with SHARCNET and Related Issues

← Older revision		Revision as of 15:44, 19 September 2025
Line 26:		Line 26:
	== Getting an Account with SHARCNET and Related Issues ==		== Getting an Account with SHARCNET and Related Issues ==

	To use SHARCNET (and any other national facilities in Alliance) one has to [https://www.alliancecan.ca/en/services/advanced-research-computing/account-management/apply-account apply for an Alliance account].		To use SHARCNET (and any other national facilities in Alliance) one has to [https://www.alliancecan.ca/en/our-services/advanced-research-computing/account-management/apply-account apply for an Alliance account].

	== Getting Help ==		== Getting Help ==

Jhqin: /* How do I edit my program on a cluster? */

2024-09-17T18:01:06Z

How do I edit my program on a cluster?

← Older revision		Revision as of 18:01, 17 September 2024
Line 403:		Line 403:
	you can then use any local editor of your choice.		you can then use any local editor of your choice.

	If you run emacs on your desktop, you can also edit a remote file from within your local emacs client using [http://www.gnu.org/software/tramp Tramp], opening and saving a file as /username@cluster.~~computecanada~~.ca:path/file.		If you run emacs on your desktop, you can also edit a remote file from within your local emacs client using [http://www.gnu.org/software/tramp Tramp], opening and saving a file as /username@cluster.alliancecan.ca:path/file.

	== Research at SHARCNET ==		== Research at SHARCNET ==

Jhqin: /* How can I best transfer large quantities of data to/from SHARCNET and what transfer rate should I expect? */

2024-09-17T18:00:39Z

How can I best transfer large quantities of data to/from SHARCNET and what transfer rate should I expect?

← Older revision		Revision as of 18:00, 17 September 2024
Line 192:		Line 192:
	====How can I best transfer large quantities of data to/from SHARCNET and what transfer rate should I expect?====		====How can I best transfer large quantities of data to/from SHARCNET and what transfer rate should I expect?====

	In general, most users should be fine using ''scp'' or ''rsync'' to transfer data to and from SHARCNET systems. If you need to transfer a lot of files ''rsync'' is recommended to ensure that you do not need to restart the transfer from scratch should there be a connection failure. Although you can use ''scp'' and ''rsync'' to any cluster's login node(s), it is often best to use gra-dtn1.~~computecanada~~.ca - it is dedicated to data transfer.		In general, most users should be fine using ''scp'' or ''rsync'' to transfer data to and from SHARCNET systems. If you need to transfer a lot of files ''rsync'' is recommended to ensure that you do not need to restart the transfer from scratch should there be a connection failure. Although you can use ''scp'' and ''rsync'' to any cluster's login node(s), it is often best to use gra-dtn1.alliancecan.ca - it is dedicated to data transfer.

	In general one should expect the following transfer rates with ''scp'':		In general one should expect the following transfer rates with ''scp'':

Jhqin: /* Unix/Linux */

2024-09-17T17:59:39Z

Unix/Linux

← Older revision		Revision as of 17:59, 17 September 2024
Line 169:		Line 169:
	Similarly, you can transfer the subdirectory between Alliance clusters. The following command		Similarly, you can transfer the subdirectory between Alliance clusters. The following command

	[username@cdr-login2:~]$ scp -r graham:/home/username/scratch/foo .		[username@cdr-login2:~]$ scp -r graham.alliancecan.ca:/home/username/scratch/foo .

	will download subdirectory <tt>foo</tt> from your scratch directory on graham to your home directory on cedar (note that the prompt indicates you are currently logged on to cedar).		will download subdirectory <tt>foo</tt> from your scratch directory on graham to your home directory on cedar (note that the prompt indicates you are currently logged on to cedar).
Line 183:		Line 183:
	Then on your local machine myhost, use <tt>scp</tt> to copy the tar file		Then on your local machine myhost, use <tt>scp</tt> to copy the tar file

	myhost$ scp graham.~~computecanada~~.ca:project/foo.tar.gz .		myhost$ scp graham.alliancecan.ca:project/foo.tar.gz .

	Note for most Linux distributions, <tt>tar</tt> has an option <tt>-z</tt> that will compress the <tt>.tar</tt> file using <tt>gzip</tt>.		Note for most Linux distributions, <tt>tar</tt> has an option <tt>-z</tt> that will compress the <tt>.tar</tt> file using <tt>gzip</tt>.

Jhqin: /* I am unable to connect to one of the clusters; when I try, I am told the connection was closed by the remote host */

2024-09-17T17:54:33Z

I am unable to connect to one of the clusters; when I try, I am told the connection was closed by the remote host

← Older revision		Revision as of 17:54, 17 September 2024
Line 332:		Line 332:
	The most likely cause of this behaviour is repeated failed login attempts. Part of our security policies involves blocking the IP address of machines that attempt multiple logins with incorrect passwords over a short period of time---many brute-force attacks on systems do exactly this: looking for poor passwords, badly configured accounts, etc. Unfortunately, it isn't uncommon for a user to forget their password and make repeated login attempts with incorrect passwords and end up with that machine blacklisted and unable to connect at all.		The most likely cause of this behaviour is repeated failed login attempts. Part of our security policies involves blocking the IP address of machines that attempt multiple logins with incorrect passwords over a short period of time---many brute-force attacks on systems do exactly this: looking for poor passwords, badly configured accounts, etc. Unfortunately, it isn't uncommon for a user to forget their password and make repeated login attempts with incorrect passwords and end up with that machine blacklisted and unable to connect at all.

	A temporary solution is simply to attempt to login from another machine. If you have access to another machine at your site, you can shell to that machine first, and then shell to the SHARCNET system (as that machine's IP shouldn't be blacklisted). In order to have your machine unblocked, you will have to email to [mailto:support@~~computecanada~~.ca support@~~computecanada~~.ca] as a system administrator must manually intervene in order to fix it.		A temporary solution is simply to attempt to login from another machine. If you have access to another machine at your site, you can shell to that machine first, and then shell to the SHARCNET system (as that machine's IP shouldn't be blacklisted). In order to have your machine unblocked, you will have to email to [mailto:support@tech.alliancecan.ca support@tech.alliancecan.ca] as a system administrator must manually intervene in order to fix it.

	NOTE: there are other situations that can produce this message, however they are rarer and more transient. If you are unable to log in from one machine, but can from another, it is most likely the IP blacklisting that is the problem and the above will provide a temporary work-around while your problem ticket is processed.		NOTE: there are other situations that can produce this message, however they are rarer and more transient. If you are unable to log in from one machine, but can from another, it is most likely the IP blacklisting that is the problem and the above will provide a temporary work-around while your problem ticket is processed.

Jhqin: /* Unix/Linux */

2024-09-17T17:52:18Z

Unix/Linux

← Older revision		Revision as of 17:52, 17 September 2024
Line 155:		Line 155:
	To transfer files to and from a cluster on a UNIX machine, you may use <tt>scp</tt> or <tt>sftp</tt>. For example, if you want to upload file <tt>foo.f</tt> to cluster graham from your machine <tt>myhost</tt>, use the following command		To transfer files to and from a cluster on a UNIX machine, you may use <tt>scp</tt> or <tt>sftp</tt>. For example, if you want to upload file <tt>foo.f</tt> to cluster graham from your machine <tt>myhost</tt>, use the following command

	myhost$ scp foo.f graham.~~computecanada~~.ca:		myhost$ scp foo.f graham.alliancecan.ca:

	assuming that your machine has <tt>scp</tt> installed. If you want to transfer a file from Windows or Mac, you need have <tt>scp</tt> or <tt>sftp</tt> for Windows or Mac installed.		assuming that your machine has <tt>scp</tt> installed. If you want to transfer a file from Windows or Mac, you need have <tt>scp</tt> or <tt>sftp</tt> for Windows or Mac installed.
Line 165:		Line 165:
	If you are transferring files between a UNIX machine and a cluster, you may use <tt>scp</tt> command with <tt>-r</tt> option. For instance, if you want to download the subdirectory <tt>foo</tt> in the directory <tt>project</tt> in your home directory on graham to your local UNIX machine, on your local machine, use command		If you are transferring files between a UNIX machine and a cluster, you may use <tt>scp</tt> command with <tt>-r</tt> option. For instance, if you want to download the subdirectory <tt>foo</tt> in the directory <tt>project</tt> in your home directory on graham to your local UNIX machine, on your local machine, use command

	myhost$ scp -rp graham.~~sharcnet~~.ca:project/foo .		myhost$ scp -rp graham.alliancecan.ca:project/foo .

	Similarly, you can transfer the subdirectory between Alliance clusters. The following command		Similarly, you can transfer the subdirectory between Alliance clusters. The following command

Jhqin: /* SSH tells me SOMEONE IS DOING SOMETHING NASTY!? */

2024-09-17T17:51:18Z

SSH tells me SOMEONE IS DOING SOMETHING NASTY!?

← Older revision		Revision as of 17:51, 17 September 2024
Line 344:		Line 344:
	'''Please first read the following:		'''Please first read the following:

	https://docs.~~computecanada~~.ca/wiki/SSH_security_improvements'''		https://docs.alliancecan.ca/wiki/SSH_security_improvements'''

	Suppose you attempt to login to SHARCNET, but instead get an alarming message like this:		Suppose you attempt to login to SHARCNET, but instead get an alarming message like this:

Jhqin: /* Does SHARCNET provide any training on programming and using the systems? */

2024-09-17T17:47:56Z

Does SHARCNET provide any training on programming and using the systems?

← Older revision		Revision as of 17:47, 17 September 2024
Line 80:		Line 80:
	=== Does SHARCNET provide any training on programming and using the systems? ===		=== Does SHARCNET provide any training on programming and using the systems? ===

	Yes. SHARCNET provides workshops on specific topics from time to time and offers courses at some sites. Every summer (usually late May to early June), SHARCNET holds [https://~~www~~.sharcnet.ca/~~help/index.php~~/Summer_Schools an annual HPC Summer School] with a variety of in-depth, hands-on workshops.		Yes. SHARCNET provides workshops on specific topics from time to time and offers courses at some sites. Every summer (usually late May to early June), SHARCNET holds [https://helpwiki.sharcnet.ca/wiki/Summer_Schools an annual HPC Summer School] with a variety of in-depth, hands-on workshops.

	SHARCNET also offers a series of online seminars (so-called "General interest webinars"), typically delivered every second Wednesday at lunch time. These are announced via the SHARCNET events mailing list and one can see the schedule at the [https://www.sharcnet.ca/my/news/calendar SHARCNET event calendar]. Past seminars are recorded and posted on [http://youtube.sharcnet.ca our youtube channel]. A full listing of the past webinars is available on the [https://helpwiki.sharcnet.ca/wiki/Online_Seminars Online Seminars] page.		SHARCNET also offers a series of online seminars (so-called "General interest webinars"), typically delivered every second Wednesday at lunch time. These are announced via the SHARCNET events mailing list and one can see the schedule at the [https://www.sharcnet.ca/my/news/calendar SHARCNET event calendar]. Past seminars are recorded and posted on [http://youtube.sharcnet.ca our youtube channel]. A full listing of the past webinars is available on the [https://helpwiki.sharcnet.ca/wiki/Online_Seminars Online Seminars] page.

Jhqin: /* How do I edit my program on a cluster? */

2024-09-17T17:39:39Z

How do I edit my program on a cluster?

← Older revision		Revision as of 17:39, 17 September 2024
Line 400:		Line 400:
	If your desktop supports FUSE, it's very convenient to simply mount your home tree like this:		If your desktop supports FUSE, it's very convenient to simply mount your home tree like this:
	mkdir sharcnet		mkdir sharcnet
	sshfs graham.~~computecanada~~.ca: sharcnet		sshfs graham.alliancecan.ca: sharcnet
	you can then use any local editor of your choice.		you can then use any local editor of your choice.

Jhqin: /* Level 2 */

2024-09-17T17:37:26Z

Level 2

← Older revision		Revision as of 17:37, 17 September 2024
Line 480:		Line 480:
	These include:		These include:
	* Programming support of more modest duration (several days to one month engagement, usually part time)		* Programming support of more modest duration (several days to one month engagement, usually part time)
	* Training on a variety of topics through [https://~~www~~.sharcnet.ca/~~help/index.php~~/Training workshops, seminars and online training materials]		* Training on a variety of topics through [https://helpwiki.sharcnet.ca/wiki/Training workshops, seminars and online training materials]
	* Consultation. This may include user-initiated interactions on particular programs, algorithms, techniques, debugging, optimization etc., as well as unsolicited help to ensure effective use of SHARCNET systems		* Consultation. This may include user-initiated interactions on particular programs, algorithms, techniques, debugging, optimization etc., as well as unsolicited help to ensure effective use of SHARCNET systems
	* Site Leaders play an important role in working with the community to help researchers connect with SHARCNET staff and to obtain appropriate help and support.		* Site Leaders play an important role in working with the community to help researchers connect with SHARCNET staff and to obtain appropriate help and support.