Wilke:Using lab cluster: Difference between revisions
(New page: {{WilkeMenu}} =Using the lab cluster= The lab cluster is phylocluster.ccbb.utexas.edu. You will need an account on the cluster to be able to use it. The operating system on the cluster i...) |
No edit summary |
||
| Line 1: | Line 1: | ||
{{WilkeMenu}} | {{WilkeMenu}} | ||
=Using the lab cluster= | =Using the lab cluster= | ||
==General== | |||
The lab cluster is phylocluster.ccbb.utexas.edu. You will need an account on the cluster to be able to use it. | The lab cluster is phylocluster.ccbb.utexas.edu. You will need an account on the cluster to be able to use it. | ||
| Line 15: | Line 17: | ||
Finally, it is good practice to copy your jobs to /state/partition1 before running them. /state/partition1 refers to the local hard drive on the compute node. We will soon provide a tutorial on how to do this exactly. | Finally, it is good practice to copy your jobs to /state/partition1 before running them. /state/partition1 refers to the local hard drive on the compute node. We will soon provide a tutorial on how to do this exactly. | ||
==Storage== | |||
There are several locations for data storage. Which location you should use depends on the nature and size of data you have to store. | |||
* '''Your home directory (<code>/home/''<your user name>''</code>).''' Your home directory is where you will end up by default after logging in to the cluster. Home directories are backed up, and they are generally a good place to store important files and data. However, home directories should not be used for large amounts of data (>10Gb is large). Also, most data (and code) related to research projects should be stored elsewhere (<code>/share/WilkeLab/work</code> or <code>/share/WilkeLab/archive</code>, see below). | |||
Revision as of 13:54, 2 April 2013
The page you are looking at is kept for archival purposes and will not be further updated.
Using the lab cluster
General
The lab cluster is phylocluster.ccbb.utexas.edu. You will need an account on the cluster to be able to use it.
The operating system on the cluster is Linux. If you have no experience with the Linux shell and command line, you will have to learn about that first. This is a good tutorial for beginners: [1]
The cluster uses the Sun Grid Engine (SGE) to distribute computing tasks over the various compute nodes that are available. You will have to start all jobs that you want to run using SGE. This is different from how you run jobs on your computer at home, so even if you have plenty of experience with Linux on your own computer you may not know your way around SGE.
A brief introduction to using SGE is given here: https://wikis.utexas.edu/display/CCBB/sge-tutorial
There's a lot of material in this document. The most important part is that every job on the cluster should use the provided bare-bones script as a starting point: https://wikis.utexas.edu/display/CCBB/sge-tutorial#sge-tutorial-BareBonesScript
If you have many independent runs that can run in parallel and don't need to talk to each other, you should also read this page: https://wikis.utexas.edu/display/CCBB/sge-task-array
Finally, it is good practice to copy your jobs to /state/partition1 before running them. /state/partition1 refers to the local hard drive on the compute node. We will soon provide a tutorial on how to do this exactly.
Storage
There are several locations for data storage. Which location you should use depends on the nature and size of data you have to store.
- Your home directory (
/home/<your user name>). Your home directory is where you will end up by default after logging in to the cluster. Home directories are backed up, and they are generally a good place to store important files and data. However, home directories should not be used for large amounts of data (>10Gb is large). Also, most data (and code) related to research projects should be stored elsewhere (/share/WilkeLab/workor/share/WilkeLab/archive, see below).
