Differences

This shows you the differences between two versions of the page.

--- lab:computing_resources [2018/02/21 13:16]
alexskr [Computing resources]
+++ lab:computing_resources [2019/01/29 13:50] (current)
alexskr
@@ Line 1: / Line 1: @@
 ====== Computing resources ======
-Lab has a number of computing resources - general purpose computing node, rstudio pro server, a database server, a VM environment,  and a small cluster. For most things, BMIR IT team (Alex Skrenchuk) and can grant you access. In order to get access, email bmir-it-team@lists.stanford.edu, cc me asking to deny if inappropriate.
+Lab has a number of computing resources - general purpose computing node, rstudio pro server, a database server, a VM environment, and a small cluster. For most things, BMIR IT team (Alex Skrenchuk) and can grant you access. In order to get access, email bmir-it-team@lists.stanford.edu, cc me asking to deny if inappropriate.
-Access to most of the resources requires connecting to the VPN (https://vpn.stanford.edu).  As a general rule you should use SUNet ID credentials for authentication.  A notable exception is MySQL.  MySQL user name as the same as your SUNet ID and password is stored in the ~/my.cnf file on your server home directory.
+Access to most of the resources requires connecting to the VPN ([[https://vpn.stanford.edu|https://vpn.stanford.edu]]). As a general rule you should use SUNet ID credentials for authentication. A notable exception is MySQL. MySQL user name as the same as your SUNet ID and password is stored in the ~/my.cnf file on your server home directory.
 Please note that none of the computing resources are HIPAA compliant so it should not be used for processing PHI or High risk data unless it is de-identified.
 ===== General Purpose Server =====
-**khatrialb-dev1.stanford.edu  ** is the server that you can use for most of your work.  If you have a project requiring high CPU/Memory needs consider using cluster.
+**khatrialb-dev1.stanford.edu ** is the server that you can use for most of your work. If you have a project requiring high CPU/Memory needs consider using cluster.
+===== Filesystem overview =====
+^Name^Type^Purpose^ Backup/Snapshot ^ Quota ^
+| Home Directory |NFS | Small, important files (source code, executables, configuration files...) | Y/Y |20G |
+| /labs/khatrilab | NFS | Project data, large datasets | Y/N | 55Tb|
+| /labs/khatrilab/users/<SUNetID> | NFS | Individual Lab member Projects | Y/N | |
+| /scratch/labs/khatrilab | NFS | | |
+| /scratch/users/<SUNetID>| NFS | Large, temporary files, raw application output | N/N | |
+| /scratch/labs/khatrilab | NFS | Large, temporary files, raw application output | N/N | |
+| /local-scratch | local | temporary files requiring high I/O |N/N | |
+=== Where should I store my files? ===
+  * personal scripts, configuration files and software installations → home
+  * group-shared scripts, software installations, and large datasets → /labs/khatrilab/users/<SUNetID>
+  * temporary output of jobs, large checkpoint files → /scratch/users/<SUNetID>
+  * Temproray files created by Rstduio (~/.rstudio)→ Create a sym link ~./rstudio to /scratch/users/<SUNetID>/rstuido
 ===== Database server =====
 **khatrilab-db1.stanford.edu** is our database server.
 Login Info
   * login: Your SUNet
-  * Password:  Your initial password is stored in ~/my.cnf
+  * Password: Your initial password is stored in ~/my.cnf
 ===== Rstudio Pro Server =====
-[[https://rstudio.khatrilab.stanford.edu| https://rstudio.khatrilab.stanford.edu]]
+[[https://rstudio.khatrilab.stanford.edu|https://rstudio.khatrilab.stanford.edu]]
 ===== Cluster =====
-We have a BMIR owned cluster (10 nodes) shared with the Shah lab. Access is via khatrilab-dev1.stanford.edu. See [[lab:bmir_cluster|here ]] for details and some usage instructions.
-  *For complete code samples see Parallel processing for tips on how to access from khatrilab-dev1
+We have a BMIR owned cluster (10 nodes) shared with the Shah lab. Access is via khatrilab-dev1.stanford.edu. See [[:lab:bmir_cluster|here ]] for details and some usage instructions.
-  *[[http://slurm.schedmd.com/pdfs/summary.pdf| Cheat sheet]] for SLURM commands
+  * For complete code samples see Parallel processing for tips on how to access from khatrilab-dev1
+  * [[http://slurm.schedmd.com/pdfs/summary.pdf|Cheat sheet]] for SLURM commands