orientation [Khatri Lab]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
<html>
<h2>
Communications
</h2>
<p>
We have a <a href="https://khatrilab.slack.com">slack page</a> for the lab for most communications (papers, conferences, etc.)
</p>
<p>
Additionally, we have a lab mailing list at khatri_lab_main_list@stanford.edu
</p>
<h2>
  Machines
</h2>
<p>
<ul>
  The Khatri Lab currently operates two servers: <b>khatrilab-dev1</b> and <b>khatrilab-db1</b>. The first is a compute server and the second a database server     that runs MySQL and PostGres (another relational database manager). Each has its own file space. Whenever a program running on dev1 needs to access MySQL, it initiates a database connection to db1 to get what it wants 

In order to access the compute server, one needs to use a terminal (Terminal app on MacOS, putty on Windows).


The address of the compute server is <b>khatrilab-dev1.stanford.edu</b> .
Username and password are your SUnet ID and password, respectively.
</ul>
<h2>
  Security
</h2>
<ul>
  <li>Lab door should never be opened or unlocked when no one is the lab; keys can be obtained from Michele King (mking [at] stanford [dot] edu)</li>
  <li>It is highly recommended that your laptop be cable-secured at all times</li>
</ul>
<p> </p>
<h2>Box</h2>
<ul>
  <li>Box is the only sharing site accepted by the School of Medicine; use no other.</li>
  <li>Box = good; e-mail = bad.<ul>
      <li>DON'T SEND files by e-mail</li>
    </ul>
  </li>
</ul>
<p> </p>
<h2>Remote access &amp; firewalls</h2>
<ul>
  <li>Accessing a local machine or printer behind the firewalls (e.g., a desktop machine or printer) requires Stanford VPN.</li>
</ul>
<p> </p>
<h2>MySQL databasing</h2>
<ul>
  <li>If you have a lot of data and/or you want to make sure you can find it and/or work from a single version, it's a darn good idea to put it in a MySQL database</li>
  <li>Databases hosted on khatrilab-db1 are named according to a specific convention listed at XYZ</li>
  <li>DBs are created by Alex Skrenchuk (alex [dot] skrenchuk [at] stanford [dot] edu), our (part-time) sys admin.</li>
  <li>Don't keep databases on your personal machine. They should reside on khatrilab-db1.</li>
  </li>
  <li>If your project relies on a database hosted in khatrilab-db1, you should create a DB named "proj_&lt;project name&gt;".<ul>
      <li>Try to come up with a short yet understandable (to third parties) name, geez…</li>
      <li>Do not store project tables in user_&lt;your name&gt;.</li>
    </ul>
  </li>
  <li>Work faster/better by use a query editor, e.g., <a href="http://www.sequelpro.com/">SequelPro</a> (Mac), <a href="http://www.sqlmanager.net/products/mysql/manager/">SQL Manager for MySQL</a> (Win) or <a href="http://www.heidisql.com/">HeidiSQL</a> (Win).</li>
</ul>
<p> </p>
<h2>Bitbucket</h2>
<ul>
  <li>
    <em>All</em> published code or database should have their own bitbucket repository</li>
    <li>
    <a href="https://bitbucket.org/khatrilab/metaintegrator_public">MetaIntegrator</a> and <a href="https://bitbucket.org/khatrilab/metaintegrator_public">MetaIntegratorPrivate</a> both have bitbucket pages</li>
</ul>
<p>It's a good idea to create a repository for ongoing projects, even when unpublished and particularly if collaborators are involved</p>
<h2>LabKey, R Studio Server, Galaxy</h2>
<ul>
  <li>All web-based systems require SUNetID and VPN to access </li>
  <li>R Studio Server: <a class="external-link" href="https://rstudio.khatrilab.stanford.edu/" rel="nofollow">https://khatrilab-dev1.stanford.edu/rstudio/</a> </li>
  <li>User accounts on LabKey  </li>
  <li>We have a Galaxy server: <a href="https://khatrilab-dev1.stanford.edu/galaxy/" style="font-size: 14.0px;line-height: 1.4285715;">https://khatrilab-dev1.stanford.edu/galaxy/</a> </li>
</ul>
<p> </p>
<h2>Backups</h2>
<ul>
  <li>Make sure your machine is backed-up … somehow – multiple options are available.</li>
  <li>You must request from Alex that "proj_" databases be backed-up regularly</li>
</ul>
<p> </p>
<h2>Security/HIPAA</h2>
<ul>
  <li>Take the required HIPAA training classes <em>promptly</em>. We are critically dependent on being in compliance, so take this seriously, not to mention a legal requirement. Compliance is monitored, so no escaping this.</li>
  <li>Whole disk encryption: If you deal in personally-identifiable health information (PHI), or if you just want to be extra safe, apply whole disk encryption: <a href="http://itservices.stanford.edu/service/encryption/wholedisk">http://itservices.stanford.edu/service/encryption/wholedisk</a>. Note that this has been deemed untrustworthy and is rumored to change to another product as of this writing.</li>
  <li>Avoid connecting via wireless, as this is fundamentally less secure than the wired network. I.e., if working with sensitive data, you may be broadcasting it... Indeed, while on wireless you are outside the <ac:link ac:anchor="_Remote_access_&amp;">
      <ac:plain-text-link-body><![CDATA[BMIR firewall]]></ac:plain-text-link-body>
    </ac:link>, in addition to low bandwidth compared with the fired network and inability to connect directly to e.g., printers.</li>
</ul>

<h2>Writing code</h2>
<ul>
  <li>Create a project under /projects/&lt;yourname&gt;. This is a backed up area, contrary to /scratch…</li>
  <li>You don't live in a bubble, even your own. Ask yourself: Will you be able to read your code three months from now?</li>
  <li>If something is missing on cluster (e.g., a library), get Alex Skrenchuk to fix it. Installing stuff in your local environment is bad process as it doesn't fix the underlying problem and makes it hard for others to run your code. Inconsiderate coders will be forced to COBOL.</li>
  <li>Use …<ul>
      <li>comments</li>
      <li>meaningful variable names</li>
      <li>subroutines</li>
      <li>functions</li>
    </ul>
  </li>
  <li>Consider diagramming tools to plan and document your code, e.g., <a href="http://www.omnigroup.com/products/omnigraffle/">OmniGraffle</a> (Mac) or <a href="http://office.microsoft.com/en-us/visio/">MS Visio</a> (Win). And no, PowerPoint doesn't cut it for decent diagrams…</li>
  <li>Think failure so write-in error handling, particularly with R<ul>
      <li>Only way to be sure that results are accurate is by being paranoid about code executing correcting. E.g., in Perl, use e.g., use strict, warnings, etc.</li>
    </ul>
  </li>
  <li>Provide a wee bit of header documentation: what does this program do? What does it need? What does it produce? What does it depend on? Who the heck wrote it?</li>
  <li>Use a code editor, or better still, an integrated development environment.<ul>
      <li>E.g., for R, save your neurons and use <a href="http://rstudio.org/">R Studio</a>.</li>
    </ul>
  </li>
  <li>Keep your code in bitbucket; it's the best way to …<ul>
      <li>keep it safe from your newborn</li>
      <li>revert back to previous versions</li>
      <li>enable others to make modifications in a controlled manner</li>
    </ul>
  </li>
</ul>
<h2>Learning to code</h2>
<ul>
  <li>A great <a href="http://tryr.codeschool.com/">introduction to R</a>
  </li>
  <li>
    <a href="http://dss.princeton.edu/training/RStudio101.pdf">Introduction to RStudio</a>
  </li>
  <li>
    <a href="http://vic.gedris.org/Manual-ShellIntro/1.2/ShellIntro.pdf">Introduction to the shell</a>
  </li>
  <li>
    <a href="https://www.codeschool.com/courses/try-git">Introduction to git</a>
  </li>
</ul>
<p> </p>

<h2> Job Submission </h2>


Our khatrilab-dev1 servers use SLURM for submitting computational jobs. 

As an example, save the following as test.sub

<!-- HTML generated using hilite.me --><div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"><pre style="margin: 0; line-height: 125%"><span style="color: #888888">#!/bin/sh </span>

<span style="color: #888888">#SBATCH --mail-type=ALL</span>

<span style="color: #888888">#SBATCH --mail-user=[YOUR USERNAME]@stanford.edu</span>

<span style="color: #888888">#SBATCH --job-name=rscript</span>

<span style="color: #888888">#SBATCH --ntasks-per-node 48 # Number of cores</span>

<span style="color: #888888">#SBATCH --nodes=1 # Ensure that all cores are on one machine</span>

<span style="color: #888888">#SBATCH --mem-per-cpu=5000</span>

<span style="color: #888888">#SBATCH --partition=khatrilab</span>

<span style="color: #888888">#SBATCH --error=job.%J.err </span>

<span style="color: #888888">#SBATCH --output=job.%J.out</span>

<span style="color: #888888">#SBATCH -t 1-00:05 # Runtime in D-HH:MM</span>

module load R/3.2.0

Rscript <span style="background-color: #fff0f0">&quot;$1&quot;</span>
</pre></div>

and run the following from khatrilab-dev1

<!-- HTML generated using hilite.me --><div style="background: #ffffff; overflow:auto;width:auto;border:solid gray;border-width:.1em .1em .1em .8em;padding:.2em .6em;"><pre style="margin: 0; line-height: 125%">sbatch test.sub test.R
</pre></div>



</html>