Submit First Job

In this tutorial we will be using a Python script to calculate the first million digits of PI using the ACCRE cluster. Although we will be using Python code, you don’t need to know Python for this tutorial. However, you will need an ACCRE account and should be familiar with Linux commands.

If you haven’t done so already, log in to ACCRE using a terminal.

Lmod

Lmod is a tool for sharing modules within a high-performance computing environment (e.g. the ACCRE cluster). We will be loading the Anaconda3 module, which allows us to use many Python libraries that would otherwise be unavailable.

In your terminal, type in the following:

module load Anaconda3

Lmod commands are case sensitive; make sure you capitalize the A in Anaconda3.

While Anaconda is fine for this tutorial, if you end up using Python for your computational research projects, we strongly recommend using the ACCRE-optimized Python builds on the cluster instead of Anaconda.

Choosing a text editor

There are many text editors available for Linux. For new users we recommend nano, which is simple and easy to use. Other text editors available are vim, which is more powerful but has a steeper learning curve, and emacs, which is extensible and customizable. We will use nano for this tutorial, but much documentation on vim and emacs is available online.

To start creating the file below with nano, type in nano pi.py. To close nano, press Control-X or Command-X. You will then be asked if you want to save the file or discard changes.

Writing the Python script

Create a file in your home directory called pi.py which includes the following:

import sys
from time import time
from mpmath import mp

digits = int(sys.argv[1])
mp.dps = digits

print("Pi is approximately equal to:")

start_time = time()
print(mp.pi)
end_time = time()
seconds = end_time - start_time

print("Done, {} digits in {} seconds".format(digits, seconds))

This is a Python script that calculates the value of pi using the mpmath library, which is included in the Anaconda module.

If you’re curious, here’s what it does, line by line:

Imports the necessary Python libraries, which Anaconda provides for us
Gets the number of digits from the command line argument and uses it to set the decimal precision
prints “Pi is approximately equal to”
starts the timer
calculates and prints the value of pi
stops the timer
calculates the time it took to calculate pi
prints the summary

Check to see if it works for small values: enter python pi.py 1000.

Pi is approximately equal to:
3.14159265358979323846264338327950288419716939937510582097494459230781640628
6208998628034825342117067982148086513282306647093844609550582231725359408128
4811174502841027019385211055596446229489549303819644288109756659334461284756
4823378678316527120190914564856692346034861045432664821339360726024914127372
4587006606315588174881520920962829254091715364367892590360011330530548820466
5213841469519415116094330572703657595919530921861173819326117931051185480744
6237996274956735188575272489122793818301194912983367336244065664308602139494
6395224737190702179860943702770539217176293176752384674818467669405132000568
1271452635608277857713427577896091736371787214684409012249534301465495853710
5079227968925892354201995611212902196086403441815981362977477130996051870721
1349999998372978049951059731732816096318595024459455346908302642522308253344
6850352619311881710100031378387528865875332083814206171776691473035982534904
2875546873115956286388235378759375195778185778053217122680661300192787661119
5909216420198
Done, 1000 digits in 0.0012884140014648438 seconds

Writing the SLURM script

Before we can submit the Python script to the scheduler, we will need to write the SLURM script. This instructs the scheduler how to run the Python script and what to do with the results.

Create a file called pi.slurm that includes the following. Replace your.email.here@vanderbilt.edu with your email.

#!/bin/bash
#SBATCH --mail-user=your.email.here@vanderbilt.edu
#SBATCH --mail-type=ALL
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --mem=2G
#SBATCH --time=0:20:00
#SBATCH --output=pi.txt

module load Anaconda3
python pi.py 1000000

The first part of the SLURM script is a series of #SBATCH directives. These tell the SLURM scheduler to do the following:

send updates to the specified email
send emails for any update of the task: begin, end, fail, or requeue
one node
one process (task) running on one CPU core
2GB of memory
20 minute maximum time
write the output to pi.txt

The next part has Linux commands we would like to run:

load the Anaconda module through Lmod
calculate pi to one million digits

Starting the SLURM script

Type the following to send the SLURM script to the scheduler:

sbatch pi.slurm

You will now see the following:

Submitted batch job (large number here)

This large number is the job ID.

In your email, you should receive an alert from SLURM letting you know that the job has started.

While the job is running or in queue, you can check the status of the job by typing:

squeue --job (job id)

Getting your results

In less than a minute your job will complete. You will get a second email that says your job is complete. In addition, the squeue command above will return nothing.

As you can imagine 1,000,000 digits of pi are difficult to display on one screen, so let’s get the beginning and end of the file. Type in:

head -c 1000 pi.txt

You will see the following

Pi is approximately equal to
3.14159265358979323846264338327950288419716939937510582097494459230781640628
6208998628034825342117067982148086513282306647093844609550582231725359408128
4811174502841027019385211055596446229489549303819644288109756659334461284756
4823378678316527120190914564856692346034861045432664821339360726024914127372
4587006606315588174881520920962829254091715364367892590360011330530548820466
5213841469519415116094330572703657595919530921861173819326117931051185480744
6237996274956735188575272489122793818301194912983367336244065664308602139494
6395224737190702179860943702770539217176293176752384674818467669405132000568
1271452635608277857713427577896091736371787214684409012249534301465495853710
5079227968925892354201995611212902196086403441815981362977477130996051870721
1349999998372978049951059731732816096318595024459455346908302642522308253344
6850352619311881710100031378387528865875332083814206171776691473035982534904
28755468731159562863882353787593751957781857780532171226806

Similarly, to get the end of the file:

tail -c 1000 pi.txt

You will see the following:

2196379757651924521867096088092137111977500087815930430729344883930957574159
2413752859777972918934538505080383198677459002518657917237080857416429715380
7884060713068680361982419715774763895072534684045691927595319372237022290155
8006560760473854735990447799674874996976942713766869553319512533776409858709
6683863263926164945608684140374568420719405950701743035469182150900466493998
5517413893851975731215682616228622318810967297476060130283311937161140874727
0676255856777511995666748615196491297019331808499410961813929649278936090212
5354433273750642606242994120327362558244174983450947309453436615907284163193
6830757197980682315357371555718161221567879364250138871170232755557793022667
8580319993081083057630765233205074001393909580790163771762925928376487479017
7274125678190555562180504876746991140839977919376542320623374717324703369763
3579258915152603156140333212728491944184371506965520875424505989567879613033
1164628399634646042209010610577945815
Done, 1000000 digits in 68.81519937515259 seconds

Reference: StackOverflow