cluster computing at iqss alex storer, research technology consultant
TRANSCRIPT
![Page 1: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/1.jpg)
Cluster Computing at IQSS
Alex Storer, Research Technology Consultant
![Page 2: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/2.jpg)
What is the RCE?
Research Computing Environment
For research in the social sciences
Get an account! [email protected]
![Page 3: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/3.jpg)
How do I access it?
![Page 4: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/4.jpg)
How do I access it?
![Page 5: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/5.jpg)
How do I access it?
![Page 6: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/6.jpg)
How do I access it?
![Page 7: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/7.jpg)
What are your needs?
Gigantic Process Many Processes
![Page 8: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/8.jpg)
Gigantic Process
RCE Powered
![Page 9: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/9.jpg)
Gigantic Process
RCE Powered
Applications
Request Up to 256gb RAM
Run a job for up to 5 days*
Graphical/Windowed
experience of Stata, Matlab,
etc.
![Page 10: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/10.jpg)
RCE Powered
![Page 11: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/11.jpg)
Many Processes
Input Output
![Page 12: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/12.jpg)
Many Processes
Input 1 Output 1Input 2 Output 2
Input 3 Output 3Input 4 Output 4
Input 5 Output 5
![Page 13: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/13.jpg)
Many Processes
Condor• Schedules which jobs go to which available machines
• Called from the command line
• Reads in 'submit' files
![Page 14: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/14.jpg)
Example: Simulating
You have a model that takes 30 minutes to run and computes a result
You want to establish confidence intervals for this number by running it many times
![Page 15: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/15.jpg)
Example .submit file
![Page 16: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/16.jpg)
Example .submit file
What command do I run?
![Page 17: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/17.jpg)
Example .submit file
What arguments do I give to the command?
![Page 18: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/18.jpg)
Example .submit file
What input do I give to the command?
![Page 19: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/19.jpg)
Example .submit file
Where do I save the outputs?
![Page 20: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/20.jpg)
Example .submit file
How many times do I run this?
![Page 21: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/21.jpg)
Example .submit file
/usr/bin/R --no-save --vanilla < simulate.R > out.1/usr/bin/R --no-save --vanilla < simulate.R > out.2.../usr/bin/R --no-save --vanilla < simulate.R > out.10
Unix Standard Input/Output
![Page 22: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/22.jpg)
How to submit a file
condor_submit simulate.submit
![Page 23: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/23.jpg)
Your out.0 file
It's just everything that R writes to the screen from the script!
![Page 24: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/24.jpg)
Example: Simulating
Instead of using the out.$(Process) structure, you can save the data in your script
You cannot expect the Processes to complete in order!
You shouldn't write to the same file until all processes are complete
Instead of calling a script, use a function
![Page 25: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/25.jpg)
Example: Simulating (with a function)
• procid is an input
• We tell the function what to save and where to save it
![Page 26: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/26.jpg)
Example: submitting a function in batch
Execute this command in R. Specifically, run the simfunction.R file which we defined on the previous slide.
![Page 27: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/27.jpg)
Example: submitting a function in batch
Execute this command in R. Specifically, call the function sim.function with the input as $(Process).sim.function(0)sim.function(1)…
![Page 28: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/28.jpg)
Example: submitting a function in batch
We are no longer using the standard input and standard output, so we can leave these blank.
![Page 29: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/29.jpg)
MATLAB Example
An example where we need to do the same thing to a number of data files and write out the results
Call a function which knows how to map the process ID to the data to load
![Page 30: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/30.jpg)
MATLAB function
![Page 31: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/31.jpg)
MATLAB function
We will pass $(process) as the function input
![Page 32: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/32.jpg)
MATLAB function
Try to load:
data_0.mat
data_1.matetc.
![Page 33: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/33.jpg)
MATLAB function
Compute the relevant result.
![Page 34: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/34.jpg)
MATLAB function
Save the results as:
result_0.mat
result_1.matetc.
![Page 35: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/35.jpg)
.submit File Example
The Arguments section is the most important, let's look at each piece individually
![Page 36: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/36.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
Start the arguments with double quotes (")
![Page 37: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/37.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
-nodisplay tells Matlab to not pop up the GUI
![Page 38: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/38.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
-singleCompThread tells Matlab to use only one core (this is what condor expects)
![Page 39: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/39.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
-r tells Matlab to execute whatever commands come next.
![Page 40: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/40.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
Put the commands to run in single quotes.
![Page 41: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/41.jpg)
.submit File Example
Do NOT try to write your entire Matlab script in the submit file!
Some arguments must be executed before calling your script, however…
![Page 42: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/42.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
The commands to Matlab will go on inside the single quotesThey must be on a single line!
![Page 43: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/43.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
setenv(''HOME'',''nfs/home/A/astorer''); cd(''/nfs/home/A/astorer/Work/outreach/matlab''); mytest($(PROCESS))
![Page 44: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/44.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
setenv(''HOME'',''nfs/home/A/astorer''); cd(''/nfs/home/A/astorer/Work/outreach/matlab''); mytest($(PROCESS))
setenv is required for Matlab to load your local preferences.You must use two single quotes instead of one single quote.Remember to set your own home directory, e.g. nfs/home/J/jdoe
![Page 45: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/45.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
setenv(''HOME'',''nfs/home/A/astorer''); cd(''/nfs/home/A/astorer/Work/outreach/matlab''); mytest($(PROCESS))
Change to the directory that contains the script you want to run.
![Page 46: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/46.jpg)
.submit File Example
Arguments = "-nodisplay –singleCompThread –r ''"
setenv(''HOME'',''nfs/home/A/astorer''); cd(''/nfs/home/A/astorer/Work/outreach/matlab''); mytest($(PROCESS))
Finally, run the function on the $(PROCESS) variable.
![Page 47: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/47.jpg)
Example submission
Because our function reads in data, we generate the data ahead of time
This is what is in our directory before submitting (ls lists directory contents)
Notice that we count starting from 0!
![Page 48: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/48.jpg)
Example submission
Use condor_submit to submit the file
Depending on the job, this may take some time to complete!
![Page 49: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/49.jpg)
Example submission
Use condor_q <username> to check the status of your jobs
Use condor_rm <username> to clear your jobs.
![Page 50: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/50.jpg)
Example submission
Use condor_q <username> to check the status of your jobs When this returns with no result, your jobs are complete.
![Page 51: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/51.jpg)
Example submission
Results!
![Page 52: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/52.jpg)
Stata Example
Universe = vanillaExecutable = /usr/local/bin/stata-mpArguments = donotification = Completenotify_user = [email protected]
input = Test.dooutput = Test.outerror = Test.errLog = Test.logQueue 1
This is like running stata-mp do Test.do
![Page 53: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/53.jpg)
Notification!
Universe = vanillaExecutable = /usr/local/bin/stata-mpArguments = donotification = Completenotify_user = [email protected]
input = Test.dooutput = Test.outerror = Test.errLog = Test.logQueue 1
You can get e-mails when your job is done!
![Page 54: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/54.jpg)
Debugging
If your results aren't as expected, first check the error files
![Page 55: Cluster Computing at IQSS Alex Storer, Research Technology Consultant](https://reader035.vdocuments.net/reader035/viewer/2022062511/551bfd4e550346ad4f8b4a39/html5/thumbnails/55.jpg)
My jobs never finish?!
Sometimes, jobs aren't well formed and condor won't know what to do
Condor will hold these jobs
Your submit file is probably wrong somehow – try looking at the log file as well as the submit file
H stands for "Held"