linux cluster and distributed resource manager
DESCRIPTION
Linux Cluster and Distributed Resource Manager using Shell ScriptTRANSCRIPT
![Page 1: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/1.jpg)
Linux Cluster and Distributed Resource
Manager
Center for Genome Science, NIH, KCDCHong Chang Bum
![Page 2: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/2.jpg)
Linux Cluster
• 3 Linux Cluster Machine
• KHAN
• KGENE
• BIO
![Page 3: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/3.jpg)
KHAN
• Total 94 Nodes (1 Master + 94 work Node)
• 64Bit PowerPC 720 4-way, 16GB RAM
• 64Bit PowerPC 770 2-way, 2GB RAM
• User Space: /home1 1.6TB
• Scratch Space: /home2/scratch 935GB
• Software: /home1/biosoftware
• EIGNSTRAT, merlin, phase, plink, R ...
![Page 4: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/4.jpg)
DRM
• Distribute Resource Manager
• Job Scheduler
• IBM LoadLeveler, PBS, OpenPBS, Torqueue, Grid Engine, Sun Grid Engine(SGE), LSF
• Job Distribute, Job Status, Queue status, Node Status, User Accounting
![Page 5: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/5.jpg)
1,000 Serial Job
• 1,000 Data Sets
• input.1, input.2... input.1000
• program -i input.(1~1000) -o output.(1~1000)
• 1,000 Job Submit Scripts
• 1,000 times Queue Submit Commands
![Page 6: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/6.jpg)
Job Array - LoadLeveler
• Large Data sets: 1,000(input.1 ~ input.1000)
1,000 times??
![Page 7: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/7.jpg)
Using Script
![Page 8: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/8.jpg)
1,000 Command Script
1,000 Job Submit Command
![Page 9: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/9.jpg)
1,000 Command Script
![Page 10: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/10.jpg)
1,000 Job Submit Command
Job Status
![Page 11: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/11.jpg)
Using R Scripts• Interactive Program
• Need to use R to analyze your data
Convert Bash Shell Script??
![Page 12: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/12.jpg)
--quiet, -q Don’t print startup message--no-save Don’t save it
Using I/O redirection Form
![Page 13: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/13.jpg)
R and LoadLeveler
![Page 14: Linux Cluster and Distributed Resource Manager](https://reader034.vdocuments.net/reader034/viewer/2022052321/554bcc23b4c9058f6c8b476c/html5/thumbnails/14.jpg)