configuring quill condor week 2007
DESCRIPTION
Configuring Quill Condor Week 2007. Execute-Only. Execute-Only. Submit-Only. = Process Spawned. schedd. master. master. master. startd. startd. Typical Condor Pool. Central Manager. = ClassAd Communication Pathway. master. negotiator. collector. What is Quill?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/1.jpg)
Greg ThainComputer Sciences DepartmentUniversity of Wisconsin-Madison
Gthain @cs.wisc.eduhttp://www.cs.wisc.edu/condor
Configuring QuillCondor Week 2007
![Page 2: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/2.jpg)
www.cs.wisc.edu/condor
Typical Condor PoolCentral Manager
master
collector
negotiator
= ClassAd Communication Pathway
= Process Spawned
Submit-Only
master
schedd
Execute-Only
master
startd
Execute-Only
master
startd
![Page 3: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/3.jpg)
www.cs.wisc.edu/condor
What is Quill?
A technology to store a read only version of the job queue and job historical data in a relational database.
![Page 4: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/4.jpg)
www.cs.wisc.edu/condor
Why Quill?
Offloads query overhead from sched Performance boost!
› Easier to make web portal RDMS access easier than SOAP/CLI
![Page 5: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/5.jpg)
www.cs.wisc.edu/condor
Job Queue Management
Job Queue
schedd
quilld
Database
Job Queue
schedd
Without Quill With Quill
![Page 6: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/6.jpg)
www.cs.wisc.edu/condor
Quill downsides
› Additional latency
› More complicated setup
› Handful of attributes not in DBMS
![Page 7: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/7.jpg)
www.cs.wisc.edu/condor
Quill and Quill++
› Quill in Condor since 6.7.11› Quill++ (quillpp) coming soon.
Support for all daemons Multiple schedds in one database Support for Oracle on some platforms Replaces quill
› We’ll talk about both
![Page 8: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/8.jpg)
www.cs.wisc.edu/condor
Typical Quill’d Condor Pool
Central Manager
master
collector
negotiator
= ClassAd Communication Pathway
= Process Spawned
Submit-Only
master
schedd
Execute-Only
master
startd
Execute-Only
master
startd
Database
postgres
query
quillquillcondor_q
![Page 9: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/9.jpg)
www.cs.wisc.edu/condor
Typical Quillpp’d Condor Pool
Central Manager
master
collector
negotiator
= ClassAd Communication Pathway
= Process Spawned
Submit-Only
master
schedd
Execute-Only
master
startd
Execute-Only
master
startd
Database
postgres
query
quillquillppcondor_q
quillpp
quillpp
quillpp
![Page 10: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/10.jpg)
www.cs.wisc.edu/condor
How to use Schema?
› We’ll talk about this in another talk Quill Front End and Schema BoF
• Thursday 11am
![Page 11: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/11.jpg)
www.cs.wisc.edu/condor
Quill (not Quill++) Deployment
› One Quill daemon per schedd
› Quill daemons must be uniquely named
› Each Quill daemon uses a unique DB name
› Currently uses PostgreSQL Recommend PostgreSQL 8.2 or later
• Better disk management
![Page 12: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/12.jpg)
www.cs.wisc.edu/condor
Quill++ deployment
› One condor_quillpp per machine
› One condor_dbmsd per database
› Manual installation of schema
› One DB per pool
› Uses Postgres or Oracle
![Page 13: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/13.jpg)
www.cs.wisc.edu/condor
Condor’s Interface to Quill
› Modified two tools to utilize the DB condor_q condor_history
![Page 14: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/14.jpg)
www.cs.wisc.edu/condor
A User Perspective: condor_q
› condor_q changes When QUILL_ENABLED, goes to rdbms
-name takes a ScheddName or QuillName
-avgqueuetime details average time in queue for all jobs
![Page 15: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/15.jpg)
www.cs.wisc.edu/condor
Condor_q -direct
› -direct rdbms (default when QUIL_ENABLE=true)
› -direct quilld (useful for firewall traversal)
› -direct schedd (100% up-to-date view)
![Page 16: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/16.jpg)
www.cs.wisc.edu/condor
A User Perspective: condor_history
› condor_history changes -name takes a Quill Name to retrieve
job histories from a remote quill’s database
![Page 17: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/17.jpg)
www.cs.wisc.edu/condor
Condor_history -direct
› There isn’t any (yet)
› Condor_history –f \ `condor_config_val HISTORY`
› No –direct quilld equivalent
![Page 18: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/18.jpg)
www.cs.wisc.edu/condor
PostgreSQL Configuration
› Add two special user accounts: quillreader and quillwriter createuser quillreader --no-createdb --no-adduser --pwprompt
createuser quillwriter --createdb --no-adduser --pwprompt
![Page 19: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/19.jpg)
www.cs.wisc.edu/condor
PostgreSQL Configuration (cont)
› Allow TCP/IP connections Edit file postgresql.conf
• Add listen_address = '*'
› Allow connections from specific hosts Edit file pg_hba.conf
• host all quillreader 128.105.0.0 255.255.0.0 password• host all quillwriter 128.105.0.0 255.255.0.0 password
› Note: only use ‘password’ authentication at this time.
![Page 20: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/20.jpg)
www.cs.wisc.edu/condor
Quill Configuration
› User quillwriter needs a password.› Store it in ›$(SPOOL)/.quillwritepassword (quill)›$(SPOOL)/.pgpass (quill++)
.pgpass has host:port:db:user:pass
› Ensure only the condor uid can read it if Condor is running as root
![Page 21: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/21.jpg)
www.cs.wisc.edu/condor
Quill Configuration (cont)
› Condor system specific attributes in file condor_config.local QUILL = $(SBIN)/condor_quill QUILL_LOG = $(LOG)/QuillLog QUILL_ADDRESS_FILE = $(LOG)/.quill_address DAEMON_LIST = …, QUILL VALID_SPOOL_FILES = …, .quillwritepassword DC_DAEMON_LIST = …, QUILL
![Page 22: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/22.jpg)
www.cs.wisc.edu/condor
Quill Configuration (cont)
› Quill specific attributes QUILL_ENABLED = TRUE # The quill name must be unique across all # quill daemons AND schedds QUILL_NAME = [email protected] QUILL_DB_NAME = psilord_db QUILL_DB_IP_ADDR = merlin.cs.wisc.edu:42999 QUILL_POLLING_PERIOD = 10 (seconds)
![Page 23: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/23.jpg)
www.cs.wisc.edu/condor
Quill Configuration (cont)
› QUILL_HISTORY_CLEANING_INTERVAL = 24 (hours)› QUILL_HISTORY_DURATION = 30 (days)› QUILL_MANAGE_VACUUM = FALSE› QUILL_IS_REMOTELY_QUERYABLE = TRUE› QUILL_DB_QUERY_PASSWD = xxx
![Page 24: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/24.jpg)
www.cs.wisc.edu/condor
Schema management
› Quill automatically loads schema Upgrades itself automatically
› Quill++ requires manual loading: Psql –
Uquillwriter<common_createddl.sql Psql –Uquillwriter<pgsql_createddl.sql
![Page 25: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/25.jpg)
www.cs.wisc.edu/condor
Conversion to Quill++
› Conversion only matters for history
› Conversion is one-way-only!
› Two steps: Dump quill history tables to file with
• Condor_dump_history Load quill++ history tables from file
with• Condor_load_history
![Page 26: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/26.jpg)
www.cs.wisc.edu/condor
Data Management
› Constrain database size History truncation
• Quill++ other tables, too Postgres Index management Oracle cleans itself
› Careful of long queries, esp with Quill
![Page 27: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/27.jpg)
www.cs.wisc.edu/condor
Data Management: Quill
› HISTORY_CLEANING_INTERVAL In hours (24 hours)
› HISTORY_DURATION How long in days (7 days)
› QUILL_SHOULD_REINDEX Boolean (false)
› QUILL_MANAGE_VACUUM (false)
![Page 28: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/28.jpg)
www.cs.wisc.edu/condor
Data Management: Quill++
› Condor_dbmsd does all the work QUILL_DBSIZE_LIMIT (20 Gb)
– Emails warning when 75% is hit
DATABASE_PURGE_INTERVAL (s (24 hours)) DATABASE_REINDEX_INTERVAL (s (24 hours)) QUILL_DB_TYPE (oracle, pgsql) QUILL_RESOURCE_HISTORY_DURATION (7 days) QUILL_JOB_HISTORY_DURATION (10 years!) QUILL_RUN_HISTORY_DURATION (7 days)
![Page 29: Configuring Quill Condor Week 2007](https://reader035.vdocuments.net/reader035/viewer/2022070409/56814457550346895db0f2be/html5/thumbnails/29.jpg)
www.cs.wisc.edu/condor
Thank you!
› Want more information?
› BOF “Databases in Condor”