osg accounting and visualization - indico...osg accounting and visualization this project is...
TRANSCRIPT
OSG Accounting and Visualization
This project is supported by National Science Foundation under Cooperative Agreement OAC-1836650. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Team: Derek Weitzel1, Marian Zvada1, Carl Edquist2
Institutions: 1University of Nebraska—Lincoln, 2University of Wisconsin—Madison
Introduction● OSG Accounting is tasked with recording and visualizing resource utilization on the Open Science Grid● Unique challenges for OSG Accounting include:○ Expansion into new types of resources (public and private clouds, allocation-based HPC resources, and GPU farms),○ Multiple destinations for reporting including WLCG & EGI,○ Record annotation from multiple information sources,○ 100+ accounting sources.
Recent and Future Work● New HTCondor-CE Gratia probe for Hosted CE
● Simplification of visualization interface
● Updated version of ElasticSearch for improvedperformance
● Collect new metrics from data transfers
Total RecordsStored
Job Records Per day
Accounting DataStored
Data Transfer RecordsPer day
Visualization● OSG usage is visualized by hosted Grafana interface
● Data go to WLCG for accounting & pledge calculations
WLCG Integration● Uploads records to APEL accounting message bus
● WLCG normalization (HS06) factors are stored inOSG Topology service
● Collaborate closely with WLCG and U.S. LHC sites for corrections to improve data quality
9.6 Billion 13 TB 1.5 Million 5 Million