Eszter HorvathUnited Nations Statistics Division

Qatar National Statistics DayDoha, Qatar, 10 December 2013

Modernization of Official Statistics (Session 2)

2

“Never has so much been expected from statistics; never have statisticians had such means at their disposal; and never has there been so much willingness to learn from each other and standardize internationally the outcomes of that learning.”

Handbook of Statistical Organization, Third Edition, 2003The Operation and Organization of a Statistical Agency

http://unstats.un.org/unsd/publication/SeriesF/SeriesF_88E.pdf

3

Why Modernizing Official Statistics?

• Demand• Increased demands and higher expectations for a wider

range of statistics to be made available more quickly.

• Challenge • Failing to address the challenges and opportunities of a

world where data are available in abundance from many sources, sometimes on an almost “real time” basis, will reduce the relevance of producers of official statistics.

• Statistical organizations need to be sufficiently flexible and agile to provide quality statistics quickly, to meet user needs at an acceptable cost. Statistical organizations have to do more with fewer resources.

4

High-Level Group (HLG) for the Modernization of Statistical Production and Services

• The High-Level Group (HLG) for the Modernization of Statistical Production and Services was created in 2010, to coordinate the response of the official statistics community. This group consists of ten heads of national and international statistical organisations, who oversee a modernisation programme to re-invent products and processes and adapt to a changing world.

• The HLG has produced a Strategic Vision and a Strategy to Implement the Vision, both of which have been endorsed by the Conference of European Statisticians. The over-arching theme in these documents is to eliminate the unnecessary diversity in statistical processes and to manage the necessary diversity more strategically.

5

Overview of the Statistical Production Process

• Measurement Framework• Defining the domain• International standards• Metadata

• Data Collection Instruments and Processes

• How to collect• What is the process

• Data Editing, Analysis and Archiving• Output Production

• Types of output• Media mode• Interpretation and explanation

Generic Framework on Generic Framework on Statistical Production ProcessStatistical Production Process

• The global statistical community needs a generic framework to review the statistical production process, with international agreed modules on each of the production process and with technical specifications

• Two models have been proposed. They aim to provide common terminology, improving communication about the production of statistics, within and between organizations. This, in turn, facilitate collaboration and exchange of good practices, leading to greater efficiency.

6

Generic Statistical Business Production Model (GSBPM)

7

Generic Statistical Business Production Model (GSBPM)

The GSBPM: •provides a framework of standard terminology to describe and define the set of business processes needed to produce official statistics;•it is intended to apply to all activities undertaken by producers of statistics, at both national and international levels, which result in data outputs; •it is designed to be independent of the data source, so it can be used for the description and quality assessment of processes based on surveys, censuses, administrative records, and other non-statistical or mixed sources.

8

Generic Statistical Information Model (GSIM)

In addition to the processes described by the GSBPM, the information that flows between those processes (data, metadata, rules, parameters, etc.) is also very important. The Generic Statistical Information Model (GSIM):•aims to define and describe these information objects in a harmonized way; •provides a common language to describe information that supports the whole statistical production process from the identification of user needs to the dissemination of statistical products.

9

Generic Statistical Information Model (GSIM)

10

Integration of Official Statistics with Geospatial Integration of Official Statistics with Geospatial InformationInformation

• The HLG has also approved initial work to explore the relationships between these standards and models with the emerging range of geo-spatial standards. The geographical dimension of data is becoming increasingly important for data integration, analysis and dissemination.

11

New Data Sources: Unstructured New Data Sources: Unstructured Data and Big DataData and Big Data

12

• Does not reside in traditional databases and data warehouses

• May have an internal structure, but does not fit a relational data model

• Generated by both humans and machines• Examples include

• Personal messaging – email, instant messages, tweets, chat

• Business documents – business reports, presentations, survey responses

• Web content – web pages, blogs, wikis, audio files, photos, videos

• Sensor output – satellite imagery, geolocation data, scanner transactions

The value of unstructured data The value of unstructured data sourcessources

• Provide a rich source of information about people, households and economies

• May enable the more accurate and timely measurement of a range of demographic, social, economic and environmental phenomena• Combined with traditional data sources• As a replacement for traditional data sources

• So present unprecedented opportunities for official statistics to• Improve delivery of current statistical outputs• Create new information products not possible with

traditional data sources• Need to be checked against accuracy, relevance,

consistency, interpretability, timeliness, and cost13

14

How to Modernize?

• Modernization of products• Detailed and integrated datasets; geocoded data• Available more rapidly• Combination of various data sources• Data solutions created by the users

• Modernization of production processes• Use of new devices for data collection• Use of SDMX to facilitate the integration of IT systems• Use of common generic business processes across all

statistical domains

• Modernization of organizational and human resources dimensions• Organization to adapt to the new data environment • Staff to be trained and equipped with relevant new

skills

Addressing increasing demand: Restructuring of the NSOs?

• All reorganizations should accomplish the following:• Create new ideas• Lead to efficiency gains• Improve the organization’s focus on strategic

objectives

• However, an organizational restructuring is not necessary to achieve these goals in many circumstances

15

Institutional Set up Today

• Centralized systems• As administrative data is increasingly used for

statistical purposes, more work is undertaken outside of NSOs

• Decentralized systems• Even in these systems, there is always a central

statistical agency• Tendency to improve coordination to improve

reliability and cohesion of statistical system• Centralized/Decentralized

• Recent trend is for these two approaches to be coming together

• Not the issue anymore but we need to:• Coordinate• Modernize• Communicate

16

17

The larger picture: from NSO to NSS: Improving Coordination

• As demand for data increases, consistency across data sources becomes more important (quality assurance framework)

• Improved efficiency can result from sharing infrastructure across statistical agencies within a National Statistical System

18

NSOs Driving the Modernization

• Leadership, vision, strategy• Adequate Statistical Legislation

• Fundamental Principles of Official Statistics endorsed by the ECOSOC and General Assembly provide a strong foundation of NSOs

• Improved structure and coordination – vertical and horizontal

• Rethought business processes along which data are produced and disseminated

• Modernized resource management – human, financial and IT resources • upgrade computers – upgrade human skills

19

Key points

• National Statistical Offices should modernize to survive

• Modernization is not only IT related

• Modernization is strategic – to define the future of Official Statistics

International Seminar on Modernizing International Seminar on Modernizing Official Statistics:Official Statistics:

Meeting Productivity and Data ChallengesMeeting Productivity and Data ChallengesTianjin, China, 24-26 October 2013Tianjin, China, 24-26 October 2013

20

Thank You!