innovative methods to collect road statistics
DESCRIPTION
TRANSCRIPT
Innovative data collection methods for road freight transport statistics
EUROSTAT, Oct 11, 2012
Page 2
Current situation
Limited relevance of data for Austria’s traffic & transport planners due to sample errors on county or smaller traffic cell levels
Insufficient precision of reported tonne-km as a result of automated imputation of the distance travelled between origin and
destination of a journey using distance matrices, ignoring potential detours in between.
Possible over-estimation of empty trips assuming that distances between places of unloading and subsequent places of loading
are empty trips
Assumed under reporting of both empty & loaded trips respondents assumed to minimize efforts, and report a vehicle to be out-of-order during
the sample week.
Limited accuracy of cargo types Respondents have limited information regarding cargo moved („mixed cargo“) Assignment of goods to NST/R classification by respondents leads to incoherent results
Quality concerns regarding road transport data
Consortium
Page 3
Gebrüder Weiss GmbH
Paradigma Unternehmensberatung GmbH
Wirtschaftsuniversität Wien – Inst. f. Transportwirtschaft und Logistik
Petschl Transporte Österreich GmbH & Co KG
Austrian Institute of Technology – Department Mobility
Process knowledge and experience of transport companies
Technology, data management und electronic data exchange
Methods and legal environment of road freight statistics in Europe
Page 4
Goals and Results of the Project
Project Goals Project deliverables
Further reduction of the respondents efforts through automation
Return to a larger sample to meet national requirements
Increase of data quality and actuality
Reduction of required ressources for preparation and processing of the data
Prototypic and fully functional implementation of the connection between data (from companies) to the XML-Interface
Test the applicability of automatic data collection technologies and algorithms to obtain precise measurements.
Legal, economic and methodical evaluation of the results with respect to the road freight transport statistic
Research Objectives
Prove the feasibility of using data available from transport companies building a working prototype, using IT data from transportation companies as a source develop a sufficiently generic standard interface (fleet information, consignment & location
data)
Assess the organizational impact on the respondents obtain empirical information on the benefits as well as potential issues
Use information about goods from transport booking data, to infer cargo types (NST/R) Train a Bayes algorithm, part of the KNIME data mining software to classify goods using
free text
Use GPS location to measure distances travelled and to infer load/unload events. Obtain route information from GPS readings – infer events and compare with order
information
Obtain experience with the technical and economic challenges to implement the standard interface industry software as well as individually developed software
Page 6
Road Transport Statistics in Austria
EU Regulation 70/2012 provides a general legal and methodological framework for the different national surveys (territoriality principle)
Stratified quarterly sample comprises 6,500 vehicles per quarter out of a total of about 72,000 registered. Meets quality criteria described in the regulation Original sample size of 26000 vehicle weeks per year significantly reduced (since 2006)
Local units, operating vehicles and drawn for the sample use paper based questionnaires or an electronic questionnaire (11% in 2009)
Efforts to complete the questionnaire have been reduced. 27.3 minutes on average to complete a questionnaire 150 work weeks per annum for the Austrian economy …
Main task for completing the questionnaires consists of collecting and preparing the information within their companies.
Target Solution Architecture
Data interfaces are in the public domain and provided to the Software- and System developers
Different information entities are consolidated according to the data model specifications and business logic
Completion of missing data and corrections are performed by the respondent
Based on the transfer-format the specific required structure for the respective country questionaire is generated
Page 8
Information elements collected
4 XML-based interface specifications are provided
FleetMasterData
Data on lorries and trailers (capacity, axles, odometer, age, license, etc.)
FleetStatusData
Information on specific lorries at certain times (driven distance, fuel usage, etc.)
ConsignmentData
order related data containing information on goods, packaging, origin and destination …
PositionData
GPS readings, country/ZIP codes, activity (loading, border crossing)
Page 9
Data Collection Service
Protoytped process deployed to move data from IT systems to eQuest
Process activities
Questionnaires as XML files are generated by the eQuest system run by Statistics Austria.
These Questionnaires contain the selection criterias for the data export.
The data export extracts the information out of the ERP/TMS-systems and saves them as XML-files (4 predefined formats).
These files are uploaded to the „SGVS-Konsole“ web-application.
The respondent can now revise the data. The web-application generates a
„completed questionnaire“ and uploads this to the eQuest system.
In the eQuest system the report is finished.
Questionnairewith data
XML-Question-naire
Selection criteria
Data export
ExportedXML data
ERP/TSM
DatabaseAccess
API
Web application „SGVS Console“
eQuest Web-application
Web-Application
Page 10
Validation Rules (Excerpt)
Every lorry or articulated vehicle mentioned in the NSI’s questionnaire must have an entry in the company’s fleet management system.
Odometer readings at the beginning and the end of the reporting period must be available, where the latter has to be greater or equal than the former. If multiple odometer readings are available over time, the sequence of readings must be non-decreasing.
Every shipment must have been allocated to one or more sections of a journey.
If events and activities such as load, unload are reported or inferred from the position data (see below), corresponding sections of journey’s have to be reported as well.
Reported sections and journeys of a given vehicle must not overlap
Automatic classification of goods
Page 12
Official statistics
Respondents
InnoRFDat-X
In compliance with national regulations all transported goods are classified by the NST/R
Hauliers and forwarding agents often use free text in their operative data. (i.e. 10 ldm bathtubs, granite, etc.)
Assignment to NST/R classification is conducted manually.
Development of a model for automatic classification according to NST/R categories for all free texts.
Experiences Assignment of goods to NST/R classification through
respondents leads to incoherent results (DE) Hauliers provide free texts, classification is done by NSO (NL)
Autoteile , Coils Stahl und Blech 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche max . 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Bleche Überbreite 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Coils 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)kompl . Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Leitschienen 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)nahtlose Stahlrohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile 12,2 m 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Profile lt . Beilage 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Rohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Sonderfahrt , Profile 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stabstahl 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl ( S320GD+Z275MB) , 2 Coil 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl Rohre 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahl Vg . 1/7 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)Stahlbleche 5 - Eisen, Stahl und NE-Metalle (einschl. Halbzeug)
Page 13
Automatic classification - experiences
Expect improvements when trained using respondents texts
Correct categorization
Pervasive use of product codes in one case
Transport provider has insufficient information („45 parcels“)
Insufficiently discriminating
Find a balance between „rote learning“ and the capability to correctly classify new descriptions
Imprecisions
Based on an algorithm trained on classification texts and applied to a sample of 1000 cargo descriptions from transport orders
Page 14
Route & Event Detection
Protoytped process deployed to generate trips using GPS data
Objective
Position data file containing the geographic details of every tour where
a tour starts with the loading of an empty lorry
ends with the unloading of the last cargo
Two-Step Heuristics applied
First to detect all stops in the data file
Second to eliminate non-loading/unloading stops
Detect Stops
Speed gradient
Geographic change
Spatial distance
Classify StopsDistance to point of loading
Mandatory rest period
Distance to motorway services, etc.
•Stop-Position•Stop-Duration
•Timestamp•GPS-Coordinates
Loading stopResting stop
Loading point order info:(ZIP-Code)
Event detection performance
Heuristics gave 100% Recall but only 28% Precision
Heuristics + Order details (ZIP-Codes) gave 100% Recall and 85% Precision
Lessons learned
The collection and use of operational available at transport companies to produce road traffic statistics is feasible Standardized interfaces can be cost-effectively implemented
Research implications Use patterns in position data to discriminate between rest and
load/unload stops (independent of consignment information) Respondent specific training sets is expected to increase the precision
of goods classification Generation of data from ERP/TMS-system would allow for continuous
reporting of transport activities
Recommended changes to the legal framework Enable NSI‘s to utilize available IT assets for the collection of raw data Adapt statistical sampling to changes in the market place (freight
forwarders) Uniform collection and production methodologies across EU member
states
Page 18
TMS Software penetration in Austria
Page 18
32
15
14
40
12
14
Sauer Bespoke SWHypersoft, -sped Keine TMSHelpten COSwareC-Logistic Transporeon (?)
Methodology
Contacts to 55 larger companies, o.w. 29 responded and provided information
Significant proportion has outsourced transport: Strabag, Rail Cargo, Lenzing AG, Magna Steyr, Borealis
Respondents represented 7,3 % of all trucks registered in Austria (SGVS 2007, Q4)
5.230 / 72.000 = 7,3 %
The way forward
Solution is not restricted to the Austria territory. The choice of application architecture and scope has been guided by
the vision of a European rather than only a national application.
A Europe wide adoption of the approach is expected to increase quality of the data collected by the member states. applicability of transport statistics all over Europe will be enhanced comparability of transport statistics across countries will be improved.
Piloted process has shown the potential to substantially reduce the administrative burden on reporting companies
Expectation to further raise the efficiency of statistical production processes leading to cost reductions and time savings.
Summary
Model deployed is capable to automatically classify cargo Sufficient precision achieved after training with respondent
specific datasets Less effort required from the respondents Quality improvements as a consequence of consistent
classification
Implementation aspects Improve „training phase“ using both descriptions and pre-
classifications of a sample of respondents Encourage consigners to provide more descriptive data of their
goods Provide functions to manually override misclassifications
Page 20