QC/QA PROCESSES REFINED IN CALIFORNIA – Pathway Services Inc. partners with Caltrans to create a QC/QA plan for APCS that improves data quality.

Caltrans White 300_300.png

In January 2019, Caltrans Senior Transportation Engineer, Haiping Zhou, presented a paper discussing the implementation of QC/QA working processes from their newly established data quality management plan at the Transportation Research Board Annual Meeting. The successes defined via a 2018 pilot project in cooperation with Pathway Services are summarized in this article.

The collection of data via pavement condition surveys has long been instrumental in understanding the status of state roadways by approximating their future condition, assigning levels of repair priority based on those condition assumptions and estimating the funding required for ongoing road maintenance. While manual pavement evaluation and distress rating continue to play a role in determining pavement condition, emerging technologies have made automated pavement condition surveys (APCS) a more efficient, reliable and safer alternative for the collections and delivery of this critical information.

Several years ago, the California Department of Transportation (Caltrans) began to evaluate the effectiveness and reliability of APCS systems, and in 2018 the state invited Pathway Services Inc. to participate in a pilot program after issuing a three-year, 58,000 lane miles collection contract. The purpose of the pilot was to assess the Quality Controls (QC) and Quality Assurance (QA) protocols established by the state’s data quality management plan (DQMP) in accordance with the Federal Highway Administrations Code of Federal Regulations (CFR) Part 490.319 (C) and to determine a workable process for their implementation.

The pilot assessed 113 lane miles in Orange County (Caltrans District 12) which represented roughly 5% of the total lane miles for the district and evaluated both asphalt and JPCP pavement types. To collect the data, Caltrans relied on Pathway Services’ independently certified PathRunner, integrating the latest pavement surveying technologies and using a South Dakota type laser profiler meeting ASTM E950 standard. See pathwayservices.com/pathrunner for a complete listing of PathRunner systems and specifications.

Prior to collection, it was important that Pathway Services establish targeted data thresholds based on Caltrans pavement priority matrix that includes IRI, cracking, and faulting as the basis for QC. Pathway Services would be tasked with the first steps in the new QC/QA process beginning with Field QC by reviewing samples of collected images, and distress and sensor data. A second stage of QC required additional review of data at the Pathway Services main office in Tulsa, Oklahoma where images and data were again to be sampled by independently certified Quality Control staff. Any errors undiscovered by field sampling would be addressed prior to submission to Caltrans through use of integrated proprietary software tools and, if necessary, through the recollection of affected road sections. Potentially addressed issues might include the collection of incorrect lane collection assignments due to traffic, impaired image quality from road congestion, improper distress identification and evaluation etc. According to the Caltrans DQMP, a contractor would typically be required to check data accuracy via an assessment of scheduled data volume for submission versus actual submitted volume and a review of a gap/overlap report, but for this pilot because the lane miles were pre-defined, a gap/overlap report was not required.

PATH VIEW

Quality Assurance review belonged to Caltrans beginning with their data completeness verifications. All data submitted was compared against the data requirements of the APCS contract for completion. Once in hand and verified, images, distress data and sensor data were then strenuously evaluated versus the required parameters documented in the DQMP ensuring they fell within the thresholds of acceptance requirements. Some of the specifically listed acceptance parameters included image clarity, continuous image stitching, geographic synchronization, distress identification and measurement, and ground truth comparison. Caltrans also completed manual distress and sensor verification with an extensive visual field evaluation versus Pathway Services’ submitted data and using alternate profiling equipment to verify IRI repeatability and reproducibility as compared to PathRunner IRI readings.

The data upload process was implemented to test the linear referencing system (LRS) and postmiles logged by a vendor’s APCS as compared to Caltrans’ pavement management system (PaveM) and to determine gaps or overlaps in the data. This process would allow the APCS location data to synchronize with that in PaveM.

Finally, Caltrans intended to perform a data consistency check by comparing 2015 and 2016 data to the 2018 data collected in the pilot. The primary objective of the analysis was to determine if a specific road section changed from a lower graded level to an increased level without scheduled repair or maintenance. The reversal of a pavement condition rating would suggest possible data inconsistency and would warrant attention.

After an initial QC/QA process loop was completed, the pilot data determined that adjustments were required to address image quality obscured by natural interruption. Fine tuning of data reporting from Pathway Services’ PathView software was also necessary to more consistently represent Caltrans’ distress definitions and measurements. However, IRI values from Caltrans’ inertial profiler and the South Dakota type on Pathway Services’ PathRunner were reproduceable with 95% confidence. Addressing data field naming convention inconsistencies in the data sets solved some initial data uploading errors.

It was expected that testing the working processes in Caltrans’ newly adopted QC/QA program would discover data quality and consistency variances between manual and APCS collection. Once discovered and communicated, Caltrans determined the numerous improvements to Pathway’s APCS system and data analysis would be tested against a larger data set collected in 2018. After initial review of the pilot’s QC/QA processes, Caltrans ultimately determined that the collaboration between their pavement management staff and Pathway Services Inc. worked extremely well in refining those processes, ensuring data quality with the adoption of APCS.

After Pathway Services’ implementation of requested improvements to software algorithms and working processes, a statewide collection was performed in 2018. The Caltrans QA results indicated: the statewide image quality was accepted with a general review reporting a passing rate of 96.8%; the statewide data and sensor verification were accepted for both asphalt and concrete pavements reporting a passing rate of 99.3% and 96.5%, respectively; data submission was found to be complete regarding contractually required items, in format and in legibility; and data upload encountered no errors and was loadable into Caltrans’ PaveM. As of the end of 2018 the data consistency check was still in process. The statewide data evaluation proved to be a resounding success both in the cooperative effort between Caltrans and Pathway Services Inc. in determining best data quality assessment practices and in the resulting data quality.

Pathway Services Inc. currently services the APCS data collection needs of the California Department of Transportation, and since the 2018 pilot program, has continually pursued data quality improvement opportunities in partnership with Caltrans.

Link to TRB paper: journals.sagepub.com/doi/full/10.1177/0361198118821374