TRLs serve as metric in gauging progress

Country music singer Kenny Chesney has a hit song that says “I went home at 2 with a 10 and woke up at 10 with a 2.” Similar subjective choice variations can also be said about the technical readiness level (TRL) process used by a variety of industries to determine the maturity level of a technology and often then for gauging its commerciality. The process has been successfully used for years, although the scale, definitions and process varies, and there is now some science and logic to it. It can be practically used in the petroleum industry in gauging technological progress and commerciality of new developments.

To the technology developer, his or her baby may look like a “10,” but investors worry about the nightmare of waking up with a technology that in the market is a “2.” And for those investors—or company managers—there’s a serious need for some science to the metric if a long-term relationship is at stake.

Continuing the country music theme, investors seriously want to “know when to hold ’em, know when to fold ’em, know when to walk away, know when to run,” as Kenny Rogers sings in “The Gambler,” and TRLs can be a metric to help make that call.

ILLUSTRATION 1. The nine-level TRL scale used by the DOE—more ‘down to earth’ than NASA—still lacks practical definition for E&P. (Reference: DOE G 413.3-4A 9/15/2011. Source: RPSEA)

The American Petroleum Institute addressed the need for TRLs in a recommended practice called API RP 17N. Leading oil companies employ TRLs as a tool to track technology development on an ongoing basis and to guide toward desired outcomes; Total has been a pioneer in that. In Houston the oil company-led deepwater technology consortium DeepStar has employed TRLs in similar ways in recent years. Before that, NASA, the U.S. Department of Defense, the U.S. Department of Energy (DOE) and the U.S. nuclear industry all developed their own scales of TRLs. The practice has seen wide application by similar organizations outside the U.S.

The Research Partnership to Secure Energy for America (RPSEA) has determined the TRL process can assist it in prioritizing its R&D efforts to meet its goal to attempt to not only develop technologies that improve U.S. reserves and productivity in a safe and environmentally friendly manner but also demonstrate their effectiveness and lead to commercialization. Examples from four projects illustrate RPSEA’s use of TRLs.

Value to regulators, investors and end users

Regulators accepting a new technology/system in a high-risk permit application may use TRL values as a metric on technology maturity to help in this kind of difficult decision. Investors making risk-based decisions similarly may find TRL numbers to be a useful, quantifiable measure as opposed to solely relying on industry judgment by people they know.

The industry frequently conducts a hazard identification exercise called HAZID before a new project is commissioned, a subjective exercise employing subject matter experts (SMEs) in a process that has made a significant contribution to improving safety and reducing risk. The same can be said for TRLs.

Objective or subjective?

There is a distinction here between: (i) subjective choices based on a single nonexpert opinion, (ii) focus groups used in political or market research assessments, (iii) carefully weighed views from a panel of experts and (iv) absolutes such as the laws of physics. TRLs fall under category (iii). A standard and pre-agreed scale of what defines each TRL is used to calibrate and hence coordinate the judgments of a panel of experts making TRL determinations.

The TRL methodology was originated by Stan Sadin at NASA in 1974. In that era NASA was faced with developing many new systems where “failure was not an option,” but there was little precedent to go on. The urgency of a search for a method to improve success for new developments from the early stages through execution along the lines of the TRL concept did not have to be space-related for the methodology to be of value; 50 years later the concept is used in many industries.

The NASA TRL scale evolved as a nine-level scale to cover technology development stages from a new idea to a reliable working embodiment of the new technology along the lines of Illustration 1. The DOE has had its own typical development process, leading to an adaptation of the nine-level scale with a set of definitions that better suited DOE projects but lacked proper practical definition for the upstream oil and gas industry.

It became clear to TRL users that the language used in the definitions of each TRL was important to enable the scale to be applied practically on the ranges of technologies within RPSEA’s ultradeepwater program. RPSEA adjusted its TRL scale to better suit the characteristics of projects with a somewhat different set of TRL definitions and with language somewhat similar to the API RP 17N scale but more tailored to petroleum industry systems as opposed to hardware. It is a seven-level scale adopted from DeepStar because it was felt that the language of the definitions worked well and was used by many people in the industry community, embracing everything from relatively small investments to very large investments in the ultimate embodiments of technologies examined. The RPSEA TRL scale is shown in Illustration 2.

ILLUSTRATION 2. The seven-level TRL scale adopted by RPSEA is used for technologies being developed for the petroleum industry. (Source: RPSEA)

The broad value of TRLs may have been underestimated. Part 2 of this article will demonstrate TRL use of a fit-for-purpose process for gauging technology progress—one that is simple and consistent and that provides value, recognizing how focus on process over function can compromise the intention.

A TRL process that can be easily understood by service providers, investors, research consortia and governments will increase the likelihood of commercialization of needed technologies.

Practical guidelines

Experience with TRL votes has shown that understanding the TRL process is based on SMEs that had a background in field development activities such as drilling, well services and production facilities, all of which see a fairly broad application of a variety of technologies that were recent or even relatively untried. It is suggested that at least five and not more than about a dozen SMEs be used to perform the TRL analysis. Adding more SMEs may get cumbersome, and fewer than five may result in “groupthink.” The SMEs might be individuals who would use the technology but cannot be enthusiasts for the technology or, worse still, be “sales monkeys” working for the originator of the technology. Responsible, professional independence is obviously critical.

It is recommended that a knowledgeable, professional and open-minded facilitator with no vested interest in the technology be chosen to lead the discussion and assessment. The facilitator is not essential but can often draw out the thinking of the SMEs.

Another principle that had to be addressed: The TRL needs to be an individual opinion and not that of the SME’s employer. In effect, the TRL vote is part of a survey.

The value of that “survey” is dependent on the background of the people who take it. A panel composed of SMEs with operating companies, service providers, manufacturers, academia and consultants that might choose to use the technology in their core business is likely to be more significant than a panel of generally knowledgeable people.

There is significant research backing up the quality of judgment made by a panel of well-qualified experts—the Delphi Method developed by Rand Corp. for the U.S. Air Force Strategic Air Command in the 1950s and 1960s. The Delphi Method was used on national security matters quite separate from technology. It relies on isolated opinions from SMEs of established judgment and frame of reference to assess a situation that may not have any proven guidelines or precedents, with objectives of seeking maximum quality and validity in judgment on possible future actions. Those people can later be brought together to amass their opinions and attempt to form a consensus if indeed one might exist.

Acknowledgment
The authors are pleased to have the permission of RPSEA to cite (i) the frame of reference of RPSEA in using TRLs and (ii) the experience with the specific RPSEA projects referred to in the examples.