Chasing Quality Isn't Easy. But An FDA Pilot Aims To Boost Quality By Appraising The Capability Of Manufacturing Sites
The pursuit of quality can be a daunting task for device-makers. One wrong step can cause costly product recalls affecting patient health – among other troubles tied to poor quality products and processes. To help firms move toward a goal of best-in-industry quality, US FDA, through its Case for Quality initiative, has convened a pilot program to measure a manufacturer's capability and maturity to help put it on a path to continuous improvement. The pilot uses an industry-tailored version of the Capability Maturity Model Integration (CMMI) model and method, developed jointly by FDA, industry and CMMI Institute.
The first of a multi-part Medtech Insight feature series on the appraisal of manufacturing capability and maturity, and what it means for the medical device industry.
[Editor's note: This is the first of a multi-part Medtech Insight feature series on the appraisal of manufacturing capability and maturity, and what it means for the medical device industry. Check out Part 2 at https://bit.ly/2K6TWRV.]
For many medical device manufacturers, the pursuit of quality for products and processes can seem elusive. Poor quality can lead to – among many other undesirable outcomes – product recalls and injured patients. These can ultimately put a dent in a company's reputation and potentially cost it a lot of money to clean up the mess.
That's why US FDA, along with the Medical Device Innovation Consortium (MDIC), has been aggressively pushing the Case for Quality initiative for several years to convince device-makers to make quality an organization-wide concern.
After all, "quality is really about delivering your best product or your best outcome for your customer, one way or another," said Cisco Vicenty, a program manager in the Office of Compliance within FDA's Center for Devices and Radiological Health (CDRH). "The fundamental definition of quality is, how do I give my customer the best experience possible?"
Vicenty – who is also a leader for the Case for Quality – sat down for an interview with Medtech Insight May 1 at a Case for Quality forum at Cincinnati's Xavier University. He stressed that the agency can't define quality for all companies.
"But what we can do is focus on a manufacturer's capability, or maturity, because that's what we can say matters," he said.
"Maturity" in the context of manufacturing means that companies have adequately developed practices and processes to ensure that quality is pervasive throughout their organization.
"I can't see what your customer wants for quality unless I'm talking to every single patient or user out there available to us right now," Vicenty said. "But I could see what your organization's capabilities are to deliver on that, and then we could figure out how to improve that."
One big way FDA is getting a bird's-eye view of device-makers' capability and maturity is with its ongoing Voluntary Medical Device Manufacturing and Product Quality Pilot Program, under the umbrella of the Case for Quality.
Launched in January, the pilot aims to elevate device and manufacturing quality in an array of firms. The pilot uses an industry-tailored version of the CMMI model and method, developed jointly by FDA, industry and CMMI Institute.
Measuring Capability: CMMI's Raison D'être
Measuring capability is CMMI Institute's raison d'être.
The widely known Capability Maturity Model Integration (CMMI) framework was first released in 1991 as the Capability Maturity Model by Carnegie Mellon University's Software Engineering Institute in Pittsburgh. With the international adoption of the model, CMMI Institute was founded and lifted out of the university in 2012.
CMMI is a process improvement model, accompanied by training and an appraisal method, used in an array of industries, including health care, automotive, information technology, defense and aerospace. It aims to improve an organization’s capability to deliver business objectives and distinguish trusted business partners.
CMMI Institute provides training and certification to CMMI instructors and lead appraisers who then make up a vast partner distribution network that provides services to organizations looking to evaluate and improve their capabilities.
One of the more recognizable aspects of the CMMI framework is the Maturity Level, wherein appraisers determine where manufacturers fall on a five-tiered maturity scale, with "1" denoting beginner efforts by a firm, and "5" signifying that a company is self-correcting, effectively managing quality and is continuously improving.
But that's not how CMMI's framework is structured as it's been applied to the medical device industry through the ongoing FDA/MDIC maturity model pilot (which the institute has branded a bit differently, calling it the Medical Device Discovery Appraisal Program, or MDDAP).
CMMI appraiser George Zack, who says CMMI is an "industry-agnostic" model, explained that – through discussions between CMMI Institute, the device industry, FDA, MDIC and other stakeholders – it was decided that the five-tiered scale format would be eschewed in favor of one that is more custom-fit for the device industry.
"Many have heard of organizations achieving [CMMI] maturity levels 1-2-3-4-5, which is a representation of the model where there are specific practices that are outlined for each maturity level, and an expectation for how [a firm] performs within those from a capability perspective," Zack, who is also principal and cofounder of Two Harbors Consulting, said at another Case for Quality forum, this time in November 2017.
The five-tiered concept "is a great framework [aspect] for many organizations," but Zack said CMMI recognized that its structure for devices must approach manufacturers as though they already have a clear understanding of industry regulations and expectations, and are in a high state of compliance.
In the device industry, "there’s already a mindset of compliance, [so] we wanted instead to focus on continuous improvement. So, we took the maturity levels out of the conversation," he said. (Also see "Quality On The Brain: FDA Maturity Pilot Aims To Shift Industry's Compliance Mentality To A 'Quality Mindset'" - Medtech Insight, 29 Sep, 2017.)
In a separate March interview with Medtech Insight, Zack went further. "There was significant concern [amongst stakeholders] that a [five-tiered model] would create a compliance mindset – that firms would say, 'OK, what do I need to do to get to level 2 or level 3?' and then be done with the program."
Therefore, CMMI opted instead to develop, along with stakeholders, a structure specific to devices that covers these 11 practice areas:
- Monitor and Control
- Configuration Management
- Managing Performance and Measurement
- Requirements Development and Maintenance
- Process Quality Assurance
- Implementation Infrastructure
- Product Integration
- Technical Solution
Zack said those practice areas "are focused on a variety of activities that are related to the standard type of engineering view that would be appropriate for medical device manufacturers.” An appraiser could then look at a firm's various capabilities within each of those practice areas.
"With this type of continuous approach, there's more of a thought process by firms of, 'How can I potentially improve and not necessarily have to worry about achieving some particular level or some particular score?'" he added.
A CMMI appraisal "is not a new enforcement tool. This is not a new mechanism for [FDA to] say, 'You are doing this wrong. You are doing worse than everybody else,'" CDRH's Cisco Vicenty says.
During the same interview, Becky Fitzgerald – also a CMMI appraiser, as well as principal and cofounder of Two Harbors Consulting, along with Zack – told Medtech Insight that within the 11 practice areas are "a slew" of individual practices that are assessed by appraisers.
In the FDA/MDIC pilot, all 11 practice areas are reviewed during an initial appraisal; firms don't have the option to pick and choose which areas they want to have assessed.
Fitzgerald noted that the practice areas could eventually be expanded beyond the initial 11. "There's plenty of room going forward for different practice areas to come into," she said, noting that the 11 areas "are a good starter set, a good foundational set, so that if an organization is capable in these areas, and can build capability in these areas, their business performance and product quality will be improved, and it will help to ensure patient safety."
Conversely, she said, after the pilot ends, it might be decided that the 11 areas should be winnowed down, although that is unlikely.
Fitzgerald noted that the 11 identified practice areas assessed under the FDA/MDIC pilot "are not unique to this pilot; rather, they're foundational core practice areas."
But she stressed: "Is it a medical device view [of the model]? Absolutely."
Firms are assessed using what CMMI Institute calls an Evaluation Appraisal method, which has been tailored for the program to meet stakeholder needs for consistency and rigor.
There are several CMMI appraisal methods to select from, but an Evaluation Appraisal "supports a higher degree of tailoring and fits the intent of the [device] program the best, so it was chosen by the FDA, industry and CMMI to use," Fitzgerald said.
A December Medtech Insight story announcing the FDA/MDIC pilot indicated that the appraisal process would use the SCAMPI-C appraisal method – the Standard CMMI Appraisal Method for Process Improvement – however, the pilot uses Version 2.0 of CMMI, in which "SCAMPI-C" is now "Evaluation Appraisal."
Sign On The Dotted Line...
Participants in the Voluntary Medical Device Manufacturing and Product Quality Pilot Program must agree to:
- Appraisals conducted by the CMMI Institute;
- Collecting and submitting developed metrics data, and providing it to CMMI for analysis;
- Being available for real-time consultations with FDA and CMMI;
- Participate in established monitoring activities with CMMI; and
- Allow for reporting to FDA by CMMI of analyzed performance data.
FDA Gets Metrics, Heat Maps, Anonymized Data. But Can Industry Trust The Agency?
On paper, the joint FDA/MDIC Voluntary Medical Device Manufacturing and Product Quality Pilot Program is small, made up of only nine manufacturers that have agreed to allow CMMI appraisers to measure the maturity of their quality systems and manufacturing processes. It began on Jan. 2 and will run through Dec. 28.
But it's become more than that.
It's grown to include 14 manufacturers and counting; 11 appraisals have taken place so far. (FDA's goal is to conduct, at a minimum, 30 appraisals this year and to enroll as many firms as possible in the pilot – more on this later.) Of those 14 firms, 19 facilities are to be appraised, which means some companies have volunteered to have more than one of their facilities to be assessed by CMMI.
But beyond the sheer numbers, the pilot is now a major piece of a new safety and quality action plan at FDA.
Announced on April 17, the agency's Medical Device Safety Action Plan is anchored by a five-pronged strategy: establishing a robust device patient-safety network; exploring regulatory options to streamline and modernize timely implementation of post-market mitigations; spurring innovation toward safer devices; advancing device cybersecurity; and integrating pre- and post-market offices and activities within FDA's device center to advance the use of a "total product lifecycle," or TPLC, approach to device safety.
The goal of the action plan is to strengthen the agency's ability to ensure device safety and to encourage device companies to continuously improve their products. (Also see "New Safety Framework Mixes Current Efforts, New Investments At US FDA " - Medtech Insight, 17 Apr, 2018.) The safety plan, including the maturity model program, is one important element of FDA's FY 2019 budget request to Congress.
FDA says in its action plan that the maturity model pilot – and eventual program, which it wants to stand up by next year – will help the agency meet its goal of driving the innovation of safe products.
Through the pilot, FDA will "evaluate participants’ organizational excellence, identify any gaps and recognize when a participant performs above a compliance baseline," the agency says in its work plan.
Results of a manufacturer's CMMI appraisal are shared with the firm itself, and a summary report is sent to FDA. The agency will use the information to help shape its regulatory, compliance and enforcement decisions.
"So we're very clear: There's nothing punitive about participating [in FDA's pilot]. There's no risk. It's a voluntary program. And you're getting the regulatory benefits by participating," consultant and CMMI appraiser George Zack says.
At the end of an appraisal, the agency, along with the participating firm, is supplied with a "colored heat map that says, here's where you performed well; here's where there were struggles, or you were partially there – or moving in that direction," CDRH's Vicenty said.
Vicenty – speaking this time on the topic at a February "Half Day with FDA" conference for the device and drug industries hosted by Xavier University – said being able to see where a manufacturer falls on a CMMI assessment heat map provides FDA with plenty of information to give it confidence that the device-maker is on the right track when it comes to attaining gold-star quality in its products and manufacturing processes.
"We don't need the granularity that's provided [by CMMI to an assessed manufacturer], but we do want to get an idea of the standing, the status, individually for the organization," he said, noting that FDA is also given an anonymized aggregated dataset by CMMI so the agency can gain a better understanding of the overall quality status of the collective set of assessed firms.
Meanwhile, FDA's recent work plan goes on to say that the maturity pilot "is intended to establish organizational performance metrics and device quality metrics that can be tracked in a 'dashboard'-like format to facilitate continuous monitoring and, accordingly, improved visibility to potential manufacturing, product quality or patient safety issues."
Quality metrics are measures used to assess the overall quality of medical device manufacturing; participants in the maturity model pilot must gauge and share them with FDA on a quarterly basis. This dovetails with ongoing work that the agency is doing, also with MDIC, on using metrics to not only aid in deciding whether to, say, inspect a device-maker, but to also raise firms' overall approach to quality above baseline – essentially the same overarching goals as for the Voluntary Medical Device Manufacturing and Product Quality Pilot Program. (Also see "At The Intersection Of Quality And Metrics: What's Ahead In FDA’s Effort To More Objectively Measure Quality" - Medtech Insight, 13 Jul, 2016.)
Being able to see metrics and outputs from CMMI assessments "gives us more visibility [into a firm's organizational capabilities] at a more frequent pace than what the agency receives now," Vicenty said. "It lets us know where the organization is targeting to drive improvements and gives us visibility into their progression."
What FDA might do with the data it collects from its pilot work is a likely concern for some manufacturers that might be on the fence about whether to join the pilot. Some firms might worry that the agency will use appraisal results to carry out enforcement actions – but Vicenty said companies can be rest-assured knowing that this maturity model process isn't about compliance.
Rather, "the appraisal and the maturity model is about driving improvement. It's about looking at how well everything is executing, and can we make it a system that is better … to deliver more value for that product, for that patient," he said. "It is not a new enforcement tool. This is not a new mechanism for [FDA to] say, 'You are doing this wrong. You are doing worse than everybody else.'"
Instead, "it's a tool for finding out where you can drive the most impactful improvements," Vicenty added. "Again, it is not a new enforcement tool. It's not a new compliance bar. And it's not a new auditor checklist."
Vicenty said the data FDA sees will provide it with an opportunity to spot "where the gaps are" in industry when it comes to quality. The data "gives us a lot more granularity into where [those gaps] might be and what the potential pathway is for driving improvement," he noted.
Consultant and CMMI appraiser Zack is also seeking to allay manufacturer fears. He told Medtech Insight that device-makers shouldn't be nervous about FDA's use of data from the pilot because the agency has assured those involved that this pilot, and the future program, has nothing to do with compliance.
"Every time the FDA has spoken of this program and the results that they receive from it, they have been adamant that they're not assigning any sort of action based upon the results that [CMMI] has reported," Zack said. "In other words, if your aggregate scores from your CMMI appraisal heat map reflect a value that is lower than some other organization's, they are not going to go to that organization and treat it any differently. FDA has been very clear that all they want in this pilot is participation."
"If you are participating [in the pilot], there isn’t a penalty for doing so. There may be rewards. But you’re also not off the hook for the things you’re otherwise on the hook for," CDRH Director Jeff Shuren says.
Zack also pointed out that only manufacturers with clean compliance records are accepted to the pilot anyway, which means they'd be the last device-makers that FDA would be looking at for potential quality systems issues or other troubles.
Companies in the pilot must have "No Action Indicated" (NAI) or "Voluntary Action Indicated" (VAI) results from FDA inspections conducted over the past five years. Favorable results from Medical Device Single Audit Program (MDSAP) audits are also accepted.
Pilot participants "have already demonstrated a clean compliance profile; otherwise, the appraisal would not happen," Zack said. "So, they're already the good students, for lack of a better description. [FDA] expects those organizations to be engaged in the appraisal and then to remain engaged by way of follow-up checkpoints."
He continued: "But [FDA isn't] ascribing any sort of expectation of results or movement on those particular results. They just want to see engagement. [The agency] isn't interested at this time in a firm's specific practice considerations, weaknesses, strengths – instead, that's for … the firm to address as a part of its process-improvement journey."
Zack understands, however, that there are firms that might struggle to trust the agency with potentially sensitive data.
"What I want those manufacturers to know is this: While we [at CMMI] will give you, the medical device organization, your specific results and the specific data that support those results, the FDA does not see that. The FDA only sees a small rollup of the practice area scores," he said.
"So we're very clear: There's nothing punitive about participating," Zack said. "There's no risk. It's a voluntary program. And you're getting the regulatory benefits by participating."
As an incentive to join the pilot, the agency is putting off regularly scheduled facility audits and waiving pre-approval inspections, unless an audit is for-cause. FDA is also giving firms streamlined options for 30-day notices, site transfer changes and pre-market submissions, among other carrots. (Also see "Gifts For Industry: From Waived Inspections To Pre-Market Leeway, US FDA Woos Firms For Maturity Pilot" - Medtech Insight, 25 May, 2017.)
Nevertheless, at November's Case for Quality forum, FDA device center Director Jeff Shuren cautioned manufacturers: "If you are participating in this aspect of Case for Quality, there isn’t a penalty for doing so. There may be rewards. But you’re also not off the hook for the things you’re otherwise on the hook for, for FDA, and that’s where there are potentially penalties involved."
What Happens If A Good Firm Breaks Bad?
While only device-makers with clean compliance records are accepted into the pilot, that doesn't mean they'll stay that way. So, what happens if a pilot participant goes off the rails?
Consultant and CMMI appraiser Fitzgerald assured Medtech Insight in an April 27 interview that if a manufacturer has a hiccup, that doesn't mean it will be booted out of the pilot program. Rather, "it means FDA has a reason to hold a conversation with that organization and find out what's going on," she said.
"How capable [a company] is to detect that it has a problem and to address it rapidly, and to effectively respond – that's the capability that the FDA is very interested in," consultant and CMMI appraiser Becky Fitzgerald says.
"It's not like FDA doesn't expect [problems] to happen or doesn't think that they might happen," Fitzgerald said. Instead, "the expectation is that there would be far fewer issues, or more setting up of capabilities in an organization so it can respond effectively if it has an event.
"That's the issue," she continued. "The issue isn't that an organization is going to prevent all events from ever happening. That's not real-world. Some things occur that can't be predicted, and when those things occur, how capable the organization is to detect that it has a problem and to address it rapidly, and to effectively respond – that's the capability that the FDA is very interested in."
CDRH's Vicenty agreed. He pointed out that "95% of medical device manufacturers" on the market "want to do the right thing."
Nevertheless, "you are always going to get that one bad decision-maker; you’re going to get that bad ethical judgment driving leadership that’s going to force some decisions in the inappropriate fashion," Vicenty said during the May 1 interview with Medtech Insight.
If egregious problems arise in a firm enrolled in the pilot, it might be asked to leave the program. But Vicenty said a situation like that would be rare.
"And I can’t imagine a manufacturer who has started to see some of the improvements [from the CMMI appraisals] suddenly saying, 'Well, we’re going to consciously choose to not be here. Let’s go back to being enforced on,'" he said.
In an interview, Kimberly Kaplan, program operations manager for the CMMI Institute, said that despite FDA's initial request for only nine manufacturer pilot participants, 14 have joined – and the group is looking to take on "as many manufacturers as the system can support."
Of course, "we might not be able to handle a thousand if they were to come in all at once, but let's just say that we aren't going to be turning anyone away," she said.
FDA imposed a cutoff of nine participants in its December announcement of the pilot because the agency was bureaucratically bound by the Paperwork Reduction Act. The law shields companies and people from being asked for an onerous amount of paperwork from the US government, but in this instance, it caused the pilot to be capped at nine firms until the amount of additional paperwork was determined and explained, despite particularly high enthusiasm from industry about the maturity model approach.
Nevertheless, CMMI Institute is considering data on all 14 participating firms' assessments (and potentially more) because the institute believes that FDA will soon find a way to resolve or work around Paperwork Reduction Act requirements, likely through a waiver of some sort. (Indeed, CDRH's Vicenty told Medtech Insight that he doesn't expect the agency's device center to encounter any significant problems in working around the act.)
"When we first heard about the restrictions imposed by that act, we asked FDA if it was something we should be worried about, and whether we needed to cap the pilot," Zack said. "And the agency basically said, 'We will take care of this, but if you can, capture data for us to help us understand the level of effort associated with the pilot.' So, we're managing that.
"Sure, if FDA came to us and said, 'Hey, we can only have nine in the pilot,' then there would be a conversation with the agency to determine how to best proceed," the consultant and CMMI appraiser added. "But right now, from the operations perspective and the execution of the appraisals, we have the green light to go forward.
"And we're going to get to that goal of 30 appraisals completed this year – or maybe even more."
Despite the current nine participant limit (for now), all pilot enrollees can still reap the benefits offered by FDA for taking part in the program.
3 Signs Of A Successful Pilot
Zack says a successful FDA/MDIC pilot boils down to "three layers of success."
First, it must be proven that the program can be executed well. "Can we perform these appraisals within [a reasonable amount of time]? Can we assemble appraisers, and get them trained and brought up to speed, and execute these appraisals in a way that is sustainable and scalable? That the organizations are gaining value out of it and that the appraisers are also happy with it? That’s everything from the business operations piece to the execution of it on the ground," he said.
Although appraisals have been performed "tens of thousands of times, this one's a little bit different," Zack said. "We have to demonstrate that all the cogs in this wheel are indeed working. So, the first piece of success is demonstrating that all of those pieces and parts work throughout the remainder of this year and through the pilot activity in 2018."
Success would mean "we have demonstrated that we've learned from this program, and maybe we can even take this program into other regulated areas," such as drugs and food, Zack says.
A second marker of pilot success would be for CMMI to provide enlightening trending results to FDA that's helpful as the agency looks to nudge device-makers to improve manufacturing quality and product safety.
Success would mean "we have demonstrated that we've learned from this program, and maybe we can even take this program into other regulated areas," Zack said. "Maybe we could take it into pharma. Maybe we could take it into food."
A third sign of success for the pilot – which Zack admits is a long-term effort – would be linking the maturity model to a product quality dashboard, where a firm's quality and performance metrics can be corralled and consistently monitored.
"Now, that’s a big vision. I don’t want anybody to walk away saying, 'We’ve got this completely figured out,'" he said. "There's an analytics program that’s being defined. And then there's the maturity model program … that’s going through this pilot activity. We have to have that long-term read on the future as to where things are going to go, but these are very much two programs that are in-work."
Are Costs Prohibitive For Small Firms?
Between the program fee and appraisal fee, it can cost anywhere from $30,000 to $88,000 for device-makers to play in the FDA/MDIC maturity model pilot program. The price is tiered for small and large manufacturers, and is otherwise mostly dependent on the scope of the assessment, which is different for different companies, Fitzgerald said.
"When you work with an organization, you determine how big it is – that is, how many different ways they're doing their work – and that's why it's not simple to quote a cost," the consultant and CMMI appraiser said.
"You can't just go by revenue size. And it doesn't have anything to do with the number of products a firm makes," Fitzgerald said. "A firm could have a thousand products and make them all exactly the same way, following the same set of practices. Or, a firm could have five different products and each one is not only uniquely approached, but maybe the subassemblies are even unique."
Nevertheless, small firms – those registered with a small business designation (SBD) number – get a slight break on fees. They are charged a $3,000 program fee, and their appraisal fee can run from $27,000 to $64,000.
By comparison, large firms pay a $5,000 program fee and an appraisal fee of $34,000 to $83,000.
Despite the slight difference in price, the sticker shock can still be real for smaller firms that don't pull in the type of cash that their larger, more profitable peers do.
"One of the things that we heard early on was the potential cost for small manufacturers or even mid-sized manufacturers may be prohibitive," CDRH's Vicenty said in April at FDAnews' 15th Annual Medical Device Quality Congress in Bethesda, Md.
"We’ve been trying to work with CMMI, and they’ve been adapting their model – their approaches – to try to make that cost a more manageable aspect moving forward," Vicenty said. "This is not about creating a new finance sector for people in the consulting space. It really is about driving improvement in the best way possible."
And CMMI's Kaplan says: "While cost has historically been a concern from the perspective of small organizations during the development of this program, about a quarter of enrolled facilities are actually small organizations."
Still, some say the pilot is a rich firm's game.
Al Crouse – whose Minneapolis, Minn.-based, venture-backed firm CVRx had a facility audited under the pilot – said a "significant challenge" to the maturity model program getting off the ground is that it's out of reach financially for many small firms.
"I’ve been a pretty vocal advocate for trying to figure out ways to bring that cost down, because right now it’s sort of a one-size-fits-all plan, and it’s expensive," Crouse said at the Quality Congress. He is senior quality director for CVRx, which makes the Barostim Neo device for the treatment of heart failure and high blood pressure.
And, Crouse said, small device-makers aren't necessarily in a position to cash in on FDA's pilot incentives around 30-day notices and pre-market submissions in the same way that large ones are.
For example, device giant Edwards Lifesciences, which had a facility assessed as part of the pilot, has said it expects to save $120,000 in manufacturing submissions costs this year alone. And Medtronic – another pilot participant – has claimed that just one of its facilities could save $1.7m by not having to submit 30-day notices. (Also see "FDA Looks For Diverse CMMI 'Maturity' Pilot Enrollees; Device-Makers Expect Big Savings" - Medtech Insight, 16 Oct, 2017.)
Crouse suggested that, because of their big FDA savings, large firms should invest back into the program.
"For the amount of money that the big firms save, they should be willing to kick in a little more for some of us small guys to help pay for this program, because small firms, we don’t have a lot of 30-day notices," Crouse said. "There’s just not a lot of efficiencies for [small firms] to do this, and there’s not a lot of gaining."
[Editor's note: CVRx – along with fellow device-makers Baxter Healthcare, Boston Scientific, Edwards Lifesciences, and others – will share its CMMI appraisal experiences in Part 3 of "Medtech's Next Top Maturity Model."]
Enrollment-To-Appraisal Target Of 90 Days
It should take no longer than 90 days from when a device-maker first enrolls in the pilot to when it is appraised by CMMI.
However, that timeframe "tends to vary depending on the organization’s needs," CMMI's Kaplan says. "Some organizations want to get this done more quickly. Some have other things that are happening in their organization that push out that timeframe a little bit. But what we’ve seen so far is that the shortest timeframe [for an assessment thus far] is 52 days – which is almost half of the target" of 90 days.
The longest time from pilot enrollment to CMMI appraisal was 144 days for one firm.
Ann Ferriter, director of the division of analysis and program operations within FDA's device center, says there's a very good reason for wanting to keep the enrollment-to-appraisal timeframe to a three-month period.
"One of the reasons why we don't want to go, say, a year between enrollment and the appraisal is that FDA is holding off on doing an inspection," Ferriter explained.
Again, as a sweetener to manufacturers to join the pilot, FDA is putting off regularly scheduled audits and waiving pre-approval inspections for enrolled facilities.
"We don't want someone to enroll and then never have an appraisal," said Ferriter, who works on Case for Quality initiatives, including for quality metrics, as well as FDA's work around benefit-risk concepts. (Also see "'A Sea Change': Device Center Compliance Chief Touts US FDA's Benefit-Risk Concepts – But Will Manufacturers Buy In?" - Medtech Insight, 7 Aug, 2017.)
[Editor's note: In the next installment of "Medtech's Next Top Maturity Model," CMMI appraisers George Zack and Becky Fitzgerald, along with CMMI Institute's Kimberly Kaplan, explain what happens when a firm is appraised.]
From the editors of The Gray Sheet