Assessing how widespread the continuous cycle of improvement of medical devices is in the EU
This article was originally published in SRA
Dutch regulators Arjan van Drongelen, Boris Roszek, Jos Kraus and Robert Geertsma find that the CCI concept is still in its infancy for many device manufacturers.
In order to guarantee the safe application of medical devices, the associated risks need to be managed throughout the entire lifecycle of the product1. Design (including labels and instructions for use), risk management and post-market surveillance are cornerstones of the EU Medical Devices Directive (93/42/EEC as amended)2. These elements interact in a process that can be referred to as the continuous cycle of improvement (CCI) of medical devices (see Figure 1).
Over the past several years, the Dutch National Institute for Public Health and the Environment (RIVM) has performed several investigations into the quality of technical documentation of medical devices3-7. These file investigations showed major shortcomings for different parts of the CCI, such as risk analyses, label/IFU and PMS procedures. Also, the necessary coherence between risk analysis and label/IFU, which can be considered part of the CCI, was often lacking8.
To gain further insight into the implementation of the CCI, the RIVM performed an investigation9. This article summarises the main results of this investigation, focusing on medical devices regulated by the MDD. The same principles, however, apply for active implantable medical devices and in vitro diagnostic medical devices10,11.
Risk management is a continuous iterative process throughout the entire lifecycle of a medical device, described as a set of repeatable steps. Once new information becomes available (eg in the post-production phase), the impact on the associated risk, as well as on the overall risk, should be evaluated. This could result in additional risk control measures, including changes in the design, label or IFU.
In selecting the most appropriate solutions for the design and construction of the devices, the manufacturer must apply the following principles in the following order:
eliminate or reduce risks as far as possible (inherently safe design and construction);
take adequate protection measures including alarms for risks that cannot be eliminated; and
inform users (label/IFU) of the residual risks due to any shortcomings of the protection measures.
The MDD requires the device manufacturer to have "a systematic procedure to review experience gained from devices in the post-production phase" and to take appropriate measures (PMS). Moreover, the manufacturer shall also notify the competent authorities of incidents and evaluate the incidents (vigilance)12,13. PMS activities imply the systematic collection, analysis and interpretation of experiences with medical devices.
Interaction between these three elements is essential. The results of PMS activities have to be fed back into the risk management process, which could result in modifications of the design, label or IFU. Furthermore, information from the design process and the risk management activities should give input into the type of PMS activities to be carried out.
The investigation: method
The RIVM began its investigation by sending to a selection of manufacturers a questionnaire on CCI-related issues concerning a particular medical device. Manufacturers of infusion pumps, endoscope washer disinfectors and medical devices for clinical investigations, which had recently been included in RIVM investigations14, were selected. In total, 32 manufacturers were included. One manufacturer received two questionnaires as two of the company's devices were included.
In summary, the RIVM questionnaire asked:
Has the medical device been modified (ie design, label and instructions for use)?
Has the risk analysis of the medical device been updated?
Which PMS findings have led to which modification(s) of the medical device?/Which PMS findings have not led to modification(s) of the medical device?
Which vigilance findings have led to which modification(s) of the medical device?/Which vigilance findings have not led to modification(s) of the medical device?
The data in the questionnaire were analysed for a number of aspects of the CCI, eg the occurrence of both design change and risk analysis update. The risk analysis was chosen as the indicator for risk management activities, as the risk analysis is explicitly required as available technical documentation for every medical device in the MDD.
Moreover, manufacturers were requested to submit PMS and vigilance procedures, which were checked for the presence of six elements.
Results and discussion
All manufacturers completed the questionnaires. One incomplete questionnaire was excluded, resulting in 32 questionnaires to be analysed. In addition to the questionnaires, 27 PMS procedures and 29 vigilance procedures were submitted. Most of the manufacturers that did not submit procedures had no other CE-marked medical device on the EU market, ie they had only devices under clinical investigation.
Although we analysed data from a relatively small sample of devices chosen from a limited number of product categories, the results are expected to be applicable to a wider range of medical devices and their manufacturers.
Device modification/risk analysis update
For 50% of the medical devices, both medical device modification and risk analysis occurred, suggesting the implementation of the principles of the CCI (see Figure 2, top left area). However, in 19% of the devices, no risk analysis update occurred, while there was a device modification, suggesting that the principles of CCI were not implemented. This was given a negative score (see Figure 2, bottom left area), as any change to a device could introduce new risks or modify the estimation of previously identified risks, and should thus initiate an update of the risk analysis.
For the other cases (see top and bottom right areas in Figure 2), no conclusions could be drawn regarding the implementation of the principles of CCI. A risk analysis update does not necessarily have to lead to a device modification, so no definite score can be given to the cases in the upper right quadrant (3/32). Also, where no device modification or risk analysis update occurred, there is no information on the implementation of CCI by the manufacturer (7/32).
Finding leading to device modification
Manufacturers were asked whether an experience-based finding had led to device modification (Figure 3). Approximately one-third of the manufacturers (n=11, resp n=13) did not indicate whether they had PMS and vigilance experiences with their device.
For the rest of the manufacturers, findings based on device experiences gathered through PMS activities often led to modification of medical devices (55%), whereas vigilance activities resulted in device modifications in 25% of the cases. This shows that in at least some of the cases, interaction such as required by the principles of the CCI took place. As no background information was available regarding the nature of the findings, no conclusions on CCI implementation could be drawn on cases where findings did not lead to device modifications.
PMS and vigilance procedures
The procedures were checked for the presence of activities considered to be essential for the proper execution of PMS and vigilance activities. Activities like risk management and corrective and preventive action have to be interlinked with the PMS and vigilance activities. Therefore, only including risk management and CAPA procedures in the reference list of PMS and vigilance procedures was considered insufficient.
A crucial aspect of PMS is gathering experiences by means of an active approach, including active customer feedback, post-market clinical follow-up (where applicable) and literature reviews, in addition to handling of passive customer feedback, ie complaints. In their PMS procedures, manufacturers described passive customer feedback more often than active feedback (see Figure 4).
Risk management activities were mentioned in only 41% of the procedures. It is conceivable that manufacturers elaborated on these activities in other procedures, for instance in a CAPA procedure. Such procedures were not requested, but in 63% of the PMS procedures a CAPA procedure was referred to. Thus, it cannot be ruled out that more than 41% of the manufacturers perform risk management activities due to findings of PMS activities. However, as only 78% referenced risk management and/or CAPA activities (data not shown), at least 20% of the manufacturers do not appear to use a systematic approach to improve product and/or safety following PMS findings.
In conclusion, a considerable number of PMS procedures lacked the description of gathering device experiences and using PMS findings as input for a review of the risk management data and for CAPA activities.
All vigilance procedures described activities for incident monitoring and evaluation and the obligation for manufacturers to notify incidents to competent authorities (see Figure 5). However, the use of vigilance findings as input for risk management and CAPA was often not described. Risk management activities were mentioned in 28% of the procedures. In 41% of the vigilance procedures, a CAPA procedure was referred to. It is conceivable that manufacturers elaborate on these activities in other procedures, for instance in a CAPA procedure.
Thus, it cannot be ruled out that more than 28% of the manufacturers performed risk management activities due to findings of vigilance activities. However, as the combined number of the manufacturers that referenced a risk management and/or CAPA procedure was 52% (data not shown), nearly half of the manufacturers do not appear to use a systematic approach to improve product and/or safety following vigilance findings.
It could be argued that risk management activities and CAPA are not within the scope of a vigilance procedure. Strictly speaking, a vigilance procedure could be solely focused on gathering information on incidents and reporting incidents to competent authorities. However, vigilance is part of the requirement in the MDD to have a systematic procedure to review device experience and apply corrective action if necessary. Therefore, we consider risk management activities and CAPA to be essential items of a vigilance procedure.
In conclusion, vigilance procedures fully addressed incident reporting and the obligation to notify competent authorities. However, a considerable number of procedures did not describe using vigilance findings as input for a review of the current risk analysis and for CAPA.
Our investigation indicates that approximately half of manufacturers implemented principles of the CCI for medical devices, but for a substantial part of manufacturers this concept is still in its infancy.
Major issues are the following:
an update of the risk analysis did not always occur while there was a device modification, although this would be expected to happen when manufacturers follow proper procedures;
risk management activities and CAPA are insufficiently addressed in PMS and vigilance procedures; and
the quality of PMS and vigilance procedures leaves room for improvement.
Further implementation of the CCI is thus necessary to optimise the safety of patients, users and others.
1. EN ISO- 14971:2009 Medical devices – Application of risk management to medical devices, www.cen.eu/CEN/Sectors/TechnicalCommitteesWorkshops/CENTechnicalCommittees/Pages/Standards.aspx?param=581003&title=CEN/CLC/TC+3
2. Directive 93/42/EEC as amended, 11 October 2007, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CONSLEG:1993L0042:20071011:en:PDF
3. Roszek B, van Drongelen AW, Geertsma RE & van Tienhoven EAE, Assessment of technical documentation of Annex II medical devices, RIVM Report No 265011003, 2005, www.rivm.nl/bibliotheek/rapporten/265011003.html
4. Hollestelle ML, de Bruijn ACP & Hilbers-Modderman ESM, Infuuspompen in de thuissituatie – Zijn risicoanalyses, gebruiksaanwijzingen, opleidingen en post-marketing surveillance hierop afgestemd?, RIVM Letter Report No 360050015, 2006, www.rivm.nl/bibliotheek/rapporten/360050015.html
5. Roszek B, de Bruijn ACP, van Drongelen AW & Geertsma RE, Assessment of technical documentation of medical devices for clinical investigation, RIVM Report No 360050001, 2006, www.rivm.nl/bibliotheek/rapporten/360050001.html
6. Hollestelle ML, Hilbers ESM & van Drongelen AW, Risks associated with the lay use of 'over-the-counter' medical devices, Study on infrared thermometers and wound care products, RIVM Letter Report No 360050002, 2007, www.rivm.nl/bibliotheek/rapporten/360050002.html
7. Roszek B, de Bruijn ACP, Pot JWGA & van Drongelen AW, Assessment of technical documentation of Class III medical devices, RIVM Report No 360050021, 2010, www.rivm.nl/bibliotheek/rapporten/360050021.html
8. de Bruijn ACP, van Drongelen AW, Hilbers ESM et al, Coherence between risk analysis and instructions for use in the EU, Regulatory Affairs Medtech, 11 March 2011
9. Roszek B, van Drongelen AW & Geertsma RE, Continuous cycle of improvement of medical devices, A questionnaire on experiences and procedures, RIVM Report No 360050014, 2009, www.rivm.nl/bibliotheek/rapporten/360050014.html
10. Directive 90/385/EEC, as amended, 11 October 2007, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CONSLEG:1990L0385:20071011:en:PDF
11. Directive 98/79/EC, as amended, http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CONSLEG:1998L0079:20090807:en:PDF
12. See Reference 2
13. MEDDEV 2.12-1 rev 6, Guidelines on a Medical Devices Vigilance System, December 2009, http://ec.europa.eu/consumers/sectors/medical-devices/files/meddev/2_12_1-rev_6-12-2009_en.pdf
14. de Bruijn ACP & van Drongelen AW, Kwaliteit van de reiniging en desinfectie van flexibele endoscopen – Reprise, RIVM Report No 360050013, 2008, www.rivm.nl/bibliotheek/rapporten/360050013.html; see also references 4 & 5
Arjan van Drongelen and Boris Roszek are scientific officers, and Robert Geertsma is a senior scientist, at the Dutch National Institute for Public Health and the Environment (RIVM). Jos Kraus was a senior inspector of medical technology at the Dutch Health Care Inspectorate (IGZ) at the time of the investigation; he presently holds a position at the Academic Medical Centre in Amsterdam. Email: firstname.lastname@example.org.