Arena Virtual Event: MEDIcept and Arena Validate
Full transcript below:
Emily Foree
Hello and thank you for attending our Arena Virtual Event with MEDIcept today. So just an overview of our agenda today, we’re going to be presenting about Arena Validate and validation in general with MEDIcept. We’re going to go over some validation requirements, a traditional non-product software validation approach, an overview of Arena Validate, an introduction of our new Arena Validate PQ templates, how to leverage Arena Validate to support a shortened CSA approach, how MEDIcept can help you with that, and what’s coming up next.
And some brief introductions. My name is Emily Foree, and I’m the Validation Program Manager at Arena. I’m joined today by Jason Gromek, who’s the Executive Vice President at MEDIcept and Marion Cappadona, who’s a Senior Consultant of Software and Systems at MEDIcept. And with that I’ll go ahead and hand it over to Jason.
Jason Gromek
Thank you, Emily and thank you, Arena for having us as guests today. As Emily said, my name is Jason Gromek, and I’m the Executive Vice President of Sales and Operations for MEDIcept. MEDIcept’s an international consulting company specializing in medical device, IVD, combination product, and biotechnologies. We offer quality and regulatory consulting services as well as a turnkey full-service clinical research organization or CRO. Our partners have over 150 years of combined experience and we offer expertise on various levels of services.
In 2015, MEDIcept and Arena’s relationship began. We had a common client and Arena was the eQMS system that was put in place. Over the next few years, we became an unofficial partner. We worked together presenting Arena benefits to our clients, and Arena presented MEDIcept’s solutions to their customers. In 2019, we became a formal partner offering validation services, and the relationship grew through the years.
In 2020, we developed a formal PQ validation package, and then in 2021 we expanded our services and continued to help Arena clients and customers. In 2022, we formalized the validation package with the FDA CSA approach, and here we are in 2023—we’re able to offer that in conjunction with Arena.
So thank you for listening today and we’re excited to help you out. And with that, I’m going to introduce Marion Cappadona, who’s one of our senior consultants, and she’ll go through the validation services that MEDIcept can provide.
Marion Cappadona
Thank you, Jason. Hi, my name is Marion Cappadona and I’m a Senior Software and Systems Consultant with MEDIcept. I’ve been in the life sciences industry for more than 30 years, working as a biomedical engineer, a quality engineer, and a consultant. I’ve specialized in design control, risk management, software development, and non-product software validations. During that time, I’ve worked with various non-product software applications from Arena, as well as other eQMS or PLM programs to ERP systems, including NetSuite and SAP.
I will be speaking to you today about traditional validation approaches towards the end of our presentation. I will discuss Arena’s new PQ phase templates and the CSA approach.
With that, let’s get started. To begin, what are the requirements for non-product software validation? In the U.S., FDA requires non-product software validation mainly in two places: 21 CFR 820.70(i), as well as within 21 CFR 11. Both require validation of software used to automate your quality system or production processes. 21 CFR 820.70(i) indicates that you need to validate your intended uses for these systems according to an established protocol. In addition, the validation activities must include the results documented as part of your validation package.
Finally, 21 CFR 820.70(i) requires that software changes be validated before approval and issuance. Part 11 includes requirements to validate software for intended use, including meeting the part 11 requirements for electronic records such as audit trails and electronic signatures. I will not be discussing details of the part 11 requirements specifically today, but we will discuss how you may use Arena Validate to show evidence of compliance later on in this presentation.
The most common standard use for regulatory compliance internationally is ISO 1345:2016. This standard expands on current U.S. regulations and really is the direction that the FDA is heading in, which we will discuss later on. As the more recent standard, ISO 1345 prescribes a more current thinking in using a risk-based approach to just about every decision or action one takes within your quality system, and non-product software is no different.
ISO 1345 first implicitly requires the establishment of a procedure for the validation of non-product software used in the quality system as part of production or service, and as part of monitoring and measurement processes such as inspection and calibration programs. ISO 1345 prescribes the need to validate these systems prior to initial use and as appropriate after changes to the software or its application. “As appropriate,” you may remember, means traditionally that it’s always appropriate unless you can document a justification why it is not needed. So how do you do that? Well, ISO 1345 also prescribes that the need to use a risk-based approach in determining both your initial software validation activities and your re-validation requirements. This feeds right into the CSA approach that we will discuss a bit more later. In addition to these requirements, you may find it helpful to look at some guidance documents.
First up is the General Principles of Software Validation Guidance that was produced by FDA a way back in 2002. This guidance has since been updated with FDA’s new CSA approach, which currently is in draft. However, this risk-based approach is not something that you need to wait until that CSA guidance is in final format, as FDA has said as much about it.
Another guidance document that is very helpful is the GAMP 5: A Risk-based Approach to Compliant GxP Computerized Systems. This is an ISPE standard that prescribes typically what is included in a non-product software validation program. And the second edition that was recently released by them has been updated to include a more CSA-type approach.
First, I’d like to talk to you today about the traditional validation approach that has been taken for many years for non-product software validation. And later we’ll get into the Arena Validate package as well as how you can use the new CSA approach to ease the pain of the documentation with the traditional approach. But traditionally, your non-product software validation program should include a standard operating procedure that governs how you will validate non-product software and when it’s required and overall for your company.
The elements of the non-product software validation typically include a Master Validation Plan, test protocols and reports for IQ, OQ, and PQ, a Traceability Matrix, a Master Validation Report, and maintenance or re-validation planning. There’s more than one way to skin a cat, so your program may not call your elements all the same things, but this is typically what one would see in a non-product software validation package.
So what do you include in the standard operating procedure? An SOP is defined as required in ISO 1345. The SOP should define how you will meet the requirements of the applicable standards or regulations you are applying overall. You should define the scope of the validation requirements and it is helpful to say what is out of scope generally. You should define the process for how you will track software used and determine which is in scope or out of scope. The SOP should also define your typical test requirements and how you will use a risk-based methodology to determine validation requirements for each application which is in scope. Finally, you should set a framework for ongoing tracking of your software and how you will maintain that software that is in scope in a validated state.
Next, the first element that would be prescribed by your SOP is the Master Validation Plan. It typically will define your intended uses for the specific software application, which the Master Validation Plan is being written for. These intended uses will be translated into a more specific software user requirements. For example, software shall prevent unauthorized access to obsolete documents. Usually you’ll have a group of these requirements which fall under a particular intended use. There is certainly more than one way to skin a cat here, thus there may be variations on how these requirements are specified in your Master Validation Plan.
The Master Validation Plan will also typically include or reference a risk assessment, which will be used to drive the software validation intensity in specific areas.
For example, not all requirements are equal and some deserve more attention in testing than others. Make use of your risk-based approach to stress the areas that are most important and most impactful on the safety of patients and compliance in your software application. Based on the risk assessment, you will then identify the software validation requirements in your test strategy. Finally, the MVP should identify the plan for validation and re-validation activities. This may be a reference to your SLP or it may be specific to the particular software application and the planning of it included in here.
So, the next part is the testing. Once the plan is complete, in a traditional validation process, the specific protocols prescribed are written and approved prior to execution.
These include the IQ, OQ, and PQ. The IQ typically verifies software was installed and/or configured properly. It may also include things like checks for compatibility with different platforms, confirmation of backup processes, et cetera.
The OQ verifies the software supports your process throughout all operating ranges. When it comes to configurable software like Arena, that is likely validating all out-of-box functionality. Finally, the PQ is where the software is verified to meet performance in its final environment with users or individuals simulating those users. The PQ will validate the system in its as-used configured state by users. The PQ typically can only be done by the manufacturer according to their intended use as expressed in their custom configuration. Manufacturers can hire others to perform this validation for them so long as they can simulate their use in the final configured production-equivalent state.
And you’ll notice that I have the IQ/OQ squared off in green and the PQ squared off in blue. And for a configurable software such as Arena, typically the green box area are the ones that would be completed by the supplier and the blue box would be the responsibility of the manufacturer. We’ll discuss that a little bit more later.
Okay, to complete your validation package, typically you would include a Traceability Matrix. And this Traceability Matrix will provide evidence that your validation is complete. You will trace your risks to the user requirements to make sure that all risks are addressed. You will trace your user requirements to a test case to make sure that all requirements have been addressed. So in the end, the idea here is just to make sure that every requirement has been addressed by testing appropriately and you’re tracing your risk to the user requirement so that there is an understanding of the level of intensity given to each test case for a particular requirement. For instance, if the risk is low risk or minimal, your testing for that may be minimal. It may be only included in your OQ and not included in your PQ, or perhaps you do not include negative testing for something that’s lower risk.
And then the final step to your Master Validation Package is typically a Master Validation Report. In addition to including a summary of all of your test reports and/or test results, you would include the final plan for tracking changes to that software, how you will monitor that software, and when changes do occur, how will you determine the impact of those changes on the software itself and on the risk that will feed into re-validation requirements. So your report should indicate when will we be looking at the software and determining if re-validation is required. That could be because of a software change. It could be because of a change to your configuration. It could be because of a change to your intended use or your requirements. So all three of those things could have an impact in addition to some others.
This is a typical V model for non-product software. It shows that the intended use and user requirements are typically verified through PQ and that functional specifications and system design is verified through IQ/OQ. The green boxes for a purchased configurable software application are typically the responsibility of the supplier while the blue box shows that a manufacturer is typically responsible for that activity.Rarely you may find a software application where the supplier can do most or all, but it is unlikely that there is not at least some user permissions or some level of validation that should be done by the manufacturer.
However, the good news we have to share with you today is that in coordination with MEDIcept, Arena Validate now includes PQ phase templates, which will get you most of the way there with much less effort.
I will discuss this further later in the presentation, but for now I’d like to turn this over to Emily to start our discussion on the latest Arena Validate package.
Emily Foree
Thank you, Marion. Now I’m going to go ahead and go over an overview of Arena Validate.
So what does Arena Validate do? Really, Arena Validate performs the majority of software validation efforts needed for our customers to achieve and maintain their regulatory compliance. On the left, I’ve listed out some bullet points which outline all of the different types of documents that we offer, as well as some additional resources such as those PQ templates that Marion mentioned, which we’ll go over in a bit more detail later.
Some of the benefits of Arena Validate are that you can really have confidence that Arena is validated to meet your regulatory standards and minimize the risk of noncompliance. We enable you to benefit from our continual innovation and software solution enhancements without fearing that long, costly, or difficult validation process.You can also rest assured knowing that you will receive support with every software release of Arena, which allows you to re-validate your Arena system with minimal disruption.
We also have subject matter experts, this partnership with MEDIcept, and additional resources and support. In addition to those PQ templates, we also offer some additional resources like tech notes, which address 21 CFR part 11 and other regulations in detail. Our experienced team is really here to guide you through the process, answer your questions, connect you to our subject matter experts with MEDIcept, and provide you with these additional resources.
A little bit on our approach to system validation. On the left, you’ll see some of those common regulations that Marion mentioned before. These are all saying that if you’re using Arena for DMR, DHF, other regulatory processes, that Arena needs to be validated. And since Arena is off the shelf, it’s configurable, but it’s not customizable. This has allowed us to develop a validation method that is able to suit a majority of our customers’ needs. So we are able to test a wide variety of configurations to really get you most of the way there in meeting your compliance requirements. Now, as Marion was mentioning, there are always some activities that are needed on the customer’s end to ensure that you are in compliance, and part of that is assessing the adequacy of our activities to establish that Arena is validated for your intended uses.
This slide is showing our Arena software development lifecycle process as well as the validation process. So the SDLC is in blue and what you can see is the validation process, which is in orange. These two things are very much so happening in parallel. So we work very closely with our engineering team, and what I’d really like to point out here is that validation approval is one of the last steps before a release of Arena goes live. So we are very involved in the release process at Arena. So how do we determine what requires validation? Should be addressing this with the next few slides.
But really, based on Arena’s QMS, we have essentially two different types of releases. We have major and update releases as well as minor releases. Major and update releases include enhancements to existing functionality or they introduce new functionality. These releases typically do have validation impact, and so for each of these major or update releases, our team performs a thorough risk analysis to determine the impact to previously validated functionality, and to allow us to plan our validation activities for that release. Once all of our activities such as IQ and OQ are completed successfully, then the validation team approves the major or update release, and that results in a package of documentation and evidence that is provided to you to show that that release of Arena is maintained in a validated state.
Now for minor releases, minor releases by definition do not have validation impact. However, I like to point out that the validation team is still very involved in the minor release process. So our team reviews every single fix that is included in a minor release and we confirm that there is no impact on the validated state. Occasionally, this might even include us performing some regression testing to ensure that there isn’t even any regression introduced. Once no impact is confirmed, then our team approves each of the fixes associated with the release and that release can then go live.
This slide is really demonstrating Arena Validate’s growth over time. So the bar graphed here, you’ll see that each bar represents the number of unique user requirements that Arena Validate covers and maintains in a validated state. At the bottom is each year, so in 2008 we covered 86 unique user requirements. In 2023, today, we cover over 1,600. So this growth is expanding every single release of Arena. You can imagine that with over 1,600 unique user requirements, we are supporting a wide variety of configurations, and really the majority of our customers leverage this OQ coverage to meet their PQ needs.
There’s a number of different factors that drive this growth. This could be just upgrades to existing validated functionality, internal and external feedback. So if there’s an area of the application that we do not currently include in our scope, but you think that it’s applicable to most regulated customers, then please reach out and let us know. We are very open to receiving feedback. And then also just adoption of new Arena products. So if a new feature or new product is released, we might wait for adoption to increase and for that new product to mature, and then we will quite likely include that in our scope moving forward.
This is a high-level overview of the validation scope. So these areas are parts of the application where we cover end-to-end validation testing. Now, there are some exclusions, and Marion will touch on this a bit later. Typically, areas that are excluded from our scope involve integrations, import and export.
A bit of background on this: Arena Validate performs our validation testing in a production-like environment with production-like data. So anytime that you are moving specific data into or out of Arena, and you need to be able to verify that that data has been transferred successfully and accurately, that might be an area that, depending on your risk analysis, would require some additional PQ testing.
Then a little bit about our deliverable schedule. So this is when you can expect to receive certain types of documents as they relate to a specific Arena release. So approximately four weeks prior to go-live is when you’ll receive our updated user requirement specification documents as well as an impact analysis for that upcoming release. The impact analysis contains detailed release notes about every single change that is happening in the Arena application as well as an assessment of risk with regards to Arena validation. We also include some additional information that might be of particular importance to admins within regulated spaces, but the impact analysis which is shared about four weeks prior to go-live contains a lot of important information and many of our customers leverage this document within their own validation process.
One to two business days prior to go-live is when you’ll see the majority of our validation testing evidence available to you. So that will include all of the trace matrices, the IQ section one, and OQ execution records. The software validation test plan, which defines in detail what exactly is being tested for that particular release, as well as a report which summarizes all of the activities that Arena Validate has taken with regards to this release. And finally, one to two business days post go-live is when you will get an updated software validation report because it has been updated to include results from IQ section two. IQ section two is performed immediately following go-live on the production environment to ensure a successful deployment. And then we also include a certificate, which is a high-level list of released documents and the signatories who have signed off on those documents.
So in addition to that validation package, which is generated for each software release, we have some additional validation resources. So these are shared under this item 05A-0008 in the Arena Corporate Workspace. All of our Validate customers have access to this item and its files. I’ve decided to just highlight some of the tech notes that are included as well as the new PQ templates.
So we have a number of just different additional resources here that I encourage everyone to take a look at. And with that, I think that I can hand it back to Marion to discuss PQ templates in more detail.
Marion Cappadona
I wanted to discuss today the new Arena PTC PQ phase templates, what they include, and how you can use them.
These templates were produced by MEDIcept in collaboration with Arena PTC. We base these templates on our expansive experience working with our clients to complete their Arena PQ validations. It is our intent and hope that these templates will get you most of the way there. But of course, if you still need help executing these templates, MEDIcept is available to give you all the assistance that you need.
Okay, so what do the templates include? The PQ phase templates include a Master Validation Plan template, a PQ test record template, a quality workflow validation template, and a Master Validation Report. In addition, Arena PTC has provided you with some other documents that may be helpful as part of your validation package, including the part 11 e- signature acknowledgement agreement form and some Arena tech notes that will help you in the areas such as part 11 compliance.
The first template in the package is the Master Validation Plan template. This plan includes sections to help you define the following. Your intended use and scope, that’s basically you’re saying: What are we using within Arena? What’s the scope of what we’re using? Which worlds are we going to be using and which worlds will we make inactive or not use? For the worlds that we include in scope, you will define software requirements. The template gives you a framework format for describing your software requirements. It gives you some examples, but it will be up to you and the template to expand that to include your specific requirements. It has a section that refers to the IQ and OQ to bolster your complete validation package, and there’s a section on risk assessment to identify risks associated with the use of it.
The next part of the MVP, Master Validation Plan, that is very helpful is providing you with a framework for the new CSA test strategy that FDA has described in their latest guidance document.
And this is a test strategy that you can use to minimize the documentation and the effort that’s provided. And I will say this, arena’s IQ/OQ certainly provides you with your best option and documentation to reduce what you need to do in the PQ. The PQ is important for you to test the system in the way that you use it and the way you configure it with your users in your environment with the computer systems and browsers that you use.
But this CSA test strategy can be used to really reduce the effort it allows for test script writing on the fly instead of having to write pre-approved test protocols if you can justify a low risk to the world that you use and your intended uses. So that provides you with a framework to do that, and MEDIcept is available to help you with writing that justification anytime that you need it. There’s also a section on configuration management, and I wanted to discuss this a little bit because, as I mentioned previously, configuration spec is something that is often forgotten. One of the things that you’ll need to do is make sure that you have defined the software’s configuration that you’re validating. Otherwise you really don’t have evidence that you’ve validated the configuration that you’re using.
There’s also a section on validation requirements, so based on risk assessment in your CSA test strategy, you’ll define your validation requirements. There is a section on re-validation planning. The test records and test cases that will be required, it doesn’t specify the test cases in this particular template, but you will identify which test records will be needed by world. It includes a section on part 11 compliance, how will you handle deviations, acceptance criteria in general, and then test reporting.
The next template is the PQ test record. Now, as I mentioned, based on the CSA test approach, this may be the template you use to create your test protocol with pre-written prescribed test scripts that are pre-approved before you execute them. However, if you have a low-risk test strategy based on risk, you can use this as your test record and write your test scripts on the fly as you execute them.
That would then not require any pre-approval, and the only pre-approval you would have would be for the Master Validation Plan. Typically, you would have a PQ test record for each world; however, that depends on your workflows that you particularly prescribe.
In your test record, you’re going to include a subset of the requirements that are stipulated in your Master Validation Plan that apply to the particular PQ test record or Arena World that you’re validating. In the screenshot, it’s showing you the format that’s in the template for the test cases themselves, where you would typically put down the steps that you write in your test script, the expected results, and then a place for you to record the actual results and reference your screenshots as well as the actual result as a pass or fail. How specific you are in your test script writing is really dependent upon your CSA test approach. Low risk, you may just give in English words how you’re going to test it and then give the specifics and the result.
If you are doing a pre-approved protocol according to a high-risk approach, you would likely have your step-by-step button clicks in your test scripts. You’ll have an attachment to your test record, which would be all of your screenshots. And depending on the CSA test approach, you can minimize the number of screenshots that you really need. You’ll identify any applicable deviations in your PQ test record. And again, as far as screenshots are concerned, you minimally will always want to have screenshots at least for your deviations. And then finally, you’ll have a place for the testers to sign and for reviewers to sign, and that is included in the PQ test record.
The next template is what we call the Quality Workflow Validation Test Record. One of the things that we realized in our experience in validating Arena is that the Quality World workflows basically all have the same or similar fields in them. So instead of going through a complete PQ test record for every single workflow that you create, you may create one workflow that you have identified as the most complicated workflow with all of the different types of fields that you will use in all of your workflows, and use that one in a PQ test record to validate the Quality World. Once you validate the Quality World, you can use this template and leverage that validation to validate each add-on workflow that is the same or less in complexity.
So the idea here with this template is first you’re going to verify that the configuration of that workflow matches what you have in your specification and everything is configured as expected. And then you’ll do a few test steps, but you don’t need to go through the entire workflow to validate the Quality World in that case. So that’s an approach that we’ve taken and we found that it works and helps to make this process a lot more efficient. So we’re providing this template in here thinking that you may be able to use the same for your process.
The final template that we include in the validation package is the Master Validation Report Template. Now, if you’re using a high-risk traditional validation approach, you may choose to write PQ test reports for each PQ test protocol written and then have a final Master Validation Report. For a lower-risk CSA test approach strategy, your completed PQ test record would just be attached to the Master Validation Report, and the report itself would provide a conclusion and summarize the results of all of those PQ test records. It would also provide a table of deviations which summarizes all of the deviations from all of the PQ test records and the resolution that you have given for those deviations. And it will also provide the final re-validation maintenance plan that you will follow through. As I mentioned, the MVR will include attachments of the executed test protocols and the screenshots for those.
And as far as re-validation is concerned, you’re going to want to include in there when do we need to go back to a Master Validation Plan and revise that and repeat the PQ test records, or can we just modify the Master Validation Report or write another Master Validation Report to summarize the re-validation test results, and your change impact assessments should indicate when it is appropriate to do one versus the other.
Finally, I just wanted to talk to you a little bit about the services that we offer to support your validation needs. We hope that the PQ phase validation templates will really help you do most of the work on your own, but if you need some help with those, we’re here to help. And there are some other areas that are not included in the IQ/OQ scope that we can also help you with. So in general, Arena will provide you with that baseline setup and they will provide you with a nearly complete validation package with the IQ/OQs. They will help you set up your integrations initially and they will provide you with training. I believe they have a training program that they offer. And they also provide data migration services for larger migrations. And of course, they have their maintenance process for releases and ongoing bug fixes that Emily had already discussed with you.
And of course, Arena Validate includes the IQ/OQ validation package, as well as the PQ templates. MEDIcept, on the other hand, will provide you with validation assistance. If you need further assistance with those PQ validation templates, we can help you with that. We can help you develop your requirements and we can even execute the validations for you if that’s what’s needed, and we would just become one of your users on your workspaces.
Typically, a validation workspace that is a duplicate of production would be used. We can help you with re-validation and maintenance. So if you receive the change impact assessment from Arena and you want to be able to document how that impacts your validated state with respect to your PQs in your configuration, MEDIcept can help you with that documentation to help you justify either no validation, no re-validation hopefully, or what level of re-validation may be needed.
In addition, you may decide at some point that you want to integrate your Arena workspace with other programs such as NetSuite or SAP, and a bridge needs to be created for that. Typically, we would point you back to Arena to have the bridge set up, and then they will let the partners that they have that are most appropriate to program that bridge. But once that’s done, the integration itself needs to be validated as well to confirm that the data is accurately pushed or pulled to and from Arena.
Another area that you may want to consider is data migration. Maybe this isn’t the first system that you have put in place and you have a legacy system that has plenty of data in it. If you’re doing a small migration, MEDIcept may be able to help you with that migration using the Arena PLM tools. However, if it’s a larger migration, such as probably more than 2,000 records, we would point you back to Arena to have them help you with a large-scale migration, and then MEDIcept can provide you with support in validating that migration that the data was completely and accurately transmitted into Arena.
Regarding configuration, when you do your validation package, as I mentioned, you should have a configuration spec to indicate the configuration that you’re actually validating in your PQ. And MEDIcept can help you review your configuration spec or even create one if you don’t have one yet. And finally, another service that MEDIcept can provide you with is an embedded Arena PLM administrator. If you’re finding that you have changed personnel or you need some help in getting through in the beginning stages, we can provide you with individuals that have experience with Arena that can help you on the job training and working with you once you’ve completed your training and setup and validation of Arena PLM.
And that’s all that I have today. I hope that was helpful, and I appreciate Emily and Arena and all the help that they’ve given us and the opportunity for us to speak to you today. Thank you. And now I think we would like to open it up for a Q&A, and we welcome any questions that you may have.